Skip to content

Commit

Permalink
some reorganisation
Browse files Browse the repository at this point in the history
  • Loading branch information
xFFFFF committed Apr 30, 2018
1 parent 283eaff commit b9071cd
Show file tree
Hide file tree
Showing 5 changed files with 96 additions and 86 deletions.
53 changes: 53 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Gekko BacktestTool
![Logo](https://i.imgur.com/G3Dcv7i.png)

## Change Log
### v0.6
- after completion of the backtests, the analysis module displays three tables with data: *ALL RESULTS*, *TOP STRATEGIES*, *TOP DATASET*
- parameter `--analyze mycsvfile.csv` for analyze any results from BacktestTool's external file
- searching optimal strategy parameters by brute force method with syntax `start..end:step` or `value1, value2, value3` in strat's toml file
- value `ALL` for exchanges and strategies for backtest machine - do backtests on all available exchanges, currencies, assets or strategies. Based on filenames in history and strategies directories.
- add binaries for Linux and FreeBSD to [releases](https://github.com/xFFFFF/Gekko-BacktestTool/releases)
- backtest-config.pl updated
- README.MD updated

### v0.5
- price: *open*, *close*, *high*, *low*, *average* for dataset period in CSV output
- [coinmarketcap.com](https://coinmarketcap.com) data in CSV output: *current marketcap*, *current rank*, *last 24h global volume*
- CSV's template - now You can choose which columns will be add to CSV file
- temporary GBT's files are in tmp directory now.
- MS Windows compatibility fix (tested on W7x64 and StrawberryPerl)
- backtest-config.pl file updated

### v0.4
- price *volality* (based on relative standard deviation) in CSV output
- sum of *volume* and *volume/day* for dataset period in CSV output
- sum of *overall exchange trades* and sum of *overall trades/day* for dataset period in CSV output
- if pair not exist in DB on parameter `-f last` then create table and import from last 24 hours.
- update parameter `--help`
- README update
- some code clean
- some fixes

### v0.3
- Gekko BacktestTool external config file support. Default config file is backtest-config.pl, but You can create own and use `backtest.pl --config BACKTESTTOOL_CONFIG_FILENAME parameter`
- using TOML files for strategies configuration as default. Can be changed in backtest-config.pl
- *percentage wins, best win, worst loss, median win, median lost, average exposed* duration added to csv file.
- parameter `--covert TOML_FILE`. Convert toml file and print strategy settings in Gekko's config file format
- parameter `-p exchange:ALL` and `-p exchange:asset:ALL` for backtest pairs. Do backtest for all available pairs. Based on non empty tables from sqlite database.
- parameter `-p exchange:ALL` and `-p exchange:asset:ALL` for import pairs. Import all available pairs from exchange. Based on exchange/exchange-markets.json file
- parameter `--import --from=last --to=now`. The `last` value checks in the database the time of the last candle for each pair and assign this value to `--from`. `now` assign current time in GMT time zone. In short: with this command you can import from the last candle from datasets to current time.

### v0.2
- *winning/losses* trades in csv file
- command line parameters support (`--import (-i)`, `--paper (-g)`, `--strat (-s)`, `--pair (-p)`, `--candles (-n)`, `--output (-o)` `--from (-f)`, `--to (-t)`, `--help (-h)`)
- showing roundtrips in terminal output (can be disabled in configuration)
- bugfixes/code clean

### v0.1
- multiple datasets import (`perl backtest.pl -i`)
- start multiple paperTraders in background (`perl backtest.pl -p`) (need improvement)
- support for multiple CandleSize and CandleHistory
- logs is moved to logs directory
- performance improvement
- bugfixes
108 changes: 22 additions & 86 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,20 @@
# Gekko BacktestTool
![Logo](https://i.imgur.com/7rSHT2u.png)
# Gekko BacktestTool
![Logo](https://i.imgur.com/G3Dcv7i.png)
[![Donate with Bitcoin](https://en.cryptobadges.io/badge/small/32G2cYTNFJ8heKUbALWSgGvYQikyJ9dHZp)](https://en.cryptobadges.io/donate/32G2cYTNFJ8heKUbALWSgGvYQikyJ9dHZp)
[![Donate with Ethereum](https://en.cryptobadges.io/badge/small/0x50b7611b6dC8a4073cB4eF12A6b045f644c3a3Aa)](https://en.cryptobadges.io/donate/0x50b7611b6dC8a4073cB4eF12A6b045f644c3a3Aa)
![badge](https://img.shields.io/github/release/xFFFFF/Gekko-BacktestTool.svg) ![GitHub Release Date](https://img.shields.io/github/release-date/xFFFFF/Gekko-BacktestTool.svg) ![badge](https://img.shields.io/github/downloads/xFFFFF/Gekko-BacktestTool/total.svg) ![GitHub last commit](https://img.shields.io/github/last-commit/xFFFFF/Gekko-BacktestTool.svg) ![badge](https://img.shields.io/github/license/mashape/apistatus.svg) ![badge](https://img.shields.io/github/languages/code-size/xFFFFF/Gekko-BacktestTool.svg) ![GitHub closed issues](https://img.shields.io/github/issues-closed/xFFFFF/Gekko-BacktestTool.svg)

CLI tool that enhances the features of [Gekko's Trading Bot](https://github.com/askmike/gekko). The tool performs a test with multiple pairs on a single run. Suppose you have a strategy that you want to test on more currency pairs. You enter all the pairs on which you want to test the strategy for the BacktestTool's configuration file. You start the application and everything happens automatically. You are just waiting for the results that appear on the screen. You will see how your strategy falls on other pairs, where it works the best, and where the worst. More detailed data is available in the .CSV file, which you can open in a spreadsheet or text editor.

CLI tool that enhances the features of [Gekko's Trading Bot](https://github.com/askmike/gekko). The tool performs a test with multiple pairs on a single run. Suppose you have a strategy that you want to test on more currency pairs. You enter all the pairs on which you want to test the strategy for the BacktestTool's configuration file. You start the application and everything happens automatically. You are just waiting for the results that appear on the screen. You will see how your strategy falls on other pairs, where it works the best, and where the worst. More detailed data is available in the .CSV file, which you can open in a spreadsheet or text editor.
![Top Dataset](http://i.imgur.com/U7TCuSn.png)

You can do the same with many strategies and CandleSize values. You can test all your strategies on eg BTC-USD pair and compare results, which will allow you to choose the best strategy you will use in live trade.

# DEMO
Backtest machine
![Alt text](images/backtest.gif?raw=true "GekkoBacktestTool running demo")
## DEMO
### Backtest machine
![GBT run](https://i.imgur.com/x1Ryyog.gif)

Database file
![Alt text](images/csv.gif?raw=true "GekkoBacktestTool CSV file demo")
### Database file
![CSV file](https://i.imgur.com/ENj1ZDM.gif)

# Features
## Features
- **Backtest** for multiple strategies and pairs with one command
- **Backtests results** are exporting to CSV file [(see sample)](https://github.com/xFFFFF/Gekko-BacktestTool/blob/master/sample_output.csv)
- **Analysis** and comparing all strategy and pair results in variables such as: *% of profitable backtests for strategy*, *% of results with profit above market*, *% of win trades*, *average P&L for trades* and more!
Expand All @@ -27,32 +24,32 @@ Database file
- **Multithreading** - in contrast to raw Gekko backtest this tool can uses 100% of your processor
- **Extended statistics** - 40 variables from single backtest result, such as: *volume*, *price (min, max, avg, volality)*, *percent of win trades*, *median P&L for trades*, *marketcap*, *CoinMarketCap Rank*, etc.

![Top Strats](http://i.imgur.com/JXjKVDT.png)
<img align="center" src=http://i.imgur.com/JXjKVDT.png>

# Minimal requirements
## Minimal requirements
- [Gekko Trading Bot](https://github.com/askmike/gekko)
- [Binaries of BacktestTool](https://github.com/xFFFFF/Gekko-BacktestTool/releases)

# Installation
**"Binaries": Easiest install way**
## Installation
### Binaries: Easiest install way
1. Download latest version from repository's [releases](https://github.com/xFFFFF/Gekko-BacktestTool/releases).
2. Extract downloaded zip.
3. Copy all extrated files to main Gekko's directory.

**Open Source: Debian, Ubuntu, Linux Mint**
### Open Source: Debian, Ubuntu, Linux Mint
1. Clone git https://github.com/xFFFFF/Gekko-BacktestTool
2. Copy files to Gekko's main directory
3. Install dependies:
`$ sudo cpan install Parallel::ForkManager Time::ParseDate Time::Elapsed Getopt::Long List::MoreUtils File::chdir Statistics::Basic DBI DBD::SQLite JSON::XS TOML File::Basename File::Find::Wanted Template LWP::UserAgent LWP::Protocol::https Set::CrossProduct DBD::CSV Text::Table File::Copy`

**Open Source: Other Unix-like OS**
### Open Source: Other Unix-like OS
1. Clone git https://github.com/xFFFFF/Gekko-BacktestTool
2. Copy files to Gekko's main directory
3. Install dependies:
`$ su`
`$ cpan install Parallel::ForkManager Time::ParseDate Time::Elapsed Getopt::Long List::MoreUtils File::chdir Statistics::Basic DBI DBD::SQLite JSON::XS TOML File::Basename File::Find::Wanted Template LWP::UserAgent LWP::Protocol::https Set::CrossProduct DBD::CSV Text::Table File::Copy`

**MS Windows**
### MS Windows
1. Install [Strawberry Perl](http://strawberryperl.com/)
2. Download Gekko BacktestTool from [here](https://github.com/xFFFFF/Gekko-BacktestTool/archive/master.zip)
3. Uncompress files from master.zip to Your main Gekko's folder
Expand All @@ -61,7 +58,7 @@ Database file
6. In appeared Window with black background enter command:
`cpan install Parallel::ForkManager Time::ParseDate Time::Elapsed Getopt::Long List::MoreUtils File::chdir Statistics::Basic DBI DBD::SQLite JSON::XS TOML File::Basename File::Find::Wanted Template LWP::UserAgent LWP::Protocol::https Set::CrossProduct DBD::CSV Text::Table File::Copy`

# Run
## Run
1. Edit backtest-config.pl in text editor.
2. In terminal/cmd go to Your main Gekko's folder ex:
Windows - `cd C:\Users\xFFFFF\Desktop\gekko`
Expand All @@ -71,7 +68,7 @@ Linux - `cd /home/xFFFFF/gekko`
a) Open Source version: `perl backtest.pl`
b) Binaries: `./backtest`

**All available commands**
### All available commands
```
usage: perl backtest.pl
To run backtests machine
Expand All @@ -97,9 +94,9 @@ Optional parameters:
-t now - 'now' is current time in GMT.
-o, --output FILENAME - CSV file name.
```
![Run](http://i.imgur.com/OY14rKb.png)
<img align="center" src=http://i.imgur.com/OY14rKb.png>

# Some examples
### Some examples
- **B**acktests of all available pairs for Binance Exchange in Gekko's scan datarange mode:
`$ perl backtest.pl -p binance:ALL`

Expand All @@ -114,70 +111,9 @@ Optional parameters:

- **S**earch best parameters for strategy: edit TOML file in config/strategies
![Strat config example](http://i.imgur.com/OkGPQSm.png)
The above example will generate 15 backtests with unique configurations. Syntax for brute force is: start..end: step (as in the case of TimePeriod) or value1, value2, value3 (example from interval). The generated values for TimePeriod are 15, 20, 25. After saving the file, run the backtest of the given strategy, eg backtest.pl -s BBRSI.

# ToDo
- add filters to analyser (avg trade day, profit, hodl, avg trades win) and limit for printed results
- 3rd sort value in analyser
- parameter `--info` for print data like strats names, available datasets etc
- printing Gekko's output without buffering
- more descriptive and readable cmd output (text bold?)
- Import sqlite file dumps (full history)
- GUI

The above example will generate 15 backtests with unique configurations. Syntax for brute force is: start..end: step (as in the case of TimePeriod) or value1, value2, value3 (example from interval). The generated values for TimePeriod are 15, 20, 25. After saving the file, run the backtest of the given strategy, eg backtest.pl -s BBRSI.
![Brute force](http://i.imgur.com/gnywgrA.png)

# Change Log
v0.6
- after completion of the backtests, the analysis module displays three tables with data: *ALL RESULTS*, *TOP STRATEGIES*, *TOP DATASET*
- parameter `--analyze mycsvfile.csv` for analyze any results from BacktestTool's external file
- searching optimal strategy parameters by brute force method with syntax `start..end:step` or `value1, value2, value3` in strat's toml file
- value `ALL` for exchanges and strategies for backtest machine - do backtests on all available exchanges, currencies, assets or strategies. Based on filenames in history and strategies directories.
- add binaries for Linux and FreeBSD to [releases](https://github.com/xFFFFF/Gekko-BacktestTool/releases)
- backtest-config.pl updated
- README.MD updated

v0.5
- price: *open*, *close*, *high*, *low*, *average* for dataset period in CSV output
- [coinmarketcap.com](https://coinmarketcap.com) data in CSV output: *current marketcap*, *current rank*, *last 24h global volume*
- CSV's template - now You can choose which columns will be add to CSV file
- temporary GBT's files are in tmp directory now.
- MS Windows compatibility fix (tested on W7x64 and StrawberryPerl)
- backtest-config.pl file updated

v0.4
- price *volality* (based on relative standard deviation) in CSV output
- sum of *volume* and *volume/day* for dataset period in CSV output
- sum of *overall exchange trades* and sum of *overall trades/day* for dataset period in CSV output
- if pair not exist in DB on parameter `-f last` then create table and import from last 24 hours.
- update parameter `--help`
- README update
- some code clean
- some fixes

v0.3
- Gekko BacktestTool external config file support. Default config file is backtest-config.pl, but You can create own and use `backtest.pl --config BACKTESTTOOL_CONFIG_FILENAME parameter`
- using TOML files for strategies configuration as default. Can be changed in backtest-config.pl
- *percentage wins, best win, worst loss, median win, median lost, average exposed* duration added to csv file.
- parameter `--covert TOML_FILE`. Convert toml file and print strategy settings in Gekko's config file format
- parameter `-p exchange:ALL` and `-p exchange:asset:ALL` for backtest pairs. Do backtest for all available pairs. Based on non empty tables from sqlite database.
- parameter `-p exchange:ALL` and `-p exchange:asset:ALL` for import pairs. Import all available pairs from exchange. Based on exchange/exchange-markets.json file
- parameter `--import --from=last --to=now`. The `last` value checks in the database the time of the last candle for each pair and assign this value to `--from`. `now` assign current time in GMT time zone. In short: with this command you can import from the last candle from datasets to current time.

v0.2
- *winning/losses* trades in csv file
- command line parameters support (`--import (-i)`, `--paper (-g)`, `--strat (-s)`, `--pair (-p)`, `--candles (-n)`, `--output (-o)` `--from (-f)`, `--to (-t)`, `--help (-h)`)
- showing roundtrips in terminal output (can be disabled in configuration)
- bugfixes/code clean

v0.1
- multiple datasets import (`perl backtest.pl -i`)
- start multiple paperTraders in background (`perl backtest.pl -p`) (need improvement)
- support for multiple CandleSize and CandleHistory
- logs is moved to logs directory
- performance improvement
- bugfixes

# See also
## See also
- [Gekko's Datasets](https://github.com/xFFFFF/Gekko-Datasets)
- [Gekko's Strategies](https://github.com/xFFFFF/Gekko-Strategies)
21 changes: 21 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Gekko BacktestTool
![Logo](https://i.imgur.com/G3Dcv7i.png)
## TO DO
- [ ] add filters to analyser (avg trade day, profit, hodl, avg trades win) and limit for printed results
- [ ] 3rd sort value in analyser
- [ ] parameter `--info` for print data like strats names (available TOML, required indicators), available datasets (scan period (first find start and and and then scan), info about volality, volume, trades, data from CMC, CC maybe, amount of official tweets)
- [ ] printing Gekko's output without buffering
- [ ] more descriptive and readable output (do text bold work on Windows?)
- [ ] Import sqlite file dumps from Google Drive
- [ ] Split, merge, replace SQLite files
- [ ] GUI
- [ ] CryptoCompare candles import
- [ ] counter for performed backtests amount, sum of backtested days, trades
- [ ] option: save all results in DB and check do specified backtest was performed in past
- [ ] fetch historical marketcap in backgroud or on command
- [ ] add required by analyser columns to template of CSV file
- [ ] option: use TOML or JSON parameters config in CSV
- [ ] some options for template of LAST RESULTS table columns
- [ ] align BF parameters from strat name in columns
- [ ] value profit/trades in CSV and analyser
- [ ] compatilibity with WinBash
Binary file removed images/backtest.gif
Binary file not shown.
Binary file removed images/csv.gif
Binary file not shown.

0 comments on commit b9071cd

Please sign in to comment.