Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proof of RL viability for trading with data leakage #1248

Open
e-gluzman opened this issue Jun 23, 2024 · 2 comments
Open

Proof of RL viability for trading with data leakage #1248

e-gluzman opened this issue Jun 23, 2024 · 2 comments
Assignees
Labels
good first issue Good for newcomers

Comments

@e-gluzman
Copy link

Hi guys,

Your FinRL project has been very helpful - I have been using the StockTradingEnv to make sure I do not mess up my environment.

However, I am encountering very low performance with RL algorithms. In order to test if the RL models are working properly I have created features that leak data about the future returns in the next 1,2,3,5 days. In theory, this should make the task very easy - if future returns are low, sell. However, the model is not able to learn any strategy other than buy and hold.

To replicate:

Do you know why the standard RL algorithm is failing even when given future information? Could you show a notebook where it is able to outperform a buy and hold strategy on a stock, while using information from the future?

Thank you,
Evgeny.

Contact: [email protected]

@BruceYanghy BruceYanghy self-assigned this Jun 23, 2024
@BruceYanghy BruceYanghy added the good first issue Good for newcomers label Jun 23, 2024
@zhumingpassional
Copy link
Collaborator

zhumingpassional commented Jun 24, 2024

@e-gluzman

The stability of RL algorithms is an important issue. Pls use some tricks such as ensemble strategy, dynamic datasets.

Setting the tick list to single tick is not a good method. you can use multiple ticks, and after training only use the action of the single tick.

We will recruit a research assistant to maintain this project.

@BruceYanghy
Copy link
Member

Hi guys,

Your FinRL project has been very helpful - I have been using the StockTradingEnv to make sure I do not mess up my environment.

However, I am encountering very low performance with RL algorithms. In order to test if the RL models are working properly I have created features that leak data about the future returns in the next 1,2,3,5 days. In theory, this should make the task very easy - if future returns are low, sell. However, the model is not able to learn any strategy other than buy and hold.

To replicate:

Do you know why the standard RL algorithm is failing even when given future information? Could you show a notebook where it is able to outperform a buy and hold strategy on a stock, while using information from the future?

Thank you, Evgeny.

Contact: [email protected]

Thank you for bringing up the issue. Currently, the FinRL library is extremely poorly maintained. Rest assured, I will reorganize a team to ensure its proper maintenance.

Best regards,

Bruce Yang

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

3 participants