Strategies falling off when saved as .csv/.xlsx



Hi


A significant number of strategies fall off (often) when saved as .csv/.xlsx. One common denominator here seems to be that the higher the fitness, the more likely they are to fall off (yes, very weird). This is a problem in itself, as you are more likely interested in the strategies with the highest fitness than vice versa.
 
Anyway, please see the attached screenshots. Should be self-explanatory from there (see underlined strategy - fell off in .csv). I did send you log files earlier. Please contact me directly if you need anything else. 


Thanks.

Attachments
saving error SQ 2.jpg
(120.25 KiB)
saving error SQ 1.jpg
(669.76 KiB)
  • Votes +1
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

AE
#1

Phronesis

20.05.2019 23:19

Task created

AE
#2

Phronesis

02.06.2019 21:57

Attachment SQ bug empty strategies.jpg added


I realize now that the issue is not in the transfer of data to .csv/.xlsx. The problem is that the strategies are "empty" of any information. 


What I mean with this is that the strategies exist, with metrics etc, in the "reults GUI", But when you try to access them (by clicking etc), they contain no information. 


See the enclosed screenshot for clarification. A strategy, with trades/metrics etc is highlighted. But the summary above is showing noting. 


Hope that helps. Thanks.

AE
#3

Phronesis

02.06.2019 21:57
Voted for this task.
AE
#4

Phronesis

03.06.2019 10:09

Attachment EMPTY Strategy 4461324.sqx added

Enclosing an "empty" strategy:
TT
#5

Tamas

03.06.2019 15:02

Status changed from New to In progress

Does it happen in Retester while the task is running ? 


If yes, you have to wait until retest is finished and then save the strategies.

TT
#6

Tamas

03.06.2019 15:03

Status changed from In progress to Waiting for information

AE
#7

Phronesis

03.06.2019 16:00
Nope. 

This has nothing to do with the Retester (althought I can't say that it might happen there as well). 

It happens all in the Builder. 

Thanks.

TT
#8

Tamas

03.06.2019 16:08
If it happens in builder, can you please attach the config file from here ? \user\projects\Build\project.cfx
AE
#9

Phronesis

03.06.2019 16:11

Attachment project.cfx added

Done
TT
#10

Tamas

05.06.2019 15:32
I am still not able to reproduce the bug.


Can you create some printscreens where we can see the whole SQ screen with builder and opened databank + exported file ? 


It can happen in Retester but I have no idea why it does not work in Builder.

AE
#11

Phronesis

05.06.2019 23:50

What do you mean with “opened data bank” and “exported file”?

 

Would it perhaps be easier/quicker if emailed me directly to iron this out? There may be some dialogue required here. We could also, for example, do a remote control if that helps?

 

Thanks.

AE
#12

Phronesis

07.06.2019 22:08

Attachment SQ error strategy.jpg added

Attachment Strategy 1301048.sqx added

Hi. Here is another example of a strategy that looks weird (related to the same issue?).


I have a theory why this could be happening. I could, of course,  be completely wrong, but it seems like it could have something to do with strategies that are not filtered out correctly. There are a couple of reasons why I think this could be the case. I just ran a number of strategies, where the results tagged a large portion of them with "warnings" such as "too many trades on the same bar", despite my automatic filer being switched on to filter out these trades. For this specific run, there were no empty strategies (which is unusual). For my next run with similar settings, there were no "warnings", but lots of empty strategies. So could it be that in some cases (for whatever reason), the filter fails, but that the strategies somehow come through as an "empty" strategies instead? Also, the average fitness for the empty strategies are typically higher than for the sample, which is symptomatic for strategies with warnings (=they are too good to be true).


Just a thought. Let me know if you need anything in addition and I will post it. I still don't fully understand what you mean with a "printscreen of a opened databank + exported file", so as soon as you have clarified that, I can post it, or any other information you might need.


Thank you.  

AE
#13

Phronesis

09.06.2019 14:57

Attachment SQ view 1.png added

Attachment SQ view 2.png added

Here is more info for you with regards to this bug. I noticed that when you change the view, the data in the GUI is "refreshed", and after the switch, some systems will appear as "empty" in the results window as well. This is arbitrary of what view you change from and to, the important event is the change itself. 


See both attached pics. 

In the first pic, all systems will appear as legit in the results window (when you click on them however, some will appear as empty). 

In the second pic, you can see the results after the view has been switched. As you will note here, some systems are now displayed as "empty", even in this window. 


Hope this helps.

TT
#14

Tamas

11.06.2019 09:26
It looks there is something wrong in your views.


Can you please attach your view config ? 

\user\settings\views\databanks


Also would be great to attach a log file from the day when it happened.

\user\log\StrategyQuant

AE
#15

Phronesis

11.06.2019 13:37

Attachment View ALL.vw added

Attachment log_2019_06_10.log added

OK - see attached files. 


You could be right, but please note that this does not seem to be a "view specific" issue. It is not like this happens only when I select a specific view. But rather, the problem presents itself when I change views, and this is completely independent of what view I am changing from, and what view I am changing to.  


Also, the fact that the filters are not working flawlessly lead me to the hypothesis that it could have something to do with strategies not being filtered out properly at first (see my reasoning above). Again, I could obvioulsy be completely wrong here...


Thanks.

TT
#16

Tamas

11.06.2019 14:34
The attached view file is correct. I need the whole log file to see the error.


AE
#17

Phronesis

11.06.2019 23:02
I'm sorry but I don't understand what you mean with "the whole log file". I did attach a log file to my message above, but it seems like you need somehing else/in addition?


What am I missing?


Thanks.

TT
#18

Tamas

12.06.2019 12:39
The attached log contains only two lines.


Do you have teamviewer installed on your pc ?

If the problem is reproducable I can connect and check what happens exactly.


My skype is ttomas22. Can you please contact me so we can solve this problem ?

AE
#19

Phronesis

12.06.2019 12:44
Today is tight but tomorrow afternoon should work. 

I'll contact you then.


Thanks.

TT
#20

Tamas

19.06.2019 10:22
Maros / Tomas potrebujem pomoct s tymto taskom.


Pripojil som sa cez TeamViewer na zakaznikov pc kde mal otvoreny SQ.

V builde mal databank Results kde bolo viac tisic strategii s Default view. Kazda mala stats a vsetko vyzeralo ok.

Ked sa prepol na hociktory iny view tak stats boli 0. Ked sa prepol spat na default view stats boli prec.

Ulozil som strategiu do file a pozrel orders.bin ktore maju 0 bytov velkost takze chybaju.


Zakaznik mi pise na skype ze mu to robi na roznych pocitacoch a nevie co s tym. 


Stiahol som si od neho vsetky views  + build config andre.zip.


Nechal som bezat build s debug mode. Log je pripojeny o task vyssie.



Mozno robi problem ze ma max databank size nastaveny na 10000 ? Build mu bezi viac hodin az dni.


Napada vas nieco ohladom tohoto tasku ? vidite nieco v logu ? 


Ma novu default instalacku, ziadne custom snippets

MF
#21

Mark Fric

24.06.2019 11:26

Status changed from In progress to Waiting for information

Attachment migration_rate.jpg added

I believe I found the bug that is causing it, it is related to migration. 


Can you please test is with migration disabled? Just set Migration rate to 0 as in attached screenshot.


Let me know if it solved the issue for you.

AE
#22

Phronesis

24.06.2019 23:49
I can confirm that when I replicate your settings, I no longer have any empty strategies generated. 

Obviously however, I don't want migration to be set to 0. But at least now it seems like the root of the bug has been defined.


Thanks.

MF
#23

Mark Fric

25.06.2019 07:39

Status changed from Waiting for information to Fixed

thank you for confirmation. 


It will be fixed in the next build, so you'll be able to use migration, but until now this is the workaround.


Votes: +1

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please