Time series Test and Score

[nasser1980]

Hello
Applying random validation on time series giving wrong results in my case . Can we force training in one direction ? From past to current time predict next point in future
I want to perform leave one out test on time series with respect to date . So every we perform training on all past data and test on the current point and do that one by one until the end of data
Also we with prediction . Can we train all data then use to predict one future point then add it to the data to predict next future point ?
Thank you

[ajdapretnar]

I believe the widget for this task is Model Evaluation. Test & Score is not optimized for timeseries prediction. I believe you are looking for the Timeseries add-on.

[nasser1980]

Thank you Adja , kNN give me good results with Test &Score. But I can't trust this results since it is done randomly. Is there a way to test supervied Models on times series as Evaluation widget?
Thank you so much for fast responding. I appreciate it really
Nasser

[nasser1980]

I am sorry. I think I close it by a mistake

[janezd]

If I'm not mistaken, the answer would have to be "no". We need to discuss whether such type of sampling (train on data points up to n (or the last n-point window), and test on the next one (or the next m-point window) belongs to Test and Score widget or to the time series add-on. It's a tough one:

• there are lots of options that would complicate the already large user interface of the widget
• but a separate widget in the add-on would replicate most of the functionality of Test and Score, which contains some complex, but very optimized code.

We could, of course, factor out the common code to a base widget, but this might be quite a project.

[ajdapretnar]

As stated above, this might be more appropriate for Model Evaluation widget from Timeseries. We would have to make it work with kNN in this case (or other models). I cannot think of a case where one would predict the next m-point window other than a timeseries.

[ajdapretnar]

Decision from the meeting:
Adapt Model Evaluation to work with core Orange models, i.e. train the model on first n instances, test it on n+(step).

[nasser1980]

Do you mean test on train option from test widget? If you mean this it will take long time to perform it on data with thousands of data to be tested .
Can you show me in simple example I will be grateful
Really
many thanks

[kernc]

Decision from the meeting

One simple, standard alternative would be to introduce the lag/shift operator:

• https://www.knime.com/blog/all-you-need-is-the-lag-column-node
• https://docs.rapidminer.com/9.2/studio/operators/modeling/time_series/transformation/lag_series.html