0

I have a problem and I am not sure what is the prolem. Here is my Pandas DataFrame : enter image description here I am trying to plot it with the follwoing python code:

ig, ax = plt.subplots(figsize=(12, 3))
plstm['y_inverse'].plot(ax=ax, label='Test', color = 'blue')
plstm['yhat_inverse'].plot(ax=ax, label='Predictions', color = 'crimson')
plt.title('LSTM Forecast', fontsize = 18)
plt.legend(fontsize = 12);

and here is the result : enter image description here

The issue is when I plot individual column, the y-axis is different, namely, y_test_inverse is from 0-50,but yhat_inverse is from approximately 20-30. This is so weird!!! I wanted to plot both columns to see if the prediction is close to the ground truth. As far as I am concerned, both columns have the same shape when plotting it out individually. Can anyone help me with it, please!

I have tried to reset y-axis like so:

plt.ylim([0,50])

but, it didn't change anything. I think I might need to rescale the yhat_inverse column. The problem is I don't know how.

Here is a reproducible example:

a = [22.49228808, 23.67596776, 25.23977312, 26.84448022, 28.81581532,
       30.33304259, 30.33647936, 31.0066119 , 31.32509943, 29.50372721,
       27.10255157, 25.2329266 , 19.34649399, 17.18931596, 14.64576531,
       13.45902426, 14.84226619, 17.34210585, 21.55897865, 22.84701125]
b= [23.13694191, 23.27906036, 23.4411087 , 23.66352272, 23.92148399,
       24.2199173 , 24.50629044, 24.60942459, 24.62063408, 24.66880989,
       24.52558517, 24.11040497, 23.7087822 , 23.08251572, 22.49897194,
       22.16906548, 22.03343964, 22.09576797, 22.44637871, 23.01521301]
d['Date'] = pd.date_range(start='2022-01-01 01:00:00', periods=len(a), freq='H')
d = pd.DataFrame({'a': a, 'b': b})
d.set_index('Date')
Ulffer
  • 13
  • 3
  • Hi Ulffer! please provide [a minimal reproducible example](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example). Don't post a photo of your data, post (a small sample of) the actual thing! – Mark Jun 27 '23 at 10:14
  • I'm guessing that when Python is plotting the 'prediction' column, it seems that the values don't vary as wildly as the Test set, so it zooms the y axis in a bit. Seems perfectly normal no? – Mark Jun 27 '23 at 10:17
  • I ran the code you provided with random data. Setting `plt.ylim([0,50])` forced the plot to be in the 0-50 range. Unable to reproduce the issue you are referring to... – Redox Jun 27 '23 at 10:19
  • @Mark, when I zoom in, it's okay, the prediction is not too bad, but the point is to plot them with the same scales, so it's easier to see the differences. – Ulffer Jun 27 '23 at 10:34
  • @Redox, I know, I did run a randomly generated example prior to my post here. I didn't have this issue. However, when I am trying to plot my actual data, the same issue occurs :(. – Ulffer Jun 27 '23 at 10:37
  • Hi - I used the data you provided and plotted the same... just the crimson line, just the blue line and both lines. All work with the range of 0-50 set. Sorry, again not able to reproduce your issue... perhaps someone else can replicate the issue – Redox Jun 27 '23 at 10:41
  • @Redox, Thank you so much for trying to help me, mate. *^_^* – Ulffer Jun 27 '23 at 10:46
  • I don't get any problem either. But from your example data, I can see that that y_inverse/a has a larger standard deviation (about 5.8) than yhat_inverse (about 0.9). Since both have a mean around 23.5, the resulting plot has a much smaller spread for the predictions. – 9769953 Jun 27 '23 at 11:34
  • So perhaps there is nothing wrong with the plot, but your actual predicted data simply has a much smaller spread than the actual data. `plstm.describe()` can give a quick idea about that. – 9769953 Jun 27 '23 at 11:35
  • @9769953, Thank you so much :). Indeed, the std is the problem. You are right about the plot; there isn't anything wrong with it. I must return to my model to see if I can improve it. – Ulffer Jun 27 '23 at 11:52

0 Answers0