0

When you already have time series data set but use internal dtype to index with date/time, you seem to be able to plot the index cleanly as here.

But when I already have data files with columns of date&time in its own format, such as [2009-01-01T00:00], is there a way to have this converted into the object that the plot can read? Currently my plot looks like the following. enter image description here

Code:

dir = sorted(glob.glob("bsrn_txt_0100/*.txt"))
gen_raw = (pd.read_csv(file, sep='\t', encoding = "utf-8") for file in dir)
gen = pd.concat(gen_raw, ignore_index=True)
gen.drop(gen.columns[[1,2]], axis=1, inplace=True)

#gen['Date/Time'] = gen['Date/Time'][11:] -> cause error, didnt work
filter = gen[gen['Date/Time'].str.endswith('00') | gen['Date/Time'].str.endswith('30')]
filter['rad_tot'] = filter['Direct radiation [W/m**2]'] + filter['Diffuse radiation [W/m**2]']

lis = np.arange(35040) #used the number of rows, checked by printing. THis is for 2009-2010.
plt.xticks(lis, filter['Date/Time'])
plt.plot(lis, filter['rad_tot'], '.')
plt.title('test of generation 2009')
plt.xlabel('Date/Time')
plt.ylabel('radiation total [W/m**2]')
plt.show()

My other approach in mind was to use plotly. Yet again, its main purpose seems to feed in data on the internet. It would be best if I am familiar with all the modules and try for myself, but I am learning as I go to use pandas and matplotlib.

So I would like to ask whether there are anyone who experienced similar issues as I.

dia
  • 431
  • 2
  • 7
  • 22
  • Of which I am aware of. The current dtype is not string. It is object. Do you know the difference between the two types? If you have read my post, that is. – dia Feb 26 '18 at 12:34
  • A pandas column which stores strings has dtype `object`; as said you need to convert to `datetime` – ImportanceOfBeingErnest Feb 26 '18 at 13:27
  • That is how it is perceived, I understand. But I cannot use slice, join, split functions that are used for string dtype of pandas series. And it was partially explained when I read another post explaining the object type contains different pointers for the combination of dtype it contains, that there is a difference. I may be wrong, but I just don't find your comment fit in this environment. I thought this community was for learning. – dia Feb 26 '18 at 13:34
  • I'm lost on what the problem is. You may look at [ask] and [mcve] on how to write questions here that can be answered. So if the question is how to convert a string like `"2009-01-01T00:00"` to a datetime object, you will want to look at other questions which have a solution to this. – ImportanceOfBeingErnest Feb 26 '18 at 14:12

1 Answers1

1

I think you need set labels to not visible by loop:

ax = df.plot(...)

spacing = 10
visible = ax.xaxis.get_ticklabels()[::spacing]
for label in ax.xaxis.get_ticklabels():
    if label not in visible:
        label.set_visible(False)
jezrael
  • 822,522
  • 95
  • 1,334
  • 1,252
  • err, I read somewhere about plotting with matplotlib being better than using pandas but true, the link also shows using pandas' plot function to display date/time properly.. so no way of avoiding it? – dia Feb 25 '18 at 08:22
  • I just found this: acc. to [link](https://pandas.pydata.org/pandas-docs/stable/visualization.html), 'If the index consists of dates, it calls gcf().autofmt_xdate() to try to format the x-axis nicely as per above.' And the page also mentions that the pandas' plot function is a wrapper around plt.plot - So I had better try. I don't have that many lines at the moment. Just a very basic frame for plot and to save to a designated path.. – dia Feb 25 '18 at 08:29
  • @HSL - My code is for ploting large Dataframe, then get exaclty problem with x-axis as you show in picture. If small data, `pandas` handle it by default. – jezrael Feb 25 '18 at 08:33
  • @HSL - I do some research and you get this problem if ploting not datetimes, but strings created from datetimes in x-axis. See [this](https://stackoverflow.com/a/42453881/2901002) sample. – jezrael Feb 25 '18 at 08:40
  • Hey thanks, your suggestion makes more sense after following your thread. `to_datetime` function seems to be what I need except that my column has 'T' in the middle of the string, which I haven't been able to remove successfully as shown in the code I uploaded.. – dia Feb 26 '18 at 04:37
  • @HSL exactly.If need custom format of datetimes, need my solution, if use datetimes, matplotlib handle it very nice. – jezrael Feb 26 '18 at 05:13