-1

I have folder names which are date time formated as 2018-08-21 to 2018-10-16

Inside each folder there is a zip files which contains time values which is a linear scale, the time goes up linearly.

I'm trying to plot for each day, which has a lot of .bz2 file time series data, the time value at that date.

Right now I'm trying to do that:

 timearr = np.asarray(data1['time'])
            ax.plot(np.asarray(timeStamps), timearr)

            ax.set_title('title')
            ax.set_ylabel('date vs time ')
            ax.grid(True)

            # Format the x-axis for dates (label formatting, rotation)
            fig.autofmt_xdate(rotation=45)
            fig.tight_layout()

            plt.show()

but I get an error message, that both dimensions doesn't match.

dateStamps are list[2018-08-21
2018-08-22
2018-08-23
2018-08-24
2018-08-25]

data1['time'] = list of EPOC values.
andre
  • 731
  • 2
  • 13
  • 27

1 Answers1

0

Unfortunately, I don't know the detailed file structure, so I have to guess a little were the problem actually is

Here is some code to generate some folders with generic bz2 files:

import bz2
import numpy as np
import datetime
import os

startDate = datetime.datetime(2000,5,2,10,15,0,0)

for day in range(5):
    theDate = startDate +  datetime.timedelta(days=day)
    folder = "{}".format( theDate.replace( microsecond = 0 ).strftime("%Y-%m-%d") ) 
    os.mkdir( folder )
    data = ""
    for k in range(100):
        zzz = theDate + datetime.timedelta(seconds=137*k)
        data += "{} ".format( zzz.replace( microsecond = 0 ).strftime("%H:%M:%S") ) 
        d = zzz.day
        m = zzz.minute
        data += " {}\n".format( .17 * d + .003 * m**2 -.001 * m )
    myZip = bz2.BZ2File(os.path.join( folder, 'dat.bz2' ), 'w' )
    myZip.write( data )
    myZip.close()

Those folders and files a treat with:

import bz2
import numpy as np
import datetime
import os
import matplotlib.pyplot as plt
import matplotlib.dates as mdates

"""
SE posts I used 
https://stackoverflow.com/questions/1574088/plotting-time-in-python-with-matplotlib
https://stackoverflow.com/questions/11264521/date-ticks-and-rotation-in-matplotlib
"""

def split_data( inData ):
    rows=data.strip().split('\n')
    rowcol = [x.split() for x in rows ]
    x,y = zip(*rowcol)
    y = [float(z) for z in  y ]
    x = [ datetime.datetime.strptime(z, '%H:%M:%S') for z in x]
    return x,y

dataDict = dict()
for root, dirs, files in os.walk("."):
   for name in files:
       if name.split('.')[-1]=='bz2':
           base = os.path.basename( root )
           myPath = (os.path.join(root, name))
           bz = bz2.BZ2File( myPath, 'r' )
           data = bz.read()
           dataDict[ base ] = split_data( data )

myFmt = mdates.DateFormatter('%H:%M')

fig = plt.figure()
ax = fig.add_subplot( 1, 1, 1 )

for key, dt in dataDict.iteritems():
    ax.plot( *dt , label=key )
ax.xaxis.set_major_formatter(myFmt)
for label in ax.get_xmajorticklabels():
    label.set_rotation(30)
ax.set_ylabel('data (arb. u.)')
ax.set_xlabel('time')
ax.legend( loc=0 )
plt.tight_layout()
plt.show()

Providing: some data

Hope I got it right.

mikuszefski
  • 3,943
  • 1
  • 25
  • 38