Plotting 2 columns from multiple csv files from a directory

Question

I have several CSV files in a directory, containing hundreds of rows as shown below:

1598310002,2020-08-25 00:00:02,27.00,52.00,29.00,61.00,265,0.8833333333333333,457.74644768518516
1598310303,2020-08-25 00:05:03,27.00,52.00,29.00,61.00,119,0.39666666666666667,206.35741240370373
1598310602,2020-08-25 00:10:02,27.00,52.00,29.00,61.00,177,0.59,306.3231369,0.6368975735483338

The first column is Unixtime, second column is gmtime and other columns different numbers.

The idea is to show a plot of data in each column with date/time as x-axis (column 0, column1) for one CSV file at a time and so on until the given directory is empty.

StackOverFlow is not a coding service. Please show the code you have tried and include any errors that you have come across. Also, please format your code. You can see: https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples — David Erickson, Oct 09 '20 at 10:10

score 0 · Answer 1 · answered Oct 09 '20 at 12:34

    import pandas as pd
    import glob
    import matplotlib.pyplot as plt
    import pylab

    columns = pd.read_csv("The_directoryForHeadrs\COLUMNS.csv").columns
    AQM1 = glob.glob(r"files_directory/*.csv")

   for AQM in AQM1:
       df = pd.read_csv(AQM,header=None)
       df.columns = columns
       df.drop(["unixtime","in","rh","out","rh2","LPO",], axis=1, inplace=True)
       df.set_index("gmtime", inplace=True)
       #correct the underscores in old datetime format
       df.index = [" ".join(val.split("_")) for val in df.index]
       df.index = pd.to_datetime(df.index)
   df

Here is my Code, when I run it I get this error: TypeError Traceback (most recent call last) ~\Anaconda3\lib\site-packages\pandas\core\arrays\datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object) 1857 try: -> 1858 values, tz_parsed = conversion.datetime_to_datetime64(data) 1859 # If tzaware, these values represent unix timestamps, so we

pandas_libs\tslibs\conversion.pyx in pandas._libs.tslibs.conversion.datetime_to_datetime64()

TypeError: Unrecognized value type: <class 'str'>

During handling of the above exception, another exception occurred:

ParserError Traceback (most recent call last) in 14 #correct the underscores in old datetime format 15 df.index = [" ".join(val.split("_")) for val in df.index] ---> 16 df.index = pd.to_datetime(df.index) 17 df 18

~\Anaconda3\lib\site-packages\pandas\core\tools\datetimes.py in to_datetime(arg, errors, dayfirst, yearfirst, utc, format, exact, unit, infer_datetime_format, origin, cache) 736 else: 737 convert_listlike = partial(convert_listlike, name=arg.name) --> 738 result = convert_listlike(arg, format) 739 elif is_list_like(arg): 740 try:

~\Anaconda3\lib\site-packages\pandas\core\tools\datetimes.py in _convert_listlike_datetimes(arg, format, name, tz, unit, errors, infer_datetime_format, dayfirst, yearfirst, exact) 438 assert format is None or infer_datetime_format 439 utc = tz == "utc" --> 440 result, tz_parsed = objects_to_datetime64ns( 441 arg, 442 dayfirst=dayfirst,

~\Anaconda3\lib\site-packages\pandas\core\arrays\datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object) 1861 return values.view("i8"), tz_parsed 1862 except (ValueError, TypeError): -> 1863 raise e 1864 1865 if tz_parsed is not None:

~\Anaconda3\lib\site-packages\pandas\core\arrays\datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object) 1846 1847 try: -> 1848 result, tz_parsed = tslib.array_to_datetime( 1849 data, 1850 errors=errors,

pandas_libs\tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas_libs\tslib.pyx in pandas._libs.tslib.array_to_datetime_object()

pandas_libs\tslibs\parsing.pyx in pandas._libs.tslibs.parsing.parse_datetime_string()

~\Anaconda3\lib\site-packages\dateutil\parser_parser.py in parse(timestr, parserinfo, **kwargs) 1372 return parser(parserinfo).parse(timestr, **kwargs) 1373 else: -> 1374 return DEFAULTPARSER.parse(timestr, **kwargs) 1375 1376

~\Anaconda3\lib\site-packages\dateutil\parser_parser.py in parse(self, timestr, default, ignoretz, tzinfos, **kwargs) 647 648 if res is None: --> 649 raise ParserError("Unknown string format: %s", timestr) 650 651 if len(res) == 0:

ParserError: Unknown string format: gmtime

Plotting 2 columns from multiple csv files from a directory

1 Answers1