2

I have Tensorboard data and want it to download all of the csv files behind the data, but I could not find anything from the official documentation. From StackOverflow, I found only this question which is 7 years old and also it's about TensorFlow while I am using PyTorch.

We can do this manually, as we can see in the screenshot, manually there is an option. I wonder if we can do that via code or it is not possible? As I have a lot of data to process.

enter image description here

Adnan Ali
  • 2,851
  • 5
  • 22
  • 39
  • I have found this script on GitHub that will convert a tensorboard event log to a CSV [script](https://github.com/theRealSuperMario/supermariopy/blob/master/scripts/tflogs2pandas.py) – gerda die gandalfziege Feb 23 '22 at 15:23

1 Answers1

3

With the help of this script Below is the shortest working code it gets all of the data in dataframe then you can play further.

import traceback
import pandas as pd
from tensorboard.backend.event_processing.event_accumulator import EventAccumulator

# Extraction function
def tflog2pandas(path):
    runlog_data = pd.DataFrame({"metric": [], "value": [], "step": []})
    try:
        event_acc = EventAccumulator(path)
        event_acc.Reload()
        tags = event_acc.Tags()["scalars"]
        for tag in tags:
            event_list = event_acc.Scalars(tag)
            values = list(map(lambda x: x.value, event_list))
            step = list(map(lambda x: x.step, event_list))
            r = {"metric": [tag] * len(step), "value": values, "step": step}
            r = pd.DataFrame(r)
            runlog_data = pd.concat([runlog_data, r])
    # Dirty catch of DataLossError
    except Exception:
        print("Event file possibly corrupt: {}".format(path))
        traceback.print_exc()
    return runlog_data
path="Run1" #folderpath
df=tflog2pandas(path)
#df=df[(df.metric != 'params/lr')&(df.metric != 'params/mm')&(df.metric != 'train/loss')] #delete the mentioned rows
df.to_csv("output.csv")
Adnan Ali
  • 2,851
  • 5
  • 22
  • 39
  • 1
    Note that the `EventAccumulator` limits the number of events stored in memory by default. Therefore, your `event_list` may not contain every event within the file. Pass the `size_guidance={"scalars": 0}` argument to the constructor to lift the limit for the scalar data. – Dudly01 May 21 '23 at 16:21