3

I use ClearML to track my tensorboard logs (from PyTorch Lightning) during training. At a point later I start another script which connects to existing task and do some testing.

But unfortenautly I do not have all information in the second script, so I want to query them from the logged values from ClearML server.

How would I do this?

I thought about something like this, but havn't found anything in documentation:

task = Task.init(project_name="Project", task_name="name", reuse_last_task_id="Task_id, continue_last_task=True)
x_value, y_value = task.get_value(key="val/acc", mode="max")
x_value2, y_value2 = task.get_value(key="epoch", mode="x", x=x_value)
  • x_value would be my epoch or global step
  • y_value the maximum value of plot "val/acc"
  • x_value2 would be my epoch or global step
  • y_value2 the value of plot "epoch" at x_value
Daniel Andres
  • 99
  • 5

1 Answers1

2

Disclaimer I'm part of the ClearML (formerly Trains) team.

To get an existing Task object for a running (or completed/failed) experiment, assuming we know Task ID:

another_task = Task.get_task(task_id='aabbcc')

If we only know the Task project/name

another_task = Task.get_task(project_name='the project', task_name='the name')

Notice that if you have multiple task under the same name it will return the most updated one. Once we have the Task object, we can do:

latest_scalar_values_dict = another_task.get_last_scalar_metrics()

Which would return all the scalars min/maxm/last values, for example:

latest_scalar_values_dict = {
            'title': {
                'series': {
                    'last': 0.5,
                    'min': 0.1,
                    'max': 0.9
                    }
                }
            }

documentation here

If you need to get the entire graphs you can use task.get_reported_scalars() see docs

Martin.B
  • 599
  • 3
  • 9