Matplotlib time-based heatmap

Question

Background: I picked up Python about a month ago, so my experience level is pretty slim. I'm pretty comfortable with VBA though years of data analysis through excel and PI Processbook.

I have 27 thermocouples that I pull data for in 1s intervals. I would like to heatmap them from hottest to coldest at a given instance in time. I've leveraged seaborn heatmaps, but the problem with those is that they compare temperatures across time as well and the aggregate of these thermocouples changes dramatically over time. See chart below:

Notice how in the attached, the pink one is colder than the rest when all of them are cold, but when they all heat up, the cold spot transfers to the orange and green ones (and even the blue one for a little bit at the peak).

In excel, I would write a do loop to apply conditional formatting to each individual timestamp (row), however in Python I can't figure it out for the life of me. The following is the code that I used to develop the above chart, so I'm hoping I can modify this to make it work.

tsStartTime = pd.Timestamp(strStart_Time)
tsEndTime = pd.Timestamp(strEnd_Time)
t = np.linspace(tsStartTime.value,tsEndTime.value, 150301)
TimeAxis = pd.to_datetime(t)

fig,ax = plt.subplots(figsize=(25,5))
plt.subplots_adjust(bottom = 0.25)

x = TimeAxis

i = 1
while i < 28:
     globals()['y' + str(i)] = forceconvert_v(globals()['arTTXD' + str(i)])
     ax.plot(x,globals()['y' + str(i)])
     i += 1

I've tried to use seaborn heatmaps, but when i slice it by timestamps, the output array is size (27,) instead of (27,1), so it gets rejected.

Ultimately, I'm looking for an output that looks like this:

Notice how the values of 15 in the middle are blue despite being higher than the red 5s in the beginning. I didnt fill out every cell, but hopefully you get the jist of what I'm trying to accomplish.

This data is being pulled from OSISoft PI via the PIConnect library. PI leverages their own classes, but they are essentially either series or dataframes, but I can manipulate them into whatever they need to be if someone has any awesome ideas to handle this.

Here's the link to the data: https://file.io/JS0RoQvDL6AB

Thanks!

Answer 1

You are going the wrong way with globals. In this case, I suggest to use pandas.DataFrame.

What you are looking for can be produced like this:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Settings
col_number = 5
start = '1/1/2022 10:00:00'
end = '1/1/2022 10:10:00'

# prepare a data frame
index = pd.date_range(start=start, end=end,  freq="S")
columns = [f'y{i}' for i in range(col_number)] 
df = pd.DataFrame(index=index, columns=columns)

# fill in the data
for n, col in enumerate(df.columns):
    df[col] = np.array([n + np.sin(2*np.pi*i/len(df)) for i in range(len(df))])

# drawing a heatmap
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(18, 5))

ax1.plot(df)
ax1.legend(df.columns)

ax2.imshow(df.T, aspect='auto', cmap='plasma')
ax2.set_yticks(range(len(df.columns)))
ax2.set_yticklabels(df.columns)

plt.show()

Here:

1. As far as you didn't supply data to reproduce your case I use sin as illustrative values.
2. Transposing df.T is needed to put records horizontally. Of course, we can initially write data horizontally, it's up to you.
3. set_yticks is here to avoid troubles when changing the y-labels on the second figure.
4. seaborn.heatmap(...) can be used as well:

import seaborn as sns
data = df.T
data.columns = df.index.strftime('%H:%M:%S')
plt.figure(figsize=(15,3))
sns.heatmap(data, cmap='plasma', xticklabels=60)

---

Update

To compare values at each point in time:

data = (data - data.min())/(data.max() - data.min())
sns.heatmap(data, cmap='plasma', xticklabels=60)