Loop live update Julia plot

Question

I'm brand new to Julia (v1.7.1) and I've been using VSCode as an IDE. In VSCode (v1.64) I've installed the Julia Extension (v1.5.10). When plotting in VSCode, the plot shows up in the plot pane by default. I'm using the Plots (v1.25.7) package and the "gr" backend, as it's one of the "faster" options.

I'm trying to make a "live" time series plot, in which the series is updated in a loop. This seems to be a popular problem, as there are many questions addressing this, but I've found no "clean" solution yet. I should emphasize that I'm not trying to make an animation, which is fabricated upon termination of the loop. I want to update the plot as the loop is running. I've looked at SmoothLivePlot, but I think this requires that I know the series size before hand, which is not the case in my application. Then again, maybe I'm misinterpreting the package.

I'm going to present what I've done this far, with hopes for improvement. I first created a plotting function

function plt_update(p,time_series,var_series)
    plot!(time_series, var_series,
        label = "",
        linecolor = :red)
    display(p)
end

Then I initialized the plot

model_time = 100
p = plot([0.0],[0.0],
    label = "",
    linecolor = :red,
    xlims = (0, model_time),
    ylims = (0, 1),

display(p)

My loop is then called (NOTE: that all the code shown in this post is wrapped in a function and run in the RPEL, hence variables do not need to be defined with "global" inside the while loop. This is due to Julia's optimization and scope design...from what I've read. See another discussion on this for an example).

run_time = 0.0
time_series = [0.0]
var_series = [0.0]
while run_time < model_time

    # Calculate new timestep
    timestep = rand(Float64)  # Be sure to add Random
    run_time += timestep

    # Computations
    sleep(timestep/10)

    # Build vector
    push!(time_series,run_time)
    push!(var_series,timestep)

    # Live plots
    plt_update(p,time_series,var_series)
    
end

I've encountered a few problems with this. First, I don't know if this is just an issue with VSCode or who to point the finger at, but putting display(p) inside the function to update the plot in the VSCode plot pane ends up creating a new plot for each iteration in the loop. Clearly this is not what is intended. I found that if I shut off the "plot in pane" option (File > Preferences > Settings > Extensions > Julia), then a single plot window is created. I'm not sure if "create new plot in pane" is expected or an issue (again, I'm new to this). Nevertheless, when plotting outside the VSCode, the above code works as I expected.

For the next issue, which I think is most important here, is that inside the plotting function, the call to plot! adds a new vector to p, while saving the previous one as well. In other words, p is not being updated with the new series, it is growing by appending a whole new vector. This is clear as the plotting comes to a grinding halt after many iterations. Also, if you remove the "color" attribute, you'll see the line changes color with each iteration. In effect, what is being plotted is many lines, all overlapping.

I then dove into p to look more closely at what is going on and made some changes to the plotting function

function plt_update(p,time_series,var_series)
    push!(p.series_list[1].plotattributes[:x],time_series[:][end])
    push!(p.series_list[1].plotattributes[:y],var_series[:][end])
    display(p)
end

Above, instead of creating a new series_list (as was the case before w/ plot!), I'm now updating the series w/ the new data. This works much better than before and behaves as expected. While it's only a slight improvement, I've further modified the function and the function call by passing a scalar instead of a vecotr

function plt_update(p,run_time,variable)
    push!(p.series_list[1].plotattributes[:x],run_time)
    push!(p.series_list[1].plotattributes[:y],variable)
    display(p)
end

in which the function call is now plt_update(p,run_time,timestep). As you can see, I sleep for a random time then divide that by 10, which I found to be about as much lag as I can afford before it loses it's "near realtime" appeal. Dividing by 100, for example, results in rather noticeable lag.

So my question is...is there a way to improve this, where the lag is reduced? Being new to Julia, I'm not aware of all the plotting options or how to access the "guts" to make improvements on my own.

EDIT:

I've just become aware of "Makie" and "Observables". I'm going to do a bit more research on those to see if this offers an improvement in the latency.

EDIT2:

In my research, I found a much cleaner way to express the last function (also see here for further confirmation of the approach)

function plt_update(p,run_time,variable)
    push!(p,1,run_time,variable)
    display(p)
end

Answer 1

The solution I found w/ Makie & observables is without a doubt the best! Between the YouTube video and code, I was able to apply it to my example above. Granted, my loop is only ~150 iterations, there is negligible lag (which was far from the case prior). Taking out the sleep function, the plot is instantaneous. I would encourage others to try this approach for "near real-time" plots.

The packages needed are

using GLMakie
using GeometryTypes

I'm not sure if GeometryTypes is needed explicitly (I thought GLMakie would bring in the necessary libs), but I was getting an error stating that Point2f was not found.

First, create the observable (note that it's a vector of type Point2f)

pt_series = Observable([Point2f(0, 0)])

Then initialize the plot

fig = Figure(); display(fig)
ax = Axis(fig[1,1])

lines!(ax, pt_series; linewidth = 4, color = :purple)

# Static elements
ax.title = "Time Series"
ax.xlabel = "Time (s)"
ax.ylabel = "Data"
xlims!(ax, 0, 100)
ylims!(ax, 0, 1)

Then run the loop

model_time = 100
run_time = 0.0 
while run_time < model_time

    # Calculate new timestep
    timestep = rand(Float64)
    run_time += timestep

    # Computations
    #sleep(timestep/1000)

    # Live plots
    push!(pt_series[], Point2f(run_time, timestep))
    notify(pt_series) # let the observable know that a change has happened

end

As I stated in the question, all of the code above should be wrapped in a function to prevent scoping errors.