1

I have a list, that can be of various lengths (less than 100 or more than 1.800.000) and I need to display all those values on a graph (a waveform) of 800 points on which I can draw.

I have tried taking a value for every step, where a step is the list length / 800. And it is the closest I can get to. I have tried taking the average value of the n surrounding of step with various n, but I can't get a waveform that satisfies me.

I know image compression work by taking an average of surrounding pixels, and I am trying to do the same with a list of values going from 0 to 200. I need to preserve the global aspect, but also the highest and lowest spikes.

rafael
  • 21
  • 2
  • I feel your question has more a mathematical element to it before you get to the code. Actually, you might be better off at first trying to get an answer on https://math.stackexchange.com/ about the algorithm. My (rusty) mathematical knowledge tells me what you most likely are looking for is a "sampling" algorithm. – adsy Nov 04 '22 at 22:40
  • Also sounds like you are trying to derive a "line of best fit" from the data. https://en.wikipedia.org/wiki/Curve_fitting comes to mind. – adsy Nov 04 '22 at 22:42
  • If it matches your problem -- libraries exist https://github.com/Tom-Alexander/regression-js. https://mathworld.wolfram.com/LeastSquaresFitting.html may be of interest. https://en.wikipedia.org/wiki/Least_squares – adsy Nov 04 '22 at 22:43
  • Do you mean that you are only allowed to plot 800 points ? –  Nov 04 '22 at 22:46
  • @YvesDaoust Yes but I have alot more points to plot, so I need a way to "shrink down" the list to 800, by keepink the global aspect, and spikes – rafael Nov 04 '22 at 22:56
  • I mean, is it impossible to plot more ? –  Nov 04 '22 at 23:01
  • 800 points out of 1800000 is really few. –  Nov 04 '22 at 23:02
  • Thanks @adsy it is close, but I have values like 0 10 200 and not points in 2d. – rafael Nov 04 '22 at 23:03
  • @YvesDaoust I have a screen, and I am limited by a horizontal resolution of 800 points. I have a list of values going from 0 to 200, but this list of values can be of any lenght (1.800.000 for exemple) I need the same global aspect, and spikes – rafael Nov 04 '22 at 23:06
  • I think you can adapt it for this case https://stackoverflow.com/questions/1400213/3d-least-squares-plane – adsy Nov 04 '22 at 23:17
  • Please provide enough code so others can better understand or reproduce the problem. – Community Nov 05 '22 at 06:24
  • Either adapt to the waveform, and be satisfied, or increase points, to how much you need. You can do this by making the points scrollable with something like a *scroll bar* or *zoom in tool* – Neptotech -vishnu Nov 05 '22 at 06:31
  • take a look at Stack overflows reputation shower, [here](https://i.ibb.co/MC9K9Qt/image.png), the zoom outed part doesn't contain exact precision as zoom in part. Some or not shown. Though it's bar graph, it gives you some idea – Neptotech -vishnu Nov 05 '22 at 06:38
  • 1
    @adsy: your suggestions are completely irrelevant. –  Nov 05 '22 at 08:37
  • That's what I thought, there is no limitation. What if you plot all available points ? Show us. –  Nov 05 '22 at 08:38

2 Answers2

0

First of all this is more resampling than scaling. As mentioned in the comments you have 2 cases:

  1. upscaling/sampling n<800

    this is relatively simple you just interpolate the missing values. I would go for piecewise cubic interpolation however you did not show any sample input so other methods might look better in some cases...

  2. downscaling/sampling n>800

    if the input dataset is many times bigger than 800 the result of simple average or slifing average can miss the true nature of your data, also aliasing can trick us to see things that are not in the data itself...

    To overcome this I usually do not plot a curve but something like this instead:

    plot

    so handle each point of your plot as 3 values local min/max and (sliding) average. This way the plot can show also oscilations of the data represented by single x value of plot which might hold many values of input data.

    See similar Correct way to draw zoomable audio waveform (it just use local min/max) like this:

    min max plot

    In case this is still not enough you might do a histogram and use it to modulate the color of each pixel for the area between min and max. That will show where the majority of values lies. For such plot I vould not do the average line as the colors itself should show it on its own ...

    Now in order to combine more such plots together just use blending because standard overdraw will not be good anymore...

Spektre
  • 49,595
  • 11
  • 110
  • 380
0

I couldn't find anything good, so I created a small program and the results are awesome:

  1. Divide the very long list into 400 smaller lists
  2. For each smaller list calculate the biggest difference, between the smaller and larger value in that list.
  3. Plot 2 points per list, one at (offset + delta / 2) and one at (offset - delta / 2)

Results : from 453932 points to 800 points before after Python code :

numberOfSmallerList = 400
small_list_len = int(len(big_list) / numberOfSmallerList)

finalPointsToPlot = []

for i in range(0, len(big_list), small_list_len):
    biggestDiff = max(big_list[i:i+small_list_len]) - 
    min(big_list[i:i+small_list_len])

    finalPointsToPlot.append(biggestDiff/2 + 100)
    finalPointsToPlot.append(100 - biggestDiff/2)


import matplotlib.pyplot as plt
plt.plot(finalPointsToPlot)
plt.show()
rafael
  • 21
  • 2