0

I am plotting error function values for around 10000 iterations and plotting them takes a lot of time. I want to avoid for-loop if possible to plot all of them in one figure, but speed it up anyway.

import time
import matplotlib.pyplot as plt
a = time.time()

for one in range(len(data)):
    plt.plot(data[one],"-o")
plt.show() 

b = time.time()
print(b-a)

What I tried was timing for:

(plt.plot(data[one],"-o") for one in range(len(data)))
plt.show()

But it didn't show any plot. So my goal is to speed up the plotting and remove for-loop if it is the bottleneck.

data is 

data = array([[  0. ,   0. ,   0. ,   0. ,   0. ],
       [-43.4, -18. , -10.5,  -7.4,  -5.7],
       [ 25.7,  18.3,  13.8,  10.7,   8.6],
       [-25. , -10. ,  -5.8,  -4.2,  -3.3],
       [ 16.1,  11.5,   8.6,   6.5,   5.1],
       [-16.2,  -6.4,  -3.8,  -2.9,  -2.4],
       [  9.6,   7.1,   5.2,   3.8,   2.9],
       [ -9.1,  -3.4,  -2. ,  -1.6,  -1.5],
       [  4.7,   3.9,   2.9,   2. ,   1.4],
       [ -4.5,  -1.3,  -0.7,  -0.8,  -0.8]])

If it matters, x axis can be taken as 

n = [i for i  in range(5)]
  • To avoid the loop, you can do `plt.plot(range(5), data.T)`. – ImportanceOfBeingErnest Aug 29 '19 at 21:48
  • The transpose of your array – ImportanceOfBeingErnest Aug 29 '19 at 21:50
  • It was 478 ms ± 35.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) for for loop, and 11.7 ms ± 2.24 ms per loop (mean ± std. dev. of 7 runs, 100 loops each) for what you proposed –  Aug 29 '19 at 21:54
  • For different timings, see [Many plots in less time - python](https://stackoverflow.com/questions/54492963/many-plots-in-less-time-python/54544965#54544965), but I don't know if any of the other options mentionned there would apply to your case. – ImportanceOfBeingErnest Aug 29 '19 at 21:58
  • I tried using LineCollection but got `ValueError: all input arrays must have the same shape` after I stacked data and n. but your first comment speeds up 10 times when I used time.time, they were ~400ms to ~40ms. As you might understand, earlier timings were from jupyter. I assume that we have exhausted matrix and LineCollection, can nans be used? –  Aug 29 '19 at 22:05
  • For a linecollection as well as the solution with nans you will need to duplicate your x values (as to have one x value per y value). – ImportanceOfBeingErnest Aug 29 '19 at 22:15
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/198685/discussion-between-ankiiiiiii-and-importanceofbeingernest). –  Aug 29 '19 at 22:22

1 Answers1

0

This is somewhat "cheating" but you can wrap a python function with a parallel processor. For example, if you have an 8 core processor, you can run your function in parallel using 7 cores or make it 7x faster.

import matplotlib.pyplot as plt
from multiprocessing import Pool, cpu_count

def multi_processor(function_name):

    # Use max number of system processors - 1
    pool = Pool(processes=cpu_count()-1)
    pool.daemon = True

    results = {}
    # for every item in your list of items, start a new process
    for one in range(len(data)):
        results[one] = pool.apply_async(your_function, args=(data[one]))

    # Wait for all processes to finish before proceeding
    pool.close()
    pool.join()

    # Results and any errors are returned
    return {your_function: result.get() for your_function, result in results.items()}

def your_function(arg1):
    try:
        return plt.plot(arg1,"-o")
    except Exception as e:
        return str(e)

if __name__ == "__main__":
    multi_processor("your_function")
    plt.show()
Calculus
  • 781
  • 7
  • 20
  • This is no doubt useful and I will be using(after learning ) it later, but as of now, the homework I am working on will be tested with limited libraries. I don't think this will be useful right now. –  Aug 29 '19 at 22:09
  • Did you actually test this? Will it produce a single plot or 5? – ImportanceOfBeingErnest Aug 29 '19 at 22:09
  • No I have not tested it specific to this question but I've tested it many times on other functions. Instead of plotting something I just use print statements to demonstrate the parallel processing functionality. Generic version is here https://stackoverflow.com/questions/48953878/calling-multiple-instances-of-python-scripts-in-matlab-using-java-lang-runtime-g/49120453#49120453 – Calculus Aug 29 '19 at 22:11
  • Ok, I would suggest to test it with an actual plot, because in my experience multiprocessing will spawn new processes which cannot be joined together into the single output plot after they are separated. – ImportanceOfBeingErnest Aug 29 '19 at 22:12
  • I will try it If I figure out where to put `data` variable. I am using a new file. and got `TypeError: your_function() takes 1 positional argument but 5 were given` –  Aug 29 '19 at 22:13
  • @ankiiiiiii I would spend too much time with this, until someone can actually show a working code which creates a single matplotlib figure from several processes. This is as far as I can see not even possible theoretically, but I would be happy to be convinced of the contrary. – ImportanceOfBeingErnest Aug 29 '19 at 22:19