0

I'm using matplotlib to make contourplots over some maps I have stored in a database but the process is taking several hours to produce each movie. The code I'm using to make this movie is:

def load_img(option, obs_id, columnshape):
    '''
    This function will load the image
    from the database and make the 
    conversion from string to nparray
    '''
    #starting a db session
    session = makesession()

    #defining the cases to query the db
    case = {'Bz': Observations.mean_bz,
            'Es': Observations.poyn_Es,
            'En': Observations.poyn_En,
            'Et': Observations.poyn_Et}

    #checking if the option selected was
    #a valid one
    while option not in case.keys():
        #feedback
        print('Invalid option. Select a valid one: ', case.keys())
        option = str(input())

    #querying the db
    s = sql.select([case[option]]).where(Observations.id == obs_id)

    #fetching the result
    rp = session.execute(s)
    result = rp.fetchone()

    #restoring the image
    img = ajuste(result[0],columnshape)

    return(img)

def db_animation(ar_id, option):
    '''
    Description
    '''
    vmin = -1e17
    vmax = 1e17
    levels = [vmin, 0.8*vmin, 0.6*vmin, 0.4*vmin, 0.2*vmin, 
              0.2*vmax, 0.4*vmax, 0.6*vmax,0.8*vmax,vmax]

    #getting the observation ids
    obs_ids = scout_obs_ids(ar_id)
    #obs_ids = [x for x in range(400,450)]

    #getting the columnshape
    columnshape = scout_colshape(obs_ids[0])

    #creating the figure objects
    fig, ax = plt.subplots(figsize = (12,8))

    #loading the first data
    data_bz = load_img('Bz', obs_ids[0], columnshape)
    data_E = load_img(option, obs_ids[0], columnshape)

    #making the image objects
    img1 = ax.imshow(data_bz, origin = 'lower', cmap = plt.cm.gray, 
                     animated = True)

    img2 = [ax.contourf(data_E, alpha = 0.35, 
                        #vmax = 1e17, vmin = -1e17,
                        levels = levels,
                        origin = 'lower',
                        cmap = 'PiYG')]

    #adding a colorbar
    fig.colorbar(img2[0], shrink = 0.75, label = 'W')


    def refresher(frame_number, img1,img2):
        '''
        description
        '''
        #taking the new data
        new_data_bz = load_img('Bz', obs_id = obs_ids[frame_number+1],
                               columnshape = columnshape)
        new_data_E = load_img(option, obs_id =  obs_ids[frame_number+1],
                              columnshape = columnshape)

        #setting the new data
        img1.set_data(new_data_bz)

        #removing the contours to start anew
        for tp in img2[0].collections:
            tp.remove()

        img2[0] = ax.contourf(new_data_E, alpha = 0.35, 
                              levels = levels,
                              origin = 'lower', cmap = 'PiYG')

        return(img1, img2[0].collections,)

    #using the animation function
    ani = FuncAnimation(fig, refresher,
                        frames=range(len(obs_ids)-1),
                        interval = 100,
                        #blit = True,
                        fargs = [img1,img2])

    #saving
    ani.save("test.mp4")

    return

On average those movies take 1200 images for each img object (a total around 2400) from the database. Each pair of images are individually loaded and restored to make the background image and the contour plot.

I was wondering about reasons why the processing time was escalating quickly as I increase the number of images to make the movie but could not get to a conclusion on my own. I find it particularly intriguing that when I set blit to True (which according to the documentation should help improve performance) I get the following error:

AttributeError: 'silent_list' object has no attribute 'set_animated'

I imagine that either my queries or the way I constructed my animation function are then highly inefficient. But I suspect more on the latter since when I'm normally using the DB the results are loaded in what I imagine is a reasonable time.

Can someone cast some light at this struggle for me?

Chicrala
  • 994
  • 12
  • 23
  • 1
    I wonder about this line `for tp in img2[0].collections:`. If it does not throw an error, it's at least not very helpful. I think what you really mean is `for tp in ax.collections:`? Of course I don't know if that is the cause of the slow saving. (See [mcve]) Blitting will not help when saving the figure. It will only speed up things on screen. – ImportanceOfBeingErnest Oct 29 '18 at 14:27
  • As far as I got the explanation from this [post](https://stackoverflow.com/questions/42386372/increase-the-speed-of-redrawing-contour-plot-in-matplotlib) the img object will carry the information that needs to be cleared. This needs to be done manually because unlike imshow the contourplot don't have a set_data attribute. I don't need things in screen anyway as long as it exports the movie. – Chicrala Oct 29 '18 at 14:36
  • 1
    Did you try my suggestion? – ImportanceOfBeingErnest Oct 29 '18 at 14:43
  • I'm currently doing it for a smaller sample of 50 entries. Still don't have the result to tell you if it is working accordingly or not. I will comment here about the output when I do. Thanks! – Chicrala Oct 29 '18 at 14:51
  • 1
    Ok, sorry, I think there is no difference to be expected between the two commands for the case shown here. As also suggested in the linked post you may first want to find out if obtaining the data or drawing the figure is the bottleneck. How large is `new_data_E` (`print(new_data_E.shape)`)? – ImportanceOfBeingErnest Oct 29 '18 at 15:01
  • Yes in the end they were taking similar times. All data for this series have 949x471. The group of images of every subject (a solar active region in this case) have over it's lifetime the same shape. I will test for time difference these steps then. Thanks for the hint. – Chicrala Oct 29 '18 at 15:05

0 Answers0