a lil bit of programming math

Question

Just wondering if anyone can help me with a little programming math i'm having trouble with.

What I am trying to create is a submit script (using python and .bat) for Nuke (vfx). The issue I am having is I can't figure out how to add the remainder of frames to the already calculated stack.

To be more clear...

In Nuke, i have to render 20 frames. I have 16 threads. Nuke only uses 1 thread. I want to write a script that takes the number of frames and divide it by the number of threads and write out a bat file using python. The issue come is when I have a remainder. I want to take the remainder and apply it back in the render stack.

example (first loop)

thread1 = 1 frame
thread2 = 1 frame
thread3 = 1 frame
thread4 = 1 frame
thread5 = 1 frame
thread6 = 1 frame
...
thread16 = 1 frame

Once it has done this...the remainder is 4. I want the remainder to be distributed among the threads. so...

example (second loops)

thread1 = 2 frame
thread2 = 2 frame
thread3 = 2 frame
thread4 = 2 frame
thread5 = 1 frame
thread6 = 1 frame
...
thread16 = 1 frame

the 4 gets added among the first few threads totalling 20 frames.

I will greatly appreciate any help, tips, comments that anyone has to offer. :)

Thanks

score 4 · Answer 1 · edited Aug 03 '19 at 05:09

This is a classic use of the Bresenham algorithm:

def partition(lst, n):
    increment = len(lst) / float(n)
    last = 0
    i = 1
    results = []
    while last < len(lst):
        idx = int(round(increment * i))
        results.append(lst[last:idx])
        last = idx
        i += 1
    return results


for i, frames in enumerate(partition(range(20),16)):
    if len(frames)>1:
        print 'thread'+str(i+1),'=', len(frames),'frames'
    else:
        print 'thread'+str(i+1),'=', len(frames),'frame'

The partition bit is from this answer.

It prints:

thread1 = 1 frame
thread2 = 2 frames
thread3 = 1 frame
thread4 = 1 frame
thread5 = 1 frame
thread6 = 2 frames
thread7 = 1 frame
thread8 = 1 frame
thread9 = 1 frame
thread10 = 2 frames
thread11 = 1 frame
thread12 = 1 frame
thread13 = 1 frame
thread14 = 2 frames
thread15 = 1 frame
thread16 = 1 frame

You could also use Mark Dickinson's solution if you don't mind the two frame threads being front loaded.

Then you have:

def partition(lst, n):
    q, r = divmod(len(lst), n)
    indices = [q*i + min(i, r) for i in xrange(n+1)]
    return [lst[indices[i]:indices[i+1]] for i in xrange(n)]

which prints:

thread1 = 2 frames
thread2 = 2 frames
thread3 = 2 frames
thread4 = 2 frames
thread5 = 1 frame
thread6 = 1 frame
thread7 = 1 frame
thread8 = 1 frame
thread9 = 1 frame
thread10 = 1 frame
thread11 = 1 frame
thread12 = 1 frame
thread13 = 1 frame
thread14 = 1 frame
thread15 = 1 frame
thread16 = 1 frame

score 1 · Accepted Answer · answered May 22 '12 at 02:43

1

frames can be a list of any object, eg dict or "Frame" object. Here I have just used ints

>>> frames = range(20)
>>> threads = 16
>>> [frames[x::threads] for x in range(threads)]
[[0, 16], [1, 17], [2, 18], [3, 19], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]]

I think you may be better off putting the frames in a Queue though, since some frames may render faster than others

answered May 22 '12 at 02:43

John La Rooy

295,403
53
369
502

I'd love to setup a Queue, but I have no idea how :(. – にか May 22 '12 at 03:30
I've never seen python written in that format. [frames[x::threads] for x in range(threads)]. – にか May 22 '12 at 03:31
@にか, It's just a list comprehension. Which part do you not understand? – John La Rooy May 22 '12 at 03:44
oh..i mean i don't know how to queue. What I'm trying to do is get Nuke to render multiple frames at once. Nuke uses only 1 thread...I think.. so I am creating this script that will take a "write" node in nuke and do the math...and write out bat files and execute them at once. its a hack but....i have no better way of doing it. – にか May 22 '12 at 03:52
would you mind rewriting your example the other way... I'm having trouble reading it...Please :) – にか May 22 '12 at 04:20

dawg · Answer 3 · 2012-05-22T06:33:58.727

0

frames=20
tPos=16
Ts=divmod(frames,tPos)
threads=[Ts[0]+1 if i < Ts[1] else Ts[0] for i in range(tPos)]

>>> threads    
>>> [2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

If you are having trouble with list comprehensions, the same thing can be written this way:

threads=[]
for i in range(tPos):
    threads.append(Ts[0]+1 if i<Ts[1] else Ts[0])

Then to format it, do something like:

for i,e in enumerate(threads):
    print 'thread{} {}frames'.format(i,e)

edited May 22 '12 at 06:33

answered May 22 '12 at 01:18

dawg

98,345
23
131
206

just as an addon...How would I format the results as a render flag...meaning...the render uses the format "-f frame range". so for example:
thread1 2frames~ "1-2" thread2 2frames~ "3-4" thread3 2frames~ "5-6" thread4 2frames~ "7-8" thread5 1frames~ "9-9" thread6 1frames~ "10-10" thread7 1frames~ "11-11" thread8 1frames~ "12-12" ... – にか May 22 '12 at 03:43
@にか: Google `python string concatenation` or `python string template` or `python string format` You would loop over the list I created, `threads`, and create each string using the value in the list. I cannot tell from your comment what you want more specifically. – dawg May 22 '12 at 05:55

score 0 · Answer 4 · answered May 22 '12 at 19:20

This is what I ended up using....Thanks for you help everyone.

frames=20 
tPos=16 
Ts=divmod(frames,tPos) 
threads=[] 
for i in range(tPos): 
    threads.append(Ts[0]+1 if i<Ts[1] else Ts[0])

start = 1
end = 0
x=1
while x <= (tPos):
    end = start +(threads[x-1]-1)
    print (start, "-", end)
    start = end + 1
    x+=1


prints:
1 - 2
3 - 4
5 - 6
7 - 8
9 - 9
10 - 10
11 - 11
12 - 12
13 - 13
14 - 14
15 - 15
16 - 16
17 - 17
18 - 18
19 - 19
20 - 20

a lil bit of programming math

4 Answers4