2

I've been looking at examples from other people but I can't seem to get it to work properly.
It'll either use a single core, or basically freeze up maya if given too much to process, but I never seem to get more than one core working at once.

So for example, this is kind of what I'd like it to do, on a very basic level. Mainly just let each loop run simultaneously on a different processor with the different values (in this case, the two values would use two processors)

mylist = [50, 100, 23]

newvalue = [50,51]

for j in range(0, len(newvalue)):

    exists = False
    for i in range(0, len(mylist)):

        #search list
        if newvalue[j] == mylist[i]:
            exists = True

    #add to list
    if exists == True:
        mylist.append(mylist)

Would it be possible to pull this off? The actual code I'm wanting to use it on can take from a few seconds to like 10 minutes for each loop, but they could theoretically all run at once, so I thought multithreading would speed it up loads

Bear in mind I'm still relatively new to python so an example would be really appreciated

Cheers :)

Peter
  • 3,186
  • 3
  • 26
  • 59
  • Maybe this answer could help you http://stackoverflow.com/questions/16470004/running-multiple-concurrent-python-programs-accessing-the-same-database-table/16470451#16470451 – Robert Lujo May 20 '13 at 21:45
  • Thanks, but that's mainly for databases, plus I did look at that python document page a bit earlier but it didn't work well with maya :P – Peter May 21 '13 at 11:42
  • Where would the code go that takes longer? Add a call to some function `f` somewhere to make it clearer which parameters it needs. You want to process the `newvalue` list concurrently, right? – Andreas Haferburg May 21 '13 at 20:41
  • @Peter the answer below is a very good answer; esp. the comments that entail. I would have accepted it :) – kartikg3 Jan 13 '15 at 20:10
  • Ah ok sure, I'll do it now, I think that this was my very first question here so that's my excuse for forgetting :P (also I was a bit disappointed that Maya can't easily use multiple cores haha) – Peter Jan 13 '15 at 20:41
  • Yeah great question too. I had the same questions and google led me here. – kartikg3 Jan 14 '15 at 01:33

2 Answers2

5

There are really two different answers to this.

Maya scripts are really supposed to run in the main UI thread, and there are lots of ways they can trip you up if run from a separate thread. Maya includes a module called maya.utils which includes methods for deferred evaluation in the main thread. Here's a simple example:

import maya.cmds as cmds
import maya.utils as utils
import threading

def do_in_main():
    utils.executeDeferred (cmds.sphere)

for i in range(10):
    t  = threading.Thread(target=do_in_main, args=())
    t.start()

That will allow you to do things with the maya ui from a separate thread (there's another method in utils that will allow the calling thread to await a response too). Here's a link to the maya documentation on this module

However, this doesn't get you around the second aspect of the question. Maya python isn't going to split up the job among processors for you: threading will let you create separate threads but they all share the same python intepreter and the global interpreter lock will mean that they end up waiting for it rather than running along independently.

You can't use the multiprocessing module, at least not AFAIK, since it spawns new mayas rather than pushing script execution out into other processors in the Maya you are running within. Python aside, Maya is an old program and not very multi-core oriented in any case. Try XSI :)

Any threading stuff in Maya is tricky in any case - if you touch the main application (basically, any function from the API or a maya.whatever module) without the deferred execution above, you'll probably crash maya. Only use it if you have to.

And, BTW, you cant use executeDeferred, etc in batch mode since they are implemented using the main UI loop.

Community
  • 1
  • 1
theodox
  • 12,028
  • 3
  • 23
  • 36
  • Thanks for the reply, just tried your code and it seems to be one of the few ones that work with maya haha, but basically, are you saying it'd be impossible to make it multithreaded? It's just one plugin I have has a multithreaded option (but it's a solver so gives off very different results if this is enabled), and that's an exclusive maya one, so surely those people must have found a way around it? – Peter May 21 '13 at 11:44
  • maya runs multithreaded as for several of the heavier operations (blendshape etc) as of 2009 and the api is mostly therad safe. However PYTHON can not use many processors well. In any case all treading needs to play nice with the min thread. But it certainly is possible to use plugins that thread, no problem. Besides mayas architecture is MUCH better suited for therading than XSI. So if you want to multiprocess use c – joojaa May 21 '13 at 15:54
  • Maya can run threads (for example, for rendering or simulation) but it's not designed from the bottom up for multiprocessing (or, for that matter for GPU acceleration - it was, after all, first written in 1996!) – theodox May 21 '13 at 15:56
  • Its been fully rewritten since the node based architecture is be design something that can be easily threaded. That is how most architectures that use multiple processes work to isolate the data such as Haskel. GPU accelereation is another thing. Even XSI sucks on thsi front – joojaa May 21 '13 at 15:56
  • 1
    I'm distinguishing between threading (maya spawns plenty of threads) and multiprocessing in response to OP's question. Maya uses OpenMP to get more cores for some things - especially simulations - but I don't think the core DAG graph evaluation is spread across cores. I certainly don't see processor spikes on more than one core when, say, playing back a scene with several complex rigs. XSI ICE, by comparison, is multiprocessor all the way done (or at least, that's what they claim). Some docs here: http://docs.autodesk.com/MAYAUL/2013/ENU/Maya-API-Documentation/index.html – theodox May 21 '13 at 17:14
  • 1
    Problem is that only so few nodes do anything so heavy that it matters. For maya the limiting factor is the ogl update. If you disable mayas ogl you will see that maya spikes much more with blendhapes and bones. So the gain is actually infinitesimal. The dg is somewhat multithreaded, but alas autodesk stopped the development halfway. To be honest ALL current 3d applications suffer form a old architecture. Even XSI is not much better. So you need a second generation of software to get any meaningful change in this. If you want to discuss more move this to a discussion. – joojaa May 21 '13 at 17:29
  • Alright, so TL;DR: Forget multithreading in Python unless you do I/O, forget multiprocessing from Python in Maya, if you need multithreading in Maya use C++, right? – Andreas Haferburg May 21 '13 at 21:20
  • 1
    In Maya Python, it's handy for anything that you don't want blocking UI (say, talking to a database or starting up a post-process on export). Anything else -- especially if you're doing big number crunching -- go through C++. – theodox May 21 '13 at 21:36
  • Thanks guys, if it can only be done in C++, I think i'll just let it drop and keep it single threaded lol, as I've found trying to learn a new language makes me forget how to use the old one haha. also, the script editor only has tabs for mel and python, I have a feeling any c++ would have to be coded outside of maya which would be hard – Peter May 22 '13 at 11:10
  • 1
    C++ would have to be written as a plugin. The SDK comes with your maya install, along with docs and examples -- but I avoid it whenever I can, since among other things you need to recompile for different versions of maya and bit depths. – theodox May 22 '13 at 15:00
1

What theodox says is still true today, six years later. However one may go another route by spawning a new process by using the subprocess module. You'll have to communicate and share data via sockets or something similar since the new process is in a seperate interpreter. The new interpreter runs on its own and doesn't know about Maya but you can do any other work in it benefitting from the multi-threaded environment your OS provides before communicating it back to your Maya python script.

SoothedTau
  • 43
  • 5
  • If done properly and carefully that's a good idea, though as a warning, where I work had a few issues with animbot doing that with their crash recovery. It'd spawn a new process when opening a scene, but would never close it. Once Maya was exited, each process lost its connection and ate up a full CPU core trying to find it again. – Peter Aug 07 '19 at 01:00