4

I'm using ruby-head and Debian wheezy x64. When I run a multithreaded ruby script, htop shows that it's using multiple cores visually with the bars at the top, and that it's using 100% CPU on the process list but it's only using 100% of the capacity of one core. I assume it's possible to have multiple cores running at 100% and this number seems too convenient to be bottle-necked by either the program logic or another hardware aspect. Is the OS limiting the amount of available instructions I'm using, if so how do I stop this?

EDIT more info:

When I mean visually using multiple cores e.g.: 47% core 1, 29% core 2, and 24% core 3. These percentages are constantly shifting up and down and to different sets of cores, but always collectively add up to 100%-102%. More than 3(/8 total) cores are being used, but any cores other than the three most burdened only utilize 2% or less capacity. I guess I should also mention this is a linode VPS.

EDIT:

Well it looks like I was reading promises that 2.0 would feature true parallel threads, and not actual release information. Time to switch to Jruby...

gtech
  • 65
  • 1
  • 6

2 Answers2

10

You failed to mention which Ruby implementation you are using. Not all Ruby implementations are capable of scheduling Ruby threads to multiple CPUs.

In particular:

  • MRI implements Ruby threads as green threads inside the interpreter and schedules them itself; it cannot schedule more than one thread at a time and it cannot schedule them to multiple CPUs
  • YARV implements Ruby threads as native OS threads (POSIX threads or Windows threads) and lets the OS schedule them, however it puts a Giant VM Lock (GVL) around them, so that only one Ruby thread can be running at any given time
  • Rubinius implements Ruby threads as native OS threads (POSIX threads or Windows threads) and lets the OS schedule them, however it puts a Global Interpreter Lock (GIL) around them, so that only one Ruby thread can be running at any given time; Rubinius 2.0 is going to have fine-grained locks so that multiple Ruby threads can run at any given time
  • JRuby implements Ruby threads as JVM threads and uses fine-grained locking so that multiple threads can be running; however, whether or not those threads are scheduled to multiple CPUs depends on the JVM being used, some allow this, some don't
  • IronRuby implements Ruby threads as CLI threads and uses fine-grained locking so that multiple threads can be running; however, whether or not those threads are scheduled to multiple CPUs depends on the VES being used, some allow this, some don't
  • MacRuby implements Ruby threads as native OS threads and uses fine-grained locking so that multiple threads can be running on multiple CPUs at the same time

I don't know enough about Topaz, Cardinal, MagLev, MRuby and all the others.

Jörg W Mittag
  • 363,080
  • 75
  • 446
  • 653
4

MRI implements Ruby Threads as Green Threads within its interpreter. Unfortunately, it doesn't allow those threads to be scheduled in parallel, they can only run one thread at a time.

See similar question here

Community
  • 1
  • 1
rudolph9
  • 8,021
  • 9
  • 50
  • 80
  • I thought ruby 2.0 fixed that. Also I'm not sure your answer makes sense because I'm clearly seeing the OS displaying multiple cores being utilized. – gtech Oct 09 '13 at 21:54
  • Nope, the [global interpreter lock](http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpreter-lock/) in ruby is a lot of work to make compatible with true multi-threading. – rudolph9 Oct 09 '13 at 21:59
  • 1
    Ah, I'm sorry. Looks like I was only reading promises of the feature. Time to switch to Jruby. – gtech Oct 09 '13 at 22:01
  • The link @rudolph9 provided to [global interpreter lock](http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpreter-lock) is 404 due to a formatting error. This comment contains a fixed link. – Excalibur May 17 '16 at 20:10
  • @Excalibur thanks for letting me know I'll update my post. – rudolph9 May 18 '16 at 14:41