43

Here is the only way I know to ask it at the moment. As Understand it Scala uses the Java Virtual Machine. I thought Jruby did also. Twitter switched its middleware to Scala. Could they have done the same thing and used Jruby?

Could they have started with Jruby to start with and not had their scaling problems that caused them to move from Ruby to Scala in the first place? Do I not understand what Jruby is? I'm assuming that because Jruby can use Java it would have scaled where Ruby would not.

Does it all boil down to the static versus dynamic types, in this case?

johnny
  • 19,272
  • 52
  • 157
  • 259

7 Answers7

40

Scala is "scalable" in the sense that the language can be improved upon by libraries in a way that makes the extensions look like they are part of the language. That's why actors looks like part of the language, or why BigInt looks like part of the language.

This also applies to most other JVM languages. It does not apply to Java, because it has special treatment for it's basic types (Int, Boolean, etc), for operators, cumbersome syntax that makes clear what is built in the language and what is library, etc.

Now, Scala is more performatic than dynamic languages on the JVM because the JVM has no support for them. Dynamic languages on JVM have to resort t reflection, which is very slow.

Daniel C. Sobral
  • 295,120
  • 86
  • 501
  • 681
  • +1, this answer is more precise than mine. Fwiw, the JVM is getting better support for dynamic languages. – Gabe Moothart Jun 04 '09 at 02:53
  • 5
    The designers of JRuby are constrained by the fact that they are implementing Ruby, which is dynamically typed. Scala is statically typed. Being statically typed means the types of every identifier and expression are known at compile-time, enabling the computer to create fixed dispatch or dynamic dispatch of methods for a class. In the case of dynamic languages -- on the JVM! -- where types are not known at compile time, the code for each method call has to make use of reflection to search for a method name on the object, and only them call it. BTW, this is different than dynamic dispatch. – Daniel C. Sobral Jun 08 '09 at 16:40
  • 1
    "performatic"? I would edit this, but I think you just invented an awesome new word. – James McMahon Jul 31 '09 at 02:22
  • 1
    Yep, not an English word. Right now I can't think of a proper translation. :-) I'll eventually edit this. Or is there a badge for new words? :-) :-) – Daniel C. Sobral Jul 31 '09 at 16:26
  • 3
    For computer programming, we already have a made-up word: "performant" – Dan Passaro Feb 10 '10 at 21:31
  • *"Scala is more performatic than dynamic languages on the JVM because the JVM has no support for them"* ... is this statement correct? Since Java SE 7, the JVM has an `invokedynamic` instruction to help with supporting dynamic languages, as I understand it. – scottb Apr 23 '16 at 14:27
11

No, not really. It's not that the JVM is somehow magic and makes things scale by its magic powers; it's that Scala, as a language, is architected to help people write scalable systems. That it's on top of the JVM is almost incidental.

chaos
  • 122,029
  • 33
  • 303
  • 309
  • So when someone says that java scales well it's because it's a superior language? – johnny Jun 03 '09 at 16:18
  • I do not understand. I thought that it was not the language so much as the interpreter, compiler, etc. You don't see to be saying that. – johnny Jun 03 '09 at 16:26
  • 3
    You're right, I'm saying the exact opposite: that the structure of a language influences the concepts people express with it and how they express them, just as in natural language semiotics. The compiler/interpreter are important, don't get me wrong, but they're not why Scala has the properties it does, or every Tom-Dick-and-Harry language built over the top of the JVM would be a world-famous paragon of scalability. – chaos Jun 03 '09 at 16:28
  • 1
    I think you guys are conflating performance and scalability. They are different things. – jshen Jun 03 '09 at 16:42
  • I don't know. I'm only at surface level on this (thus my question). All I know is that Twitter went from Ruby to Scala on the middleware because it "scaled." I read that Scala uses the JVM so I wondered by they didn't start with Jruby that also uses the JVM. – johnny Jun 03 '09 at 16:51
  • Performance and stability are separate things only to the same extent that speed and acceleration are different things, i.e. not much. Acceleration is change in speed over time; scalability is change in performance over scale. – chaos Jun 03 '09 at 16:54
  • In the real world, scalability is about the code you write, not the language you write it in. Dumb algos, bad DB queries, etc are the issue 99% of the time...special circumstances notwithstanding. – DarkSquid Jun 03 '09 at 17:05
  • In the real world, particular tools are better for particular purposes. Yes, it's always easy for a bad programmer to write bad code. And sure, a sufficiently good programmer could write Twitter in Fortran, just as if I really wanted to, I could cut down a tree with a butter knife. That doesn't make the tools involved neutral to the equation. – chaos Jun 03 '09 at 17:10
  • @chaos, wouldn't the compiler take care of language constructs and generate an improved version of the code in bytecode? shouldn't a for-each in java and an each + block in jruby be equivalent? – Pablo Fernandez Jun 03 '09 at 17:48
  • for the record, I know this is not true, but why? – Pablo Fernandez Jun 03 '09 at 17:50
  • language performance and scalability are not analogous to speed and acceleration. Ruby is slower than scala, but this is roughly a constant in terms of scalability. For the same of argument let's say that ruby needs 10 boxes for every 8 that scala needs for the same load. They scale the same, ruby simply has a higher constant as they stand today. Btw, having a different constant means that they scale the same! – jshen Jun 03 '09 at 17:54
5

You have to separate out different meanings of scaling:

  1. Scaling in terms of growing the number of requests per second that can be handled with a proportionate increase in hardware
  2. Scaling in terms of growing a code base without it becoming a tangled mess

Scala helps on the first point because it compiles to Java bytecode that's really similar to Java, and therefore usually has the same performance as Java. I say "usually," because Scala there are some cases where idiomatic Scala causes large amount of boxing to take place where idiomatic Java would not (this is slated to change in Scala 2.8).

Performance is of course different than scaling. Equivalent code written in JRuby would scale just as well, but the slope of the line would be steeper - you'd need more hardware to handle the same number of requests, but the shape of the line would be the same. But from a more practical perspective the performance helps because you rarely can scale in a perfectly linear fashion with respect to adding core or especially servers and having better performance slows the rate at which you have to add capacity.

Scala helps with the second point because it has an expressive, compile-time enforced type system and it provides a lot of other means for managing the complexity of your code, such as mixins. You can write spaghetti code in any language, but the Scala compiler will tell you when some of the noodles are broken while with JRuby you'll have to rely solely on tests. I've personally found that for me Python breaks down at about 1000 closely related LOCs, and which point I have to refactor to either substantially reduce of the LOCs or make the structure more modular. Of course this refactoring would be a good idea regardless of what your language, but occasionally the complexity is inherent. Dealing with a large number of tightly couple LOCs isn't easy in any language, but it is much easier in Scala than it is in Python, and I think the analogy extends to Ruby/JRuby as well.

Erik Engbrecht
  • 3,174
  • 17
  • 23
5

I don't really think that the language is the biggest problem here. Twitter grew insanely fast, which always leads to a code mess. And if you do a rewrite, it is a good idea to go for a different language - that bars you from building your own mistakes again and/or to "reuse some parts". Also, Ruby is not really meant for that kind of heavy data handling that the twitter backend does. The frontend remains Ruby, so they still use it.

Skade
  • 1,406
  • 1
  • 11
  • 14
4

Scala is a statically typed language. JRuby is dynamically typed. That is why Scala is faster than JRuby, even though both run on the JVM. JRuby has to do a lot of work at runtime (method resolution, etc.) that Scala does at compile-time. For what it's worth, though, JRuby is a very fast Ruby implementation.

Gabe Moothart
  • 31,211
  • 14
  • 77
  • 99
  • 1
    Is this true? Is that the only reason why it performs better? – Pablo Fernandez Jun 03 '09 at 17:09
  • Performance is complicated, but it is certainly the primary reason that Scala performs better. – Gabe Moothart Jun 03 '09 at 17:23
  • 2
    No, this isn't an absolute truth. the jvm itself is a good counter-example. You can do a lot of optimizations at runtime that can't be done at compile time. – jshen Jun 03 '09 at 17:23
  • Doesn't Ahead of time compilation take care of this? – johnny Jun 03 '09 at 17:38
  • are you talking about jit? If so, that can be done on dynamic languages as well. – jshen Jun 03 '09 at 17:49
  • @jshen there are certainly lots of tricks you can do to speed up dynamic languages. But can you name a single one that is faster than a modern statically-typed language? – Gabe Moothart Jun 03 '09 at 17:55
  • Lisp is dynamically typed, compiled, and can be extremely fast. – RHSeeger Jun 03 '09 at 19:28
  • The JVM is primarily optimized to run Java (no surprises here). The Scala compiler takes care of generating bytecode as similar as possible to Java – GClaramunt Jun 03 '09 at 22:08
  • @gabe this is a hard thing to measure for real world uses, but some of the common lisps are really fast as well as some of the recent javascript engines. There has also been a lot more work put into the static languages, but I expect that to change. – jshen Jun 04 '09 at 02:23
  • Isn't JRuby written in Java which is run on the JVM? Vs. Scala running natively on the JVM? – Bijan May 13 '12 at 04:19
4

Scalability is not an inherit language capability. You are talking about speed.

A better question to ask would be "Why is Scala faster than other JVM languages (or is it)?". As others have pointed out, it's a static vs. dynamic language thing.

wmoxam
  • 41
  • 2
3

There's an interesting discussion from the Twitter developers themselves in the comments of this post. They've evaluated the different options and decided to implement the back-end in Scala because: it ran faster than the Ruby/JRuby alternatives and they felt they could benefit from static typing.

GClaramunt
  • 3,148
  • 1
  • 21
  • 35
  • 1
    Very interesting posts there. The one from Charles Nutter (the Jruby project lead) comes off as really bitter about them not choosing Jruby. – James McMahon Jul 31 '09 at 02:32