Am I using Julia right?

Question

I started using Julia a couple of months ago, deciding to give it a try after weeks of hearing people praise various features of it. The more I learned about it, the more I liked its style, merging ease of expressing concepts in high-level language with a focus on speed and usability. I implemented a model that I'd also written in C++ and R in Julia, and found the Julia version ran much faster than the R version, but still slightly slower than C++. Even so the code was more legible in Julia than it was in both other languages. This is worth a lot, and in particular as I've generalized the model, the amount of work to do to broaden the scope of the Julia code was far less than the comparable amount of worked threatened to be in the other languages.

Lately, I have been focused on getting my Julia code to run faster, because I need to run this model trillions of times. In doing so I've been guided by @code_warntype, @time, @profile and ProfileView and the track-allocation flag. Great. This toolkit isn't as nice as some other languages' profiling tooling, but it still pointed out a lot of bottlenecks.

What I find is that I have in my code precisely the high-level expressivity that I like in Julia, and when I rewrite that expressivity to avoid unnecessary allocations, I lose that very expressivity. As a trivial example, I recently changed a line of code that read

sum([x*y for x in some_list, y in similar_list])

to a loop that iterates over the lists and adds to a state variable. Not rocket science, and I understand why it's a little faster not to have to allocate the array. Actually it's a lot faster though. So I've done similar things with avoiding using Dicts or the composite types that "feel" right for the problem, when I can instead just manually keep track of indices in temporary parallel arrays, a style of coding I abhor but which evidently runs a lot faster when I'm repeating the particular operation of creating and briefly using a small data structure lots and lots of times.

In general, this is largely fine because I've taken to heart the instructions to write short methods, so the higher-level methods that compose behavior of my own shorter methods don't need to "worry" about how the shorter methods work; the shorter ones can be clunky to read without making the core of my program clunky to read.

But this has me wondering if I'm "missing something." If the whole point of the language (for me as a not-theory-concerned end user) is in part to combine speed with ease of development/thought/reading/maintenance etc., i.e. to be a writable and usable technical computing language, then doesn't that mean that I shouldn't ever be spending time thinking about the fastest way to add up a bunch of numbers, shouldn't be rewriting "easy to read" or elegant code that takes advantage of mappings, filters, and high-level function concepts into "clunky" code that reinvents the wheel to express those things in terms of low-level keeping track of array indices everywhere? (Some part of me expects a language with so much intelligence behind its design to be smart enough to "figure out" that it doesn't actually need to allocate a new array when I write sum([x*y]), and that I'm just too obtuse to figure out the right words to tell the language that except by literally "telling it" the whole loop business manually. At one point I even thought about writing @macros to "convert" some quickly expressed code into the long but faster loopy expressions, but I figured that I must be thinking about the problem wrong if I'm essentially trying to extend the compiler's functions just to solve fairly straightforward problems, which is what led me to write this question.)

Perhaps the answer is "if you want really performant code, you pay the price no matter what." Or put differently, fast code with annoyingly unpleasant to read loops, arrays, keeping track of indices, etc. is on the efficient frontier of the speed-legibility tradeoff space. If so, that's perfectly valid and I would not say therefore that I think any less of Julia. I'm just wondering if this sort of programming style really is on the frontier, or if my experience is what it is because I'm just not programming "well" in the language. (By analogy, see the question What is your most productive shortcut with Vim? , where the accepted and excellent answer is essentially that the OP just didn't "get" it.) I'm suspicious that even though I've succeeded in getting the language to do much of what I want, that I just don't "get" something, but I don't have a good sense of what to ask for as the thing I fear I don't get is an unknown unknown to me.

TL;DR: Is it expected for best-practice in Julia that I spend a lot of time "decomposing" my high-level function calls into their primitive implementations in terms of loops and arrays in order to get faster performance, or is that indicative of me not thinking about programming in / using the language correctly?

Answer 1

I think this topic is closely in a line with a discussion that was already going on the Julia-users group Does Julia really solve the two-language problem? and I would like to cite here one paragraph from that discussion:

@Stefan Karpinski:

There are different styles and levels of writing code in every language. There's highly abstract C++ and there's low-level pointer-chasing C++ that's basically C. Even in C there's the void*-style of programming which is effectively dynamically typed without the safety. Given this fact, I'm not sure what solving the two language problem would look like from the perspective this post is posing. Enforcing only one style of programming sounds like a problem to me, not a solution. On the contrary, I think one of Julia's greatest strengths is its ability to accommodate a very broad range of programming styles and levels.

My own experience in Julia programming, shows that it can fill the blank box of a modern programming language that could bring high level features like parallel processing, socket server and ... in the hand of scientists, engineers and all computation gurus and pragmatic programmers that want to do their work in an efficient, maintainable and readable manner, using an all-in-one programming language.
In my opinion you are using Julia in the right way, Julia like other languages represents different styles of programming for different situations, you can optimize bottlenecks for speed, and keep the other parts more readable. also you could use tools like Devectorize.jl to avoid rewriting issue.

Answer 2

This is a difficult question to answer satisfactorily. Therefore I'll try to make a short contribution in the hope that a "pot" of different opinions might be good enough. So here are 3 opinions going through my mind currently:

1. A programming language may be likened to an object with momentum. The user-base being its mass and their styles exerting force on it. The initial users/developers can pull a language in a certain direction because it still has low mass. A language can still evolve even if extremely popular (e.g. C/C++), but it's harder. The future of Julia is still unwritten, but its promise is in the initial direction imbued by its creators and early users.

2. Optimization is best delayed until correctness is well tested. "Premature optimization is the root of all evil" (D. Knuth). It never hurts to remember this, again. So, keeping the code readable and correct is better until the optimization stage which may obfuscate bounded areas of the code.

3. The expression sum([x*y ...]) may require the compiler to be too clever, and it might be better to simply define a sumprod(x,y). This would allow sumprod to harness the generic function multiple dispatch framework of Julia and stay optimized for x and y and possibly later even more optimized for specifically typed x and y.

That's it, for now. Opinions are many. So let the discussion continue.

Answer 3

Background: I'm an R/Rcpp programmer who has been using Julia for about a month now. At least half the R code I've written calls C++ code that I've written.

Conclusion: I think Julia solves the two programming language problem as well as is possible. This doesn't mean that one can write high performance code as though it was native R/Python, but it greatly lowers the amount of work required in writing high performance that can be easily used by others.

Further Discussion: First of all, it's my belief that its simply not possible to write high performance code without some concern for C/C++ type issues, i.e., declaring types, thinking about efficient data structures rather than readable data structures, etc. So it's my view that if one is hoping that they will have a high performance language in which one doesn't need to care about data structures, they are out of luck. Of course, I'm a statistician and not a CS person, so I admit I could be wrong here.

However, I'm a bit amazed how much of my work time seems to be cut away by using Julia. In particular, a lot of my work involves first writing some C++ code for some computationally intense part. Once that's up and running, I'm basically writing R wrappers and data processors in native R to prepare data for my C++ code. Preprocessing is usually not computationally expensive and R has a ton of nice tools to do this with. If everything is meticulously planned ahead of time, this is theoretically not such a bad process.

However, in reality, what actually happens is I get everything up and running and then I realize I want to alter some of C++ code for some reason or another. This is where the huge headaches come in. Unfortunately, at this point I often end up with two versions of all my data; one that's passing things around to a bunch of native R preprocessing tools and another that's passing things around as C++ objects. Mucking with some C++ objects can have a lot of consequences for all the R objects. Honestly, I waste too much time trying to sort all this out.

My experiences with Julia so far is that this headache is gone. There's no longer an R version and C++ version of the data I'm working with. Just Julia. This also means it's really easy to introduce other people's code into the high performance part of my code. Just as a recent example, I've been wanting to work with the Matern covariance function. This is a non-trivial function and I don't feel like implementing it myself. I can find lots of people who have R packages with this implemented with nice, well documented calls in native R. However, in my case, I'd really like to call this from C++ without all the native R overhead. This means either (a) I need to search through everyone's not necessarily well documented, nor necessarily user friendly C++ code in their R package or (b) reinvent the wheel. On the other hand, if I was using Julia, it should be that if someone writes a nice Matern covariance function for the world to use, I should be able to use it directly rather than dig around in their code for the call I actually want to use.

It's possible that as I start building more complicated Julia projects, I may find some similar nuisances that I'm not aware of just yet. But so far, it seems to really improve the process of combining easy to use tools (i.e., R, Python-like calls) with high performance code.

Answer 4

Could you write a class that abstracts these notions for you (i.e. FastDict or FastList or whatever)? You would then have the same legibility (if a little more black-box-ish) while having performant code.