sum(a) is now 30% slower than NumPy

[stevengj]

I was updating my performance-optimization lecture notes from last year to Julia 1.0, which start with a comparison of C, Python, and Julia sum functions, and I noticed something odd:

Both the Julia sum(::Vector{Float64}) function and the NumPy sum function are faster than last year (yay for compiler improvements?). Last year, Julia and NumPy sum had almost identical speed, but now the NumPy sum function is now about 30% faster than Julia.

I'm running a 2016 Intel core i7, the same as last year. So apparently the NumPy sum function has gotten some new optimization that we don't have? (I did switch from Python 2 to Python 3; I'm using the Conda Python.) Some kind of missing SIMD optimization?

I'm not so concerned about sum per se, but this is a pretty basic function — if we are leaving 30% on the table here, then we might be missing performance opportunities in many other places too.

[andreasnoack]

I (and the audience) noticed this last week when I used the notebook in a presentation. Looks like Intel has been looking into this since I suspect this could be numpy/numpy#10251.

[KristofferC]

This PR numpy/numpy#11113 removes all the intrinsic stuff and says the compiler can do it by itself with the same performance. So not sure why we get less with LLVM then.

[stevengj]

Note that numpy11113 does not seem to specifically benchmark sum, so it could be that the intrinsics are still better than LLVM there? Or maybe Conda's numpy is using a different compiler (icc?) or different options?

[StefanKarpinski]

I've been seeing the same thing on a 2018 2.7 GHz Intel Core i7, so this is something that's widespread across a pretty wide range of Intel CPUs. Makes the "variations on sum" / "Julia is fast" summation notebook a bit of a sad trumpet demo since NumPy wins.

[JeffreySarnoff]

Are the summed values equal?

[SyxP]

Are the summed values equal?

Yes. They are within error for floating points, but not exactly equal as floats.

[andreasnoack]

Just an update. I don't think this is caused by the PR I linked to. I tried out older versions of Numpy and they are all (back to and including 1.11) as fast.

[tkf]

FYI, if you are using conda to install Numpy 1.11, you may be installing a very new build https://anaconda.org/anaconda/numpy/files?version=1.11.3 (says 6 days old). Anaconda could be using a newer/better compiler?

[andreasnoack]

I built the Numpy version from source with the default gcc (Apple's LLVM based version). The odd thing is that Numpy is also significantly faster than the C version regardless of flags I try to use.

[JeffreySarnoff]

Can you see the code generated by Numpy?
Annotating a Julia sum function's loop with @simd gives that ~30% speedup on my machine (until the number of summands > ~35,000, when the speedup starts to decrease).

[KristofferC]

If you look in the notebook you can see @simd is (of course) used.

[JeffreySarnoff]

I meant "do we know the instructions that Numpy generates to perform the summation and how those instructions (emitted assembler that is the summation loop) differ from what Julia via LLVM executes?"

[AzamatB]

Would be interesting to see whether updating Julia's LLVM to the latest version would solve the issue.

[c42f]

Oddly enough, the julia builtin sum doesn't vectorize on my machine. See also #30320 where there is some discussion of vectorization in the current mapreduce implementation.