I haven’t been a believer, to express it charitably. But I wanted to be certain, so I set out to devise my own experiment to test modern JS performance.
Up Front Summary
I was extremely surprised that my experiment demonstrated JS performance FAR beyond my expectations. There might be something to these claims of magnficent JS speed in numerical applications. Basically, here were my thoughts during the process:
- Here’s a straightforward C program to perform a simple yet numerically intensive operation.
- Let’s compile the C program on gcc and get some baseline performance numbers.
- Ha! Pitiful JS performance, just as I expected!
- Try the same program under Firefox, since Firefox is supposed to have some crazy optimization for asm.js code, allegedly emitted by Emscripten.
- LOL! Firefox performs even worse than Chrome!
- Wait a minute… the Emscripten documentation mentioned using optimization levels for generating higher performance JS, so try ‘-O1’.
- Umm… wow: Chrome’s performance increased dramatically! What about Firefox? Not only is Firefox faster than Chrome, it’s faster than the gcc-generated code!
- As my faith in C is suddenly shaken to its core, I remembered to compile the gcc version with an explicit optimization level. The native C version pulled ahead of Firefox again, but the Firefox code is still close.
- I proceed to try the same converted program on a variety of mobile platforms.
- The mobile platforms perform rather admirably as well.
- I am surprised.
I wanted to run a simple yet numerically-intensive and relevant benchmark, and something I am familiar with. I settled on JPEG image decoding. Again, I wanted to keep this simple, ideally in a single file because I didn’t know how hard it might be to deal with Emscripten. I found NanoJPEG, which is a straightforward JPEG decoder contained in a single C file.
I altered nanojpeg.c (to a new file called nanojpeg-static.c) such that the main() program would always load a 1920×1080 (a.k.a. 1080p) JPEG file (“bbb-1080p-title.jpg”, the Big Buck Bunny title), rather than requiring a command line argument. Then I used gettimeofday() to profile the core decoding function (njDecode()).
Compiling with gcc and profiling execution:
gcc -Wall nanojpeg-static.c -o nanojpeg-static ./nanojpeg-static
Optimization levels such as -O0, -O3, or -Os can be applied to the compilation command.
/path/to/emscripten/emcc nanojpeg-static.c -o nanojpeg.html \ --preload-file bbb-1080p-title.jpg -s TOTAL_MEMORY=32000000
The ‘–preload-file’ option makes the file available to the program via standard C-style file I/O functions. The ‘-s TOTAL_MEMORY’ was necessary because the default of 16 MB wasn’t enough. Again, the -O optimization levels can be sent in.
For running, the .html file is loaded (via webserver) in a web browser.
Want To Try It Yourself?
I put the files here: http://multimedia.cx/emscripten/. The .c file, the JPEG file, and the Emscripten-converted files using -O0, -O1, -O2, -O3, -Os, and no optimization switch.
Results and Charts
Here is the spreadsheet with the raw results.
I ran this experiment using Ubuntu Linux 12.04 on an Intel Atom N450-based netbook. For this part, I was able to compare the Chrome and Firefox browser results against the C results:
These are the results for a 2nd generation Android Nexus 7 using both Chrome and Firefox:
Here is the result for an iPad 2 running iOS 7 and Safari– there is no Firefox for iOS and while there is a version of Chrome for iOS, it apparently isn’t able to leverage an optimized JS engine. Chrome takes so long to complete this experiment that there’s no reason to muddy the graph with the results:
Interesting that -O1 tends to provide better optimization than levels 2 or 3, and that -Os (optimize for size) seems to be a good all-around choice.
Don’t Get Too Smug
Implications For Development