huikau?

After this post, I got a lot of suggestions to re-run the benchmarks using JDK 6. I’ll let you read the previous post to see what exactly I did… here’s what the runtime looks like for this run:

• Ubuntu 7.10 / Dell Latitude D830
• Ruby 1.8.6
• JRuby 1.1.1
• JDK 1.6.0_03

*NOTE: could not run benchmarks without jdbc drivers (jdbcmysql adapter) due to JRUBY-2303

First the Rails console benchmark numbers:

5.15 seconds - Ruby console
2.41 seconds - JRuby console (jdbc)

(side-note: re-running the console test with JDK 5 on same machine, best out of 6 runs: 3.31 seconds)

Next the Web Server numbers: (numbers in parens are the number of benchmark runs)

272.84  Mongrel (6)
257.02  GlassFish (40)
235.17  Mongrel (jruby/jdbc) (20)
188.12  Glassfish (-n 2, 20)
179.76  Mongrel (jruby/jdbc) (10)
168.52  Glassfish (6)
161.12  Mongrel (jruby/jdbc) (6)
151.04  Glassfish (-n 2, 6)
131.89  Tomcat 6 (30)
105.72  Tomcat 6 (6)

The console numbers show JDK 6 really kicking in… contrast that with the numbers for actually serving the Rails app and you can see there is still some work to be done to make Ruby on Rails serve faster on the JVM.

Again, not very scientific, just something to show where things stand for now. If you have better benchmarks, please link to them from the comments section below.

Posted in Programming

UPDATE: Next up will be the same benchmarks using Java 6 on ubuntu.

Just before JRuby 1.1.1 was released, I was surfing around and found this post which shows JRuby running basic Rails bench tests slower than MRI. I wondered what those numbers would look like now, so I got to testing.

I started by downloading the latest binary release of JRuby here.

I setup a Rails app following the theme mentioned by the blog post above, then ran the console benchmark 6 times with each runtime and list the lowest time below:

jruby -J-server script/console production
>> Benchmark.measure {10000.times {Person.find :first}}.total
=> 10.352999925613403

ruby script/console production
>> Benchmark.measure {10000.times {Person.find :first}}.total
=> 4.98

Now I wanted to see if the ActiveRecord-JDBC drivers would help the situation:

jruby -S gem install activerecord-jdbcmysql-adapter
vi config/database.yml ("mysql" -> "jdbcmysql")

jruby -J-server script/console production
>> Benchmark.measure {10000.times {Person.find :first}}.total
=> 4.134999990463257

Sped things up quite a bit… Now for Mongrel:

> mongrel_rails start -e production
Requests per second:    202.27 [#/sec] (mean)

jruby -S gem install mongrel
jruby -J-server -S mongrel_rails start -e production
Requests per second:    96.47 [#/sec] (mean)

# now with adapter: jdbcmysql about the same... got a bit faster after warm-up
Requests per second:    120.05 [#/sec] (mean)

MRI’s Mongrel beats JRuby version even after a little warm-up period. I also wanted to see what GlassFish would do :

jruby -S gem install glassfish

jruby -S glassfish_rails start ../person_test/
Requests per second:    132.93 [#/sec] (mean)

# now with jdbcmysql adapter
jruby -S glassfish_rails start ../person_test/
Requests per second:    139.96 [#/sec] (mean)

Just goofing around, I used an experimental setting to see what would happen. I repeated the benchmark for some time and you can then really start seeing the JVM ‘harden’ the execution path… I’m sure the tests above may have benefited in the same way had I let them run but here are the numbers of a glassfish_rails server with 2 Rails instances (probably ran the benchmark about 10-15 times?)

jruby -J-server -J-Djruby.compile.fastest=true -S glassfish_rails start -n 2 ../person_test/
Requests per second:    211.64 [#/sec] (mean)

I also wanted to see some numbers for Tomcat 6, a war’d Rails app using Warbler. The numbers were not at all what I was expecting… even though I know things out of the box are not tuned for this type of deployment. The best numbers I could get were:

Requests per second:    67.68 [#/sec] (mean)

So what does this tell me? JRuby is getting faster each release. There is more work to be done to make JRuby on Rails apps faster. GlassFish is a very easy way to deploy JRoR apps. And last, I am definitely doing something wrong with my war’d depoyment!

I could not find much else in the way of JRuby on Rails benchmarks… if you know of any, please link in comments!

P.S.
All these benchmarks were run on a MacBook 2GHz Core Duo with 1GB mem., Java 5, JRuby 1.1.1, Ruby 1.8.6, Ruby on Rails 2.0.2, MySQL 5.0.27.

Posted in Programming