Monday, April 28, 2008

Some Rails perf numbers on JRuby 1.1.1

UPDATE: Next up will be the same benchmarks using Java 6 on ubuntu.

Just before JRuby 1.1.1 was released, I was surfing around and found this post which shows JRuby running basic Rails bench tests slower than MRI. I wondered what those numbers would look like now, so I got to testing.

I started by downloading the latest binary release of JRuby here.

I setup a Rails app following the theme mentioned by the blog post above, then ran the console benchmark 6 times with each runtime and list the lowest time below:
[sourcecode language='css']
jruby -J-server script/console production
>> Benchmark.measure {10000.times {Person.find :first}}.total
=> 10.352999925613403

ruby script/console production
>> Benchmark.measure {10000.times {Person.find :first}}.total
=> 4.98

Now I wanted to see if the ActiveRecord-JDBC drivers would help the situation:
[sourcecode language='css']
jruby -S gem install activerecord-jdbcmysql-adapter
vi config/database.yml ("mysql" -> "jdbcmysql")

jruby -J-server script/console production
>> Benchmark.measure {10000.times {Person.find :first}}.total
=> 4.134999990463257

console results

Sped things up quite a bit... Now for Mongrel:
[sourcecode language='css']
> mongrel_rails start -e production
Requests per second: 202.27 [#/sec] (mean)

jruby -S gem install mongrel
jruby -J-server -S mongrel_rails start -e production
Requests per second: 96.47 [#/sec] (mean)

# now with adapter: jdbcmysql about the same... got a bit faster after warm-up
Requests per second: 120.05 [#/sec] (mean)

MRI's Mongrel beats JRuby version even after a little warm-up period. I also wanted to see what GlassFish would do :
[sourcecode language='css']
jruby -S gem install glassfish

jruby -S glassfish_rails start ../person_test/
Requests per second: 132.93 [#/sec] (mean)

# now with jdbcmysql adapter
jruby -S glassfish_rails start ../person_test/
Requests per second: 139.96 [#/sec] (mean)

Just goofing around, I used an experimental setting to see what would happen. I repeated the benchmark for some time and you can then really start seeing the JVM 'harden' the execution path... I'm sure the tests above may have benefited in the same way had I let them run but here are the numbers of a glassfish_rails server with 2 Rails instances (probably ran the benchmark about 10-15 times?)
[sourcecode language='css']
jruby -J-server -J-Djruby.compile.fastest=true -S glassfish_rails start -n 2 ../person_test/
Requests per second: 211.64 [#/sec] (mean)

I also wanted to see some numbers for Tomcat 6, a war'd Rails app using Warbler. The numbers were not at all what I was expecting... even though I know things out of the box are not tuned for this type of deployment. The best numbers I could get were:
[sourcecode language='css']
Requests per second: 67.68 [#/sec] (mean)

Rails perf chart

So what does this tell me? JRuby is getting faster each release. There is more work to be done to make JRuby on Rails apps faster. GlassFish is a very easy way to deploy JRoR apps. And last, I am definitely doing something wrong with my war'd depoyment!

I could not find much else in the way of JRuby on Rails benchmarks... if you know of any, please link in comments!

All these benchmarks were run on a MacBook 2GHz Core Duo with 1GB mem., Java 5, JRuby 1.1.1, Ruby 1.8.6, Ruby on Rails 2.0.2, MySQL 5.0.27.


LeonB said...

Nice post!

I always read Charles Nutter recommending Java 6 for running jruby. It's supposedly much faster with jruby. Maybe nice to add to the tests?

ANdrew said...

I heared JRuby is quite faster on JVM 6.

Charles Oliver Nutter said...

Very nice graphs. I've learned from this post that really BIG blocks provide better visual cues.

And the numbers look pretty good. I can't wait to see your Java 6 numbers. Of course there's always more work to do; we should be faster than this.

Mike McKinney said...

Charles: just used Google Docs Spreadsheet and the newly integrated graphs...

Huikau? » Blog Archive » Some Rails perf numbers on JRuby 1.1.1 part 2 (Linux, JDK 6) said...

[...] this post, I got a lot of suggestions to re-run the benchmarks using JDK 6. I’ll let you read the [...]

Huikau? » Blog Archive » JRuby slowdown from Rails 2.0.2 to 2.1.0 said...

[...] you may want to take a look at the performance. I was running the console benchmark from this blog post with jruby trunk and saw some numbers I wasn’t [...]