Mongrel in JRuby


As I have told you before, we have been working on getting Mongrel working in JRuby off and on for a long while. One or two months ago, I got the Ragel definition ported correctly and the rest working. The plan is to cooperate with Zed and create a JRuby-native Mongrel gem out there, but until then I will describe the steps you need to take to get this working for yourself.

First of all, you need to install gem_plugin. The easiest way is to use gem:

jem install -y gem_plugin --no-rdoc --no-ri

After this is done, you can either download Mongrel-support from svn://rubyforge.org/var/svn/jruby-extras/trunk/mongrel-support, and build the extension manually, or you can download it directly here: http://opensource.ologix.com/http11.jar. If you decide to build it yourself, you first need a current version of JRuby from trunk. Check out mongrel-support from the Subversion repository and execute these commands:

cp $JRUBY_HOME/lib/jruby.jar lib
ant jar

After that you will have a http11.jar-file in lib. Regardless of how you get the http11.jar-file, place it in $JRUBY_HOME/lib/ruby/site_ruby/1.8/. Then you can proceed in a few different ways. You could copy the Mongrel-files from your MRI installation, or you could download Mongrel from version control. Regardless of how, you need to copy mongrel.rb, mutex_fix.rb and the mongrel-directory from Mongrel into $JRUBY_HOME/lib/ruby/site_ruby/1.8/. You can also copy bin/mongrel_rails into $JRUBY_HOME/bin and change the shebang to point to /usr/bin/env jruby. After this, you are good to go with JRuby and Mongrel.



On ActiveRecord-JDBC performance


I have been a bit concerned about the performance of our component that connects ActiveRecord with JDBC. Since ActiveRecord demands that every result of a select should be turned into a big array of hashes of strings to strings, I suspected we would be quite inefficient at this, and I wasn’t sure I could put all my faith in JDBC either.

So, as a good developer, I decided to test this, with a very small microbenchmark, to see how bad the situation actually was.

Since I really wanted to check the raw database and unmarshalling performance, I decided to not use ActiveRecord classes, but do executions directly. The inner part of my benchmark execution looks like this:

conn.create_table :test_perf, :force => true do |t|
t.column :one, :string
t.column :two, :string
end

100.times do
conn.insert("INSERT INTO test_perf(one, two) VALUES('one','two')")
end

1000.times do
conn.select_all("SELECT * FROM test_perf")
end

conn.drop_table :test_perf

It is executed with a recent MySQL Community Edition 5 server, locally, with matching JDBC drivers. The MRI tests is run with 1.8.6, and both use ActiveRecord 1.15.3. ActiveRecord-JDBC is a prerelease of 0.2.4, available from trunk. My machine is an IBM Thinkpad T43p, running Debian. It’s 32bit and Java 6.

The results were highly interesting. First, let’s see the baseline: the Ruby results:

      user     system      total        real
7.730000 0.020000 7.750000 ( 8.531013)

Frankly, I wasn’t that impressed with these numbers. I thought Ruby database performance was better. Oh well. The interesting part is the JRuby AR-JDBC results:

      user     system      total        real
6.948000 0.000000 6.948000 ( 6.948000)

WOW! We’re actually faster in this quite interesting test. Not what I had expected at all, but very welcome news indeed. Note that there is still much block overhead in the interpreter, so the results are a little bit skewed in MRI’s favour by this, too.



SuperRedCloth


RedCloth is cool. SuperRedCloth is cooler, faster, nicer, and all in all such a treat. And it works on JRuby now. At the time of writing, I haven’t really wired together a gem of it yet, but I hope to get it done soon. Until then, if you want to try this, you can download the necessary jar-file from http://opensource.ologix.com/superredcloth_scan.jar, install the (very small) SuperRedCloth lib-file into your JRuby, add the superredcloth_scan.jar into your site_ruby for JRuby and everything should be set to go. Be warned, this requires a very recent trunk version of JRuby (something like newer than April 1:st).

SuperRedCloth is another of those nice Ragel based libraries. I’m finding Ragel such a joy to work with; just change the few C-thingies into Java-thingies and everything works exactly the same. Neat.



Some JRuby updates


So, there is prime time for JRuby now. So much, in fact, that I haven’t had time to blog properly for a while. And won’t have for a few days more. In the meantime, check what we’ve been busy with: http://headius.blogspot.com/2007/03/activerecord-100-performance-doubling.html.



Post Rails meetup


Last evening was very nice. We usually have a good time at the Rails meetups, hosted by Valtech. Yesterday it started out with Dr Nic William, who spoke about some metaprogramming techniques. Very neat, and the best part was his Magic Wiggly Lines. I’m not sure if it’s genius or mental. But it’s cool, at least; it’s basically a const_missing hook, which makes it possible to use class names even if you spell them wrong. Hehe.

After that I spoke about new JRuby thing. I have a cold, so I took it pretty easy, and it was very free form with questions. There was many interesting questions, actually, and I think Nic taped it all…

After that, we drank beer and talked technology, which is always fun!



Rails meet in Stockholm


And it’s time again for a Rails meetup in Stockholm, at Valtech. This Tuesday (20th) 6pm at Valtechs offices. It is usually very interesting and neat to meet other people involved in Rails and Ruby here in Stockholm, and it’s usually very nice, all around.

I will have a short presentation on the new features in JRuby 0.9.8. I will provide a fast intro to JRuby, but this is not the introductionary presentation; I will also try a somewhat different presentation technique, so it is sure to be interesting. At least for me… =)

Show up! It’s worth it. You need to reserve a place, though, which can be done at http://www.rails.se/rails/show/Railstr%C3%A4ff+20+Mars+2007



ThoughtWorks


I have resigned from my position at Karolinska Institutet. Instead, I will work for ThoughtWorks. I am extremely excited about this, obviously. I will start in London in the beginning of June. I’ll stay in London for 6 months and then move to the San Francisco office.

As you may know, ThoughtWorks is doing lots of Ruby projects in the enterprise and are really helping spread Ruby to the masses. And that’s what I will be working with. Ruby and JRuby for ThoughtWorks.

What’s more, I will be doing development on the JRuby core at ThoughtWorks!

As an aside, I would like to mention two very interesting things that TW have announced very recently. First, CruiseControl.rb, which looks very nice, and secondly, Mingle. Both of these products have great potential.

I’m just so excited about this!



JRuby Regular Expressions


The Regular Expression support in JRuby is about to be revamped. I will here detail my plans for this work, and also some of the reasons for it. This post is as much for people interested in JRuby, as for the JRuby developers.

Stage 0: java.util.regex
JRuby has traditionally used java.util.regex. We stopped doing that March 11:th, 2007. The main reasons are because of the disconnect with MRI. Some of the operators work very differently, there are some problems with UTF-8, we can’t support SJIS or EUC, nor posix-classes. And java.util.regex also uses a recursive implementation which means it can’t handle certain large inputs. Further, we would like to be able to modify the implementation to work with the same stuff that backs the RubyString, to increase performance.

Stage 1: JRegex
Yesterday (March 11:th), I merged JRegex as the main regular expression engine for JRuby. The main reasons for this is twofold. First, we can change the implementation quite easily, and second, it is an iterative algorithm, which means it doesn’t fail on the input that java.util.regex does. Since this caused problems with some Rails tests (and also in multipart handling in all libraries using cgi), I decided to merge this as a stopgap until the next incarnation of regex support.

Stage 2: REJ
In about 2 weeks, I hope to be able to merge REJ with JRuby. At the point of merging, it should be a better replacement than both JRegex and java.util.regex. REJ is a project I’ve started, which will be a direct port of the MRI 1.8.6 regular expression engine. The important thing about this is that the semantics for JRuby will match MRI very closely. We will be able to match UTF-8, SJIS and EUC regular expressions, and we are able to have the same quirks as MRI, even though people shouldn’t depend on such quirks. In the process of writing REJ, I will also create a large suite of test cases for regular expressions, based on Henry Spencer’s test files. I’ll probably submit something initial to a separate repository very soon. If I get my wish, REJ is what will be the regular expression engine for JRuby 1.0.

Stage 3: Ojiguruma
After 1.0 has been released, I think it’s time to make the Regexp engine in JRuby really extensible, and provide an interface from Ruby to change which engine to use. After that is done, I would be very interested in doing a port of Oniguruma to Java, which would give us far better multilanguage support, and also some interesting features. The reason I’m choosing to not do this right now is because Oniguruma is just too large.

Stage 4: (No official name yet)
Another engine that some in the JRuby/Ruby community has started working on is an engine which will be based on Ragel for parsing and a modified version of Thompson NFA and TCL-style backreferences for matching. It’s an interesting project but it will take some time before it’s usable.



Two more days in Kraków


I was very lazy on Saturday. Took it very easy, then went to a dinner with some of the presenters and organizers for SFI. And after that I gave my presentation, which I personally felt went very well. I had to adapt the technical level some, because it seemed most of the audience didn’t know much about compilers. After that, Lukas held his presentation about Seaside, which also went well.

Then we celebrated. Then there was headache. And a look at the Cathedral. And a taxi driver who didn’t understand one word of English. And he took me to the wrong terminal at the airport. But I finally got on the right flight and got home.



Two days in Kraków


So, I’ve been in Kraków for two days now. I will fly back on Sunday. It’s a very interesting town. On one part, it’s lovely. The buildings are absolutely amazing, and very beautiful. On the other hand, many of them are very run down, and the wear and tear is obvious all around. To me, the whole town seems sort of depressing, but on the other hand the people here are very up beat, and from some of the presentations I’ve seen, the technological future for Kraków looks very bright indeed.

I haven’t really been able to see most of the presentations. Almost all have been in Polish, sadly enough. I did see two interesting ones yesterday. The first was Michael Foords talk about IronPython. This was very neat and I’ve talked some with him after that too. I’ll get back to that. The other talk was about Google Ads, by Greg Badros. As always, hearing about how Google does things internally is always amazingly interesting. I would have enjoyed hearing more about the machine learning and NLP stuff they’re doing, but obviously they can’t discuss that too much.

Today I haven’t been able to see any presentation, due to Polish. On the other hand me, Michael and Lukas Renggli have had some very interesting discussions both yesterday and today. Now, the company Michael’s part of has the largest IronPython code base in the world, as far as I understand it. It’s basically 90 000 LOC, where 20 000 is production code and 70 000 are testing code. That sounds about right… =)

Lukas Renggli is one of the core developers of Seaside, a framework which I’m quite fond of. What we three have in common is our interest in dynamic languages, so we had some very common ground to talk about. What’s nice is that many of the things Michael talked about in his presentation is stuff we in JRuby also speak much about. In the same manner, it seems the IronPython guys have had basically the same problems we in JRuby has had. It seems there is some common ground to be found here, and possibly also a basis for conversations. I for one would find that very interesting, since I enjoy hearing about dynamic languages getting foot holds on statically typed virtual machines; that is just such a sweet spot.

Tomorrow evening is my talk. I’m going to be at an official dinner in the middle of the day, and then get over to the conference and give my presentation at 17:00. I think it’s going to be interesting, and it will be an obvious counterpart to Michael’s talk, since the reason for existing is so similar for IronPython and JRuby. It will be interesting to see how the audience react.

After that, I’ll see Lukas talk about Seaside, and after that people are talking about having a party, so tomorrow will be quite intense. And finally, on Sunday I’m going back to Sweden. As far as I know now, I won’t travel that much for at least a month now, which feels sort of nice. This week has been far to intense, and also have had some really great – but tiring – moments.