Next Wednesday (the 15th) I will present about JRuby in Bergen. I mentioned this in an earlier post. More information can be found here: http://www4.java.no/web/show.do?page=42;7&appmode=/showReply&articleid=5493.
One of the major problems when running automated testing with JRuby is that all the standard Test::Unit assertions would never be JIT compiled, meaning that they would be quite slow. Actually, assertions seems to be very slow when running interpreted in JRuby. I have a small test case, courtesy of Michael Schubert:
require 'test/unit'
require 'benchmark'
class A < Test::Unit::TestCase
[10_000, 100_000].each do |n|
define_method "test_#{n}" do
puts "test_#{n}"
5.times do
puts Benchmark.measure{n.times{assert_equal true,true}}
end
end
end
end
This code will show quite nicely how large the overhead of asserts are, by using assert_equal. Now, the numbers for MRI for this benchmark looks like this:
Loaded suite test_assert
Started
test_10000
0.150000 0.000000 0.150000 ( 0.155817)
0.150000 0.000000 0.150000 ( 0.158376)
0.160000 0.000000 0.160000 ( 0.155575)
0.150000 0.000000 0.150000 ( 0.154380)
0.160000 0.000000 0.160000 ( 0.157737)
.test_100000
1.520000 0.010000 1.530000 ( 1.539325)
1.530000 0.000000 1.530000 ( 1.543889)
1.520000 0.010000 1.530000 ( 1.540376)
1.530000 0.000000 1.530000 ( 1.543742)
1.530000 0.010000 1.540000 ( 1.558292)
.
Finished in 8.509493 seconds.
2 tests, 550000 assertions, 0 failures, 0 errors
And for JRuby without compilation:
Loaded suite test_assert
Started
test_10000
1.408000 0.000000 1.408000 ( 1.408000)
0.582000 0.000000 0.582000 ( 0.582000)
0.425000 0.000000 0.425000 ( 0.426000)
0.419000 0.000000 0.419000 ( 0.418000)
0.466000 0.000000 0.466000 ( 0.467000)
.test_100000
4.189000 0.000000 4.189000 ( 4.190000)
4.196000 0.000000 4.196000 ( 4.196000)
4.139000 0.000000 4.139000 ( 4.139000)
4.165000 0.000000 4.165000 ( 4.165000)
4.162000 0.000000 4.162000 ( 4.162000)
.
Finished in 24.181 seconds.
2 tests, 550000 assertions, 0 failures, 0 errors
It’s quite obvious that something is very wrong. We’re about 2.5-3 times slower.
Now, the way the JRuby compiler works, we build it piece by piece and the JIT will try to compile a method that’s used enough. If there is any node that can’t be compiled it will fail and fall back on interpretation. In the case of assertions, all Test::Unit assertions use a small helper method called _wrap_assertion that looks like this:
def _wrap_assertion
@_assertion_wrapped ||= false
unless (@_assertion_wrapped)
@_assertion_wrapped = true
begin
add_assertion
return yield
ensure
@_assertion_wrapped = false
end
else
return yield
end
end
When I started out on this quest, there were two things in this method that doesn’t compile. The first is the ||= construct, which I mentioned in an earlier blog post. The problem with it is that it requires that we can compile DefinedNode too, and that one is large. The second problem node is Ensure. After lots of work, I’ve finally managed to implement most of these safely, and falling back on interpretation when it’s not safe. Without further ado, the numbers after compilation with these features added:
Loaded suite test_assert
Started
test_10000
0.996000 0.000000 0.996000 ( 1.013000)
0.415000 0.000000 0.415000 ( 0.415000)
0.110000 0.000000 0.110000 ( 0.110000)
0.099000 0.000000 0.099000 ( 0.100000)
0.109000 0.000000 0.109000 ( 0.109000)
.test_100000
1.012000 0.000000 1.012000 ( 1.012000)
1.008000 0.000000 1.008000 ( 1.000000)
1.017000 0.000000 1.017000 ( 1.017000)
1.039000 0.000000 1.039000 ( 1.039000)
1.024000 0.000000 1.024000 ( 1.024000)
.
Finished in 6.966 seconds.
So we’re looking at over 4 times improvement in speed, and about 33% percent faster than MRI. Try your test cases; hopefully it will show up.
Next week I’m coming to Norway to do two presentations at the Java User Groups in Bergen and Oslo. I will visit Bergen on the 15th, and Oslo on the 16th. I will have 2×45 minutes to talk, so there will lots of time for interesting detours, and hopefully some discussion too. If you’re in the area, do show up! More information about the Oslo evening can be found here: http://www4.java.no/web/show.do?page=42;7&appmode=/showReply&articleid=5447.
After Norway I’m going to Sweden. I’ll be in Gothenburg for a few days, Malmö a few more, and then stop by in Stockholm the 23rd to 26th. I’m not going to present while there, though. It’s purely recreational.
So, after some discussion on the #jruby IRC channel, the core team has decided to go with a 5+ strategy on trunk. The reasons for this is that almost everyone who commented on the issues advised us to move on, and the features of 5 is quite compelling. We are moving to annotations for describing Java method bindings, we will use enumerations for many values, and we will be able to use the native concurrency libraries. All of these are large wins.
Of course, we are not leaving 1.4 users completely behind. First of all, we provide a Retrotranslator/Retroweaver (we haven’t decided which one yet) version, and will continue doing so. Secondly, the 1.0 branch will continue supporting Java 1.4.
This is the current state. First versions of the annotation based method bindings are already in trunk.
I’ve been spending some time trying to implement a compiler for the defined?-feature of Ruby. If you haven’t seen it, be happy. It’s quite annoying, and incredibly complicated to implement, since you basically need to create a small interpreter especially just for nodes existing within defined?. So why is defined? so important? Well, for one it’s actually needed to implement the construct ||= correctly. And that is used everywhere, which means that not compiling it will severely impact our ability to compile code. Also, it just so happens that OpAsgnOrNode (as it’s called), and EnsureNode, are the two nodes left to implement to be able to compile Test::Unit assert-methods, since the internal _wrap_assertion uses both ensure and ||=.
So, now you know why. Next, a quick intro to the compilation strategy of JRuby. Basically we try to compile each script and each method into one Java method. We try to use the stack as much as possible, since we in that way can link statements together correctly. And that’s about it.
The problem enters when you need to handle exceptions in the emitted Java bytecode. This isn’t a problem in the interpreter, since we explicitly return a value for each node, and the interpreter doesn’t use the Java stack as much as the compiler does. We also want to be able to use finally blocks at places, especially to ensure that ensure can be compiled down, but also to make the implementation of defined? safe.
So what’s the problem? Can’t we just emit the catch-table and so on correctly? Well, yes, we can do that. But it doesn’t work. Because of a very annoying feature of the JVM. Namely, when a catch-block is entered, the stack gets blown away. Completely. So if the Ruby code is in the middle of a long chained statement, everything will disappear. And what’s worse, this will actually fail to load with a Verifier exception, saying “inconsistent stack height”, since there will now be one code path with things on the stack, and one code path with no values on the stack, and the way JRuby works, these will end up at the same point later on. And the JVM doesn’t allow that either.
This makes it incredibly hard to handle these constructs in bytecode, and frankly, right now I have no idea how to do it. My first approach was to actually create a new method for each try-catch or try-finally, and just have the code in there instead. The fine thing about that is that the surrounding stack will not be blown away since it’s part of the invoking method, and not in the current activation frame. And that approach actually works fairly well. Until you want to refer to values from outside from the try or catch block. Then it breaks down.
So, right now I don’t know what to do. We have no way of knowing at any specific place how low the stack is, so it’s not possible to copy it somewhere, and then restore it in the catch block. That would be totally inefficient too. In fact, I have no idea how other implementations handle this. There’s gotta be a trick to it.
Every time my feed reports an update on XKCD, I get all happy. It’s without a doubt the best online comic. The blend is perfect. Just take a look at todays:
How perfect isn’t that?
I landed in New York this morning, and I’ll stay until Thursday. If anyone feels like taking a beer and discussing programming languages (or something else), don’t hesitate to mail me or comment on this blog.
Martin Fowler writes about one language, and neatly encapsulates what I think about the subject in the way I would have written it, if I were actually a good writer. Go read.
I have been asked for the slides of my FOSCON presentation. They can be downloaded in PDF format from here: http://ologix.com/JRubyWhirlwindTour.pdf.
As you almost certainly now at this time, IronRuby was released the beginning of this week (read more here, here and here). Now, this of course begs a few question. But first things first: note that Microsoft have decided to host the project on RubyForge, and will do so sometime next month. Not only that, it’s actually a very open source license called MsPL: Microsoft Permissive License. If you look at the license, you will see that it’s almost exactly a BSD license, except that the patent restrictions have been clarified. Obviously, I’m not a lawyer, but I can say that this license seems very nice and will allow good usage of IronRuby in many cases.
The code base is strictly limited right now. The things that actually work is most of the core language structure, the Array and String classes and .NET Interop. Of course, that is an extremely large achievement to have this working so well. Also, the compiler seems to generate very good code and judging from microbenchmarks, it has the possibility of having very good performance. It’s important to remember that many of the more important corner cases aren’t supported yet, and it is these that usually drags the performance of Ruby implementations down. I’m confident that Lam and company can handle this, though.
Overall, I would say that this is looking very good. I would recommend .NET people to take a look at it, and also try to contribute. Very soon, you will be able to contribute code back to everything except the core compiler and DLR.
Finally, the question on everyones mind: when will IronRuby be finished? I don’t know, but I think it can happen with 6-12 months. The concerns raised two months ago have been resolved internally by Microsoft, which makes IronRuby a very real and important project.
|
I am a software developer working for Centro de AutonomÃa Digital, mainly with privacy, security and cryptography issues. I have worked on many different things in my life, including programming language design and implementation, security, privacy and cryptography, artificial intelligence and more. |
|
Quito |