One of the major problems when running automated testing with JRuby is that all the standard Test::Unit assertions would never be JIT compiled, meaning that they would be quite slow. Actually, assertions seems to be very slow when running interpreted in JRuby. I have a small test case, courtesy of Michael Schubert:

require 'test/unit'
require 'benchmark'

class A < Test::Unit::TestCase
[10_000, 100_000].each do |n|
 define_method "test_#{n}" do
   puts "test_#{n}"
   5.times do
     puts Benchmark.measure{n.times{assert_equal true,true}}
   end
 end
end
end

This code will show quite nicely how large the overhead of asserts are, by using assert_equal. Now, the numbers for MRI for this benchmark looks like this:

Loaded suite test_assert
Started
test_10000
0.150000   0.000000   0.150000 (  0.155817)
0.150000   0.000000   0.150000 (  0.158376)
0.160000   0.000000   0.160000 (  0.155575)
0.150000   0.000000   0.150000 (  0.154380)
0.160000   0.000000   0.160000 (  0.157737)
.test_100000
1.520000   0.010000   1.530000 (  1.539325)
1.530000   0.000000   1.530000 (  1.543889)
1.520000   0.010000   1.530000 (  1.540376)
1.530000   0.000000   1.530000 (  1.543742)
1.530000   0.010000   1.540000 (  1.558292)
.
Finished in 8.509493 seconds.

2 tests, 550000 assertions, 0 failures, 0 errors

And for JRuby without compilation:

Loaded suite test_assert
Started
test_10000
1.408000   0.000000   1.408000 (  1.408000)
0.582000   0.000000   0.582000 (  0.582000)
0.425000   0.000000   0.425000 (  0.426000)
0.419000   0.000000   0.419000 (  0.418000)
0.466000   0.000000   0.466000 (  0.467000)
.test_100000
4.189000   0.000000   4.189000 (  4.190000)
4.196000   0.000000   4.196000 (  4.196000)
4.139000   0.000000   4.139000 (  4.139000)
4.165000   0.000000   4.165000 (  4.165000)
4.162000   0.000000   4.162000 (  4.162000)
.
Finished in 24.181 seconds.

2 tests, 550000 assertions, 0 failures, 0 errors

It’s quite obvious that something is very wrong. We’re about 2.5-3 times slower.

Now, the way the JRuby compiler works, we build it piece by piece and the JIT will try to compile a method that’s used enough. If there is any node that can’t be compiled it will fail and fall back on interpretation. In the case of assertions, all Test::Unit assertions use a small helper method called _wrap_assertion that looks like this:

def _wrap_assertion
 @_assertion_wrapped ||= false
 unless (@_assertion_wrapped)
   @_assertion_wrapped = true
   begin
     add_assertion
     return yield
   ensure
     @_assertion_wrapped = false
   end
 else
   return yield
 end
end

When I started out on this quest, there were two things in this method that doesn’t compile. The first is the ||= construct, which I mentioned in an earlier blog post. The problem with it is that it requires that we can compile DefinedNode too, and that one is large. The second problem node is Ensure. After lots of work, I’ve finally managed to implement most of these safely, and falling back on interpretation when it’s not safe. Without further ado, the numbers after compilation with these features added:

Loaded suite test_assert
Started
test_10000
  0.996000   0.000000   0.996000 (  1.013000)
  0.415000   0.000000   0.415000 (  0.415000)
  0.110000   0.000000   0.110000 (  0.110000)
  0.099000   0.000000   0.099000 (  0.100000)
  0.109000   0.000000   0.109000 (  0.109000)
.test_100000
  1.012000   0.000000   1.012000 (  1.012000)
  1.008000   0.000000   1.008000 (  1.000000)
  1.017000   0.000000   1.017000 (  1.017000)
  1.039000   0.000000   1.039000 (  1.039000)
  1.024000   0.000000   1.024000 (  1.024000)
.
Finished in 6.966 seconds.

So we’re looking at over 4 times improvement in speed, and about 33% percent faster than MRI. Try your test cases; hopefully it will show up.

No Comments | By Ola Bini | In: Uncategorized | tags: assertions, compilation, jruby, test/unit. | #

I’ve been spending some time trying to implement a compiler for the defined?-feature of Ruby. If you haven’t seen it, be happy. It’s quite annoying, and incredibly complicated to implement, since you basically need to create a small interpreter especially just for nodes existing within defined?. So why is defined? so important? Well, for one it’s actually needed to implement the construct ||= correctly. And that is used everywhere, which means that not compiling it will severely impact our ability to compile code. Also, it just so happens that OpAsgnOrNode (as it’s called), and EnsureNode, are the two nodes left to implement to be able to compile Test::Unit assert-methods, since the internal _wrap_assertion uses both ensure and ||=.

So, now you know why. Next, a quick intro to the compilation strategy of JRuby. Basically we try to compile each script and each method into one Java method. We try to use the stack as much as possible, since we in that way can link statements together correctly. And that’s about it.

The problem enters when you need to handle exceptions in the emitted Java bytecode. This isn’t a problem in the interpreter, since we explicitly return a value for each node, and the interpreter doesn’t use the Java stack as much as the compiler does. We also want to be able to use finally blocks at places, especially to ensure that ensure can be compiled down, but also to make the implementation of defined? safe.

So what’s the problem? Can’t we just emit the catch-table and so on correctly? Well, yes, we can do that. But it doesn’t work. Because of a very annoying feature of the JVM. Namely, when a catch-block is entered, the stack gets blown away. Completely. So if the Ruby code is in the middle of a long chained statement, everything will disappear. And what’s worse, this will actually fail to load with a Verifier exception, saying “inconsistent stack height”, since there will now be one code path with things on the stack, and one code path with no values on the stack, and the way JRuby works, these will end up at the same point later on. And the JVM doesn’t allow that either.

This makes it incredibly hard to handle these constructs in bytecode, and frankly, right now I have no idea how to do it. My first approach was to actually create a new method for each try-catch or try-finally, and just have the code in there instead. The fine thing about that is that the surrounding stack will not be blown away since it’s part of the invoking method, and not in the current activation frame. And that approach actually works fairly well. Until you want to refer to values from outside from the try or catch block. Then it breaks down.

So, right now I don’t know what to do. We have no way of knowing at any specific place how low the stack is, so it’s not possible to copy it somewhere, and then restore it in the catch block. That would be totally inefficient too. In fact, I have no idea how other implementations handle this. There’s gotta be a trick to it.

6 Comments | By Ola Bini | In: Uncategorized | tags: compilation, defined?, inconsistent stack height, jruby. | #

I guess the title says it all. Oh, it doesn’t? You want what? A tutorial on how to play with this? Well, alright then.

What you will need to get started is first and foremost JRuby trunk. Download and build that. You will also need either a very recent version of YARV or Ruby trunk. Download either and build that too. I have the source in $YARV_SRC and $RUBY_TRUNK_SRC. I’ve also installed them with suffixes, so I have ruby-yarv and ruby-trunk globally available. That’s a quite good setup.

Before compiling some Ruby, we need to change the change the compile-script slightly. So open up the file in $YARV_SRC/tool/compile.rb and disable the peephole_optimization and inline_const_cache, since JRuby doesn’t really support these features yet. If you use Ruby trunk, you also need to do a global search-and-replace in this file, from YARVCore to VM.

Now you’re finished to compile something. The goal for this execution have been to get an iterative fibonacci sequence running, so save this code in a file called fib.rb:

def fib_iter_ruby(n)
  i = 0
  j = 1
  cur = 1
  while cur <= n
    k = i
    i = j
    j = k + j
    cur = cur + 1
  end
  i
end

puts fib_iter_ruby(5000)

Next, compile this file:

ruby-yarv $YARV_SRC/tool/compile.rb -o fib.rbc fib.rb

Modify as needed if you use Ruby trunk instead. Now, how to run this? With the y-flag to JRuby:

jruby -y fib.rbc

As you can see, simple files like this just work. Of course, there are much missing yet, but basic interpretation works really nice. Some more additions to the YARV machine will come quite soon, probably. But the next step will be to implement a compiler for YARV too. From that point it should be possible to execute Ruby-files in JRuby in several different ways; by regular interpretation, YARV compilation, YARV interpretation, Java bytecode AOT compiling, Java bytecode JIT compiling and also possibly Rubinius interpretation. The future is, as always, interesting.

1 Comment | By Ola Bini | In: Uncategorized | tags: compilation, interpretation, jruby, ruby, yarv. | #