Shotgun Rewrite Underway

Some big changes are underway on the Rubinius VM at present: Shotgun is being completely rewritten! This change was brought about by some fairly significant rework required in order to change the behavior of argument evaluation in method calls.

Currently, Rubinius evaluates method arguments from right-to-left, whereas MatzRuby and JRuby evaluate arguments left-to-right. So code like the following:

a = [1,2,3]
foo(a.shift, a.shift, a.shift)

evaluates to foo(1,2,3) in MatzRuby and JRuby, but to foo(3,2,1) in Rubinius.

While it is generally considered unwise to rely on argument evaluation order (languages such as C specify that argument evaluation order is undefined, and at the discretion of the compiler writer), it turns out there is a significant base of Ruby code that does in fact depend upon this behavior, not the least being Rails’ ActiveSupport.

As a result, it was decided to rework Rubinius to also evaluate method arguments in left-to-right order. This requires changes to both the compiler and to the Shotgun VM, since the order in which arguments are passed on the stack now needs to be reversed.

From C to C++

While making the necessary changes to Shotgun, a tipping point was reached, and Evan decided to bite the bullet and re-write Shotgun in C++. The reasons he gave for this decision were as follows:

  1. Tests!: Shotgun had evolved from an initial prototype, and unlike the rest of the Rubinius code base, had very little in the way of test coverage. The substantial changes required to the VM internals to accommodate the argument order reversal, and the lack of tests to validate the changes, was the single biggest factor leading to the decision. The new VM aims to have 100% test coverage using CxxTest.
  2. Modularity: The opportunity provided by starting fresh, as well as the better code organisation capabilities (classes, namespaces, etc) provided by C++ mean that the new VM will be more modular, which should make it easier to extend and maintain.
  3. Better match between VM and Ruby semantics: The use of an object-oriented language for the VM provides a better semantic match to Ruby. Language support for method chaining, exceptions, and so forth mean that the VM implementation will more closely mirror the semantics of Ruby. This (combined with a cleaner architecture) should make it more understandable to Rubinius contributors, as well as potentially a better target for Garnet/Duby style code generation than C.
  4. STL: Many of the built-in types required by the VM (tuple, array, hash, list, etc) can be built on classes provided by the C++ Standard Template Library.
  5. Stronger typing: C++ is a stronger typed language than C, and this should help reduce problems such as the “Attempted field access on non-reference” errors that often occur when working on the VM.

The decision to rewrite Shotgun was a big one, and will certainly set back progress a little in the short-term. However, the cleaner architecture, test coverage, and other advantages accruing from the change should pay-off substantially over time.


3 Responses to “Shotgun Rewrite Underway”

  1. I’m sure there is a reason such a seemly simple change will initiate a rewrite but it still seems surprising.

  2. I’m looking forward to run this new version of the VM. And Evan also confirmed that the C++ rewrite did effectively reduce the amount of lines of code. The better semantic match of C++ means less boilerplate repetitive code than C. Neat, event less non-Ruby in Rubinius 🙂

  3. Guess I’m surprised. It will “Reduce” the lines of code, but increase the overhead. Now looks like I need to write my own vm to push it into small environments.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: