Implementing a Jenkins Extension Point with the Native Java API inside a Ruby Plugin

In which I elaborate why the idomatic Ruby API is sometimes not enough, and describe a method to harness the full power of the underlying Jenkins API while still happily coding your extension in Ruby

The Ruby API

One of the primary goals of the effort to bring Ruby to Jenkins is to enable developers to extend Jenkins in a way that looks and feels like a normal Ruby environment. This means using rake, bundler ERB, and plain old Ruby objects to roll your plugin.

For example, the native BuildStep class is made available through the Ruby BuildStep. They are similar to be sure, but the Ruby object is much less complicated, and actually bears no relation to its Java equivalent inside the Java hiearchy of inheritance.

Exposing this simplicity is a worthy goal, but to do so requires careful consideration of each Jenkins Java API and how to provide its functionality in the Ruby way –no mean feat given the breadth of the Jenkins.

Sadly, it means that these Ruby-friendly APIs must necessarily lag behind their Java counterparts.

This can be a serious source of frustration for those looking to dive into Jenkins Ruby development right now. Initial excitement is quickly dulled when you find out that the extension point that you wanted to implement is unavailable from Ruby land. You might get discouraged and feel like you might as well be coding your plugin with Java.

Well don’t lose heart! I’d like to share with you a super easy way to write your extension points in Ruby even when there isn’t a friendly wrapper. We’ll actually implement a very handy extension point called RunListener within a Ruby plugin even though it is not part of the official Ruby API. We’ll do it by scripting the Java class directly with JRuby’s Java integration feature, and then register it directly with the plugin runtime.

The Extension Point

We’ll be working with the Jenkins RunListener interface. This is a wonderful extension point that allows you to receive callbacks at each point during the actual running of a build. There’s currently no nice ruby API for it, but we won’t let that stop us.

First, let’s create a new plugin called my-listener. We’ll do this with the jpi new command.

legolas:Jenkins cowboyd$ jpi new my-listener
    create  my-listener/Gemfile
    create  my-listener/my-listener.pluginspec

Fun fact: ‘jpi’ is an acronym for (J)enkins (P)lug-(I)n. You can install the tool with rubygems: gem install jpi

Next, we’ll cd into our new plugin and create our listener class inside the models/ directory. Jenkins will automatically evaluate everything in this directory on plugin initialization.

legolas:Jenkins cowboyd$ cd my-listener/
legolas:my-listener cowboyd$ mkdir models
legolas:my-listener cowboyd$ touch models/my_listener.rb

Our ultimate goal here is to implement a RunListener, so let’s go ahead and start our class definition inside that file.

class MyListener < Java.hudson.model.listeners.RunListener
  def initialize()
    super(Java.hudson.model.Run.java_class)
  end
end

There’s a couple key takeaways here. First, notice that we use JRuby integration to extend the class hudson.model.listeners.RunListener directly. Second, and this is a gotcha anytime you extend a Java class: you must invoke one of the Java super constructors if it does not have a default constructor. I can’t tell you how many times I’ve been bitten by this.

In our case, the RunListener class filters which jobs it will provide callbacks for by class. By providing a more specific class to the constructor, you limit the scope of jobs you’ll receive to subclasses of that class. For our purposes, we cast a pretty wide net by selecting all builds via the AbstractBuild Java class.

Pro Tip: when you’re implementing a native Java API, it really helps to have the javadoc open in one window so that you can view the documentation and crib from the source

Now that we’ve got our class defined, let’s implement some methods! We’ll add callbacks for when a build is started and when it’s completed.

class MyListener < Java.hudson.model.listeners.RunListener
  def initialize()
    super(Java.hudson.model.AbstractBuild.java_class)
  end
  def onStarted(run, listener)
    listener.getLogger().println("onStarted(#{run})")
  end
  def onCompleted(run, listener)
    listener.getLogger().println("onCompleted(#{run})")
  end
end
Jenkins.plugin.register_extension MyListener.new

And finally, on the last line, we actually inform Jenkins about the existence of our new Listener with the call to Jenkins.plugin.register_extension

And that’s about it. We can start up our test server with our jpi tool to see our listener in action.

legolas:my-listener cowboyd$ jpi server
Listening for transport dt_socket at address: 8000
Running from: /Users/cowboyd/.rvm/gems/jruby-1.6.5/gems/jenkins-war-1.446/lib/jenkins/jenkins.war
...
Jan 16, 2012 12:46:15 AM ruby.RubyRuntimePlugin start
INFO: Injecting JRuby into XStream
Loading /Users/cowboyd/Projects/Jenkins/my-listener/models/my_listener.rb
INFO: Prepared all plugins
...
INFO: Jenkins is fully up and running

To view the output, create a freestyle build called HelloWorld that doesn’t have any build steps at all, build it and view the console output. You should see something like this:

Started by user anonymous
onStarted(#<Java::HudsonModel::FreeStyleBuild:0x2870068a>)
onCompleted(#<Java::HudsonModel::FreeStyleBuild:0x2870068a>)
Finished: SUCCESS

The Sweet Reality

Even though we were dealing more directly with Java, we were still able to use all of the simplicity that comes with developing plugins in Ruby. Furthermore, even though our extension point was just scripted Java, it doesn’t mean that inside its methods it can’t call as much pure Ruby code as its heart desires.

I hope that if you’re considering writing your next (or your first!) Jenkins plugin in Ruby, you’ll feel confident that you can always fall back to the native APIs at any point.

Ruby for Jenkins Goes Pre-Alpha

2011-09-13T00:00:00-07:00

To quote Dave Bowman, “something wonderful happened” last week during the weekly Jenkins-Ruby hack session.

We were able to boot a plugin written in pure Ruby into a Jenkins server just by executing a single command from the command line (rake server). Furthermore, this command did not involve a compilation step, and the plugin that it booted did not bare a trace of evidence that it was to be loaded into a server written in Java.

That’s right. Using nothing but good ‘ol Bundler and Rake, we hoisted a pure Ruby plugin into the Jenkins runtime. And I can tell you that after hacking on this for almost a year: it felt… so…. good…. man.

It’s not time to declare victory yet, but at least the first two of our major goals have been realized, and that is definitely a cause to celebrate.

Practically, this means that rubyists can check out the source and get hacking without needing any knowledge of Java or any Java toolchain. That’s why I think it’s safe to officially declare the project to be in a pre-alpha state.

It isn’t yet for the faint of heart, and you’re sure to find many things confusing and difficult. But at least those who know only Ruby can contribute bug reports, identify pain points, and contribute to the creative process.

It’s 8:20am, but it still feels like Miller time.

The Ruby Racer isn't threadsafe... yet.

~~In which I explain problems unique to running The Ruby Racer in a multithreaded environment, which users are affected by these problems, and what is to be done for them.~~
UPDATE: This issue has been resolved with version 0.9.1. You should be able to stop reading here, update your Gemfile, and have a great day. That said, please don’t let me discourage you from continuing down the page. It’s riveting.

There have been a number of crashes reported in several different places for which The Ruby Racer is the culprit. For the most part, these are people running an Rails 3.1 application in a multithreaded webserver such as WEBrick, but as we’ll see, it could be anybody that is using The Ruby Racer in a mulithreaded environment. In order to speak to these and future issues in one place, as well as prevent the spread of any misinformation or impressions, I wanted to very quickly explain what exactly is causing these crashes, who is affected by them and where, and finally how we’re going to make them go away.

Background

When an instance of V8 starts running, it associates all of its global state with the currently running native OS thread. By global state, I mean things like heap storage, stacks, callbacks… everything that it needs to make JavaScript happen. Every time you eval() a script or you allocate an object with new, it retrieves that state from the executing thread to perform that operation.

In a single threaded application, that’s the end of the story. This is because every V8 operation happens from the same thread as the rest of your code, so the V8 state is always right there ready at all times. If you’re running your app in a rack server like Passenger or Unicorn that uses a multi-process model to achieve concurrency, you’ll never see this crash because you don’t use threads. Also, if you’re running on MRI 1.8.7 which uses green threads, you will also not be affected. While you may have more than one Ruby thread in your application, you still only have one native OS thread which is what V8 will use.

The problem comes in when you are using Ruby 1.9 in a multi-threaded environment and you access V8 from more than one thread. This is because 1.9 uses native OS threads under the covers for its own threading API. When The Ruby Racer starts up V8, it uses whatever thread happens to be running as the home for its runtime data. Then, when you try and do some JavaScript operation from another thread, V8 can’t find its state, freaks out and crashes your entire process.

Luckily, V8 does give us some help here. It provides a locker API as a way to deal with threading issues. It acts as both mutex on a V8 instance while at the same time providing a facility to access it from different threads. We can wrap calls that touch V8 with a Locker in order to prevent these crashes. e.g.

V8::C::Locker() do
  #... code that calls V8
end

That sounds simple enough: just never access V8 without a lock and you’re good. In fact, ExecJS does this whenever it invokes its therubyracer runtime, and it works… almost.

I would have gotten away with it too if it hadn’t been for you pesky GCs

If ExecJS locks V8, then why am I still crashing, and at seemingly random intervals?

The problem comes when we need to run garbage collection. The awesome thing about The Ruby Racer is that it lets you hold on to V8 objects from Ruby. It does this by using a proxy –a ruby object that implements all its behavior by delegating to an underlying JavaScript object. As long as Ruby holds a reference to that proxy, the proxy will in turn hold a reference to its underlying V8 object. However, when Ruby no longer holds a reference to the proxy and the proxy is garbage collected, we need to tell V8 that we aren’t holding a reference to the underlying object anymore. And, as you may have guessed, in order to remove references to V8 objects, you need to hold what? Yep, the V8 lock. To operate properly, GC also needs to lock V8. Sadly, it does not, which explains why crashes continue to happen at random places.

You might be asking yourself, “why not just lock V8 in your GC routines and have done with it?” The answer is that it would be a terribly dangerous and irresponsible thing to do.

Suppose that thread A has V8 locked and is doing some fancy JavaScript stuff. At some point, we do a context switch over to thread B which is not doing anything even remotely JavaScriptful like loading ActiveRecords from the database or something. Thread B is chugging threw lots of memory, and all of a sudden triggers GC. There are some V8 objects that need cleanup due to our JavaScript activities over in thread A, so it tries to lock V8, but uh-oh, thread A already holds the V8 lock, so GC may have to wait forever until it becomes available. Deadlock!

The answer is that instead of immediately releasing v8 objects inside the GC thread, we need to equeue them somewhere where they can be released. This probably means starting a thread per V8 instance to consume this reference queue and release them in the context of a V8 lock that isn’t contending for any other resources. While the answer is not trivial, I don’t expect it to be that complex.

What to do? What to do?

In the mean time, there is a workaround for locking V8 with each request, which will ensure that GC running in that request will have already acquired the V8 lock. Bear in mind that this is not a silver bullet, and you still could encounter crashes and deadlocks, but they should be very few and far between. If, as is the case with most people, you are just using a multithreaded server in your development environment but deploy to multi-process model, then this should be sufficient. If on the other hand you run multi-threaded in production, then you should definitely not be locking V8 in your middleware since this will effectively synchronize every request. Not a good idea.

No matter which category you fall into, I expect that we’ll have a beta fix by the end of this week, and if all goes well, a patch release after another. This may seem like a long time for such a small thing as coordinating locking with GC, but my experience tells me that threading is not a thing best done by humans, and so there will inevitable stumbles and cursing.

Also, I’m keen to make sure that any solution is both invisible to you the developer, as well as compatible with other Ruby versions and implementations. This deserves some thought.

What? you’re still here?

Alright, time to stop talking and get started with the repairs. I just wanted to take a moment to let you know what the problem was, why it was happening, how’re you’re affected, what you can do now, and when you can expect a more permanent fix.

What it takes to bring Ruby to Jenkins

2011-05-12T00:00:00-07:00

“Jenkins Ruby Plugins” are at an important, yet fragile stage of their life. Over the past several months, we have made tremendous progress towards making extending Jenkins with nothing but Ruby a blissful reality. However, at this moment, they exist only as a collection of hacks, or, to frame it more kindly, “proofs of concept” all buried quite deeply in a terribly named repo on github.

To name a few of them, we are now able to

automatically discover Ruby plugins and register them
work with Jenkins internals like BuildWrapper, RootActions, etc.. via a native Ruby-like API
persist mixed hierarchies of Java and Ruby objects transparently as Jenkins configuration
use ERB to template views, while still having access to the standard UI helpers
extend the Jenkins REST API from Ruby

This is by no means an exhaustive list, but I want to stress just how far we’ve come before I talk about where it is that we need to go.

While the work done so far represents some phenomenal accomplishments, in the end, they are only technical accomplishments which do not together create a coherent developer experience. So now, for this endeavor to succeed, we must focus our attention on making that entire experience, from project start to release after release, as frictionless as possible.

To that end, I propose the following high level goal for the Jenkins Ruby Plugins project:

To provide the facility to create, develop and release extensions for Jenkins with nothing but knowledge of the language, tools and best practices of the Ruby community.

This is, I think, the most important ideal to aspire to when claiming to provide a Ruby development experience. Unless it feels like the real McCoy, then you aren’t really developing Ruby at all, you’re just scripting Java with JRuby. You won’t fool anybody, and there will be far less enthusiasm and adoption.

That said, several things follow logically from this thesis, the first of which is:

Absolutely, Positively, No Maven!

This is not an ideological stance on the value of Maven as a ~~wretched pile of toad sick~~ software development tool. Rather, it is an explicit acknowledgement that it is not a Ruby tool, it doesn’t fit with the community’s style, and if you’re using it you might not be doing Java, but you certainly aren’t doing Ruby. Instead, we should turn to the Ruby tools that occupy this same space: Rake for scripting the build, and Bundler for dependency management.

Ruby-Like Project Structure

If we aren’t going to use Maven, then why should we be constrained by the project directory structure that it imposes. src/main/resources/what? Nope. Just one glance at Jenkins Ruby Plugin’s layout in an editor or file system browser should put a Rubyist’s mind right at ease. You see that? There’s your Gemfile just where you left him. And oh, your pals lib/ and spec/ are there too. Let’s get right to work!

Managing The Project Life-Cycle

From inception to release, you should be able to manage all aspects of your Jenkins plugin with Ruby. There must be single commands (or rake tasks) to

conjure a new plugin into existence
generate model/view boilerplate
run a server with the plugin loaded
package, push, tag and release a plugin to update-center

Testing

The importance of testing in the Ruby community cannot be overstated. The Ruby Tools need to provide a clear path to testing the plugin at both the unit and functional levels.

Documentation

The “Ruby-like” API that plugin development provides is similar, but not equivalent to its native Java equivalent. It is critical that this API be well documented, so that developers can understand it, and do not find themselves constantly referring to the JavaDoc, which they may very well not understand (it being Java), and might be misleading given the natural asymmetries of the two APIs.

Heed the Call!

There are plenty of people who would love to see this project get done, but don’t know where to start. I know this because I was one of them. The upshot is that now, as I’ve just described, there is plenty of work to be done that doesn’t require any knowledge of Java and very little knowledge of Jenkins internals. That means that you can contribute in a meaningful way!

Next week, I’ll be publishing some specifications in the form of Cucumber scenarios for some of the features up above that we can discuss and then hopefully start implementing.

But the most important thing that you can do is to join in on the conversation!

irc, and email are just a few of the ways to reach out!

Update. I have posted the first scenario: creating a plugin. The first bounty is officially posted.

Wrapping each Hudson distribution in its own RubyGem

Update: As of version 1.396, this is applicable to the Jenkins project. 1.395 will be the last version of the hudson-war gem

As you may or may not know, we here at The FrontSide have been working since late last year to create a proper ruby environment for extending hudson. This is no mean feat because it implies not only the ability to use nothing but ruby code to implement hudson plugins, but just as importantly, the ability to use nothing but ruby tools to develop them. That means no javac, no jar files, no war files, and for the love of all that is holy: absolutely, positively, no maven. In short, the only folks you’ll need to know to get in the door with Hudson are your pals Ruby, Rake and Bundler.

Of course, all those jars and wars and perhaps some class generation will be there behind the scenes. After all, we’ll still need to compile java stubs that depend on core Hudson classes. We’ll still need to package your plugin along with its ruby code and dependencies in a jar file, and we’ll still need to take care of all the blah, blah, MANIFEST, blah, META-INF, WEB-INF, blah, blah, blah…

The War

If we’re going to do the compilation against Hudson core from within ruby and Rake instead of maven, then we’re going to need a reliable way to get at the core hudson classes so that we can use them in our compilation classpath. Not only that, but as part of the development process, you’re going to want to test your plugins against a fully functioning hudson server. Luckily, both of these are contained in the full hudson distribution that you download as a single war file. Wouldn’t it be nice if you had a simple way to get at a specific version of one of those wars and then get at all those java goodies contained therein?

The Gem

To my thinking, this sounds like a job for: RubyGems! That’s why we’re introducing the new hudson-war gem, which does nothing except wrap a single hudson distribution so that it can be managed coherently just like any other ruby component. Want the latest warfile?

gem install hudson-war

Or perhaps you need hudson 1.386? That’s fine, hudson-war versions are the same as the hudson distributions they wrap.

gem install hudson-war --version 1.386

It’s not just a war file either. In keeping with the best traditions of OOP data comes bundled with behavior. The gem comes with a ruby API and a CLI baked in to help us do all the nifty things we might want to do with a war file. For starters, you might want to know where it is.

legolas:~ cowboyd$ hudson.war location
/Users/cowboyd/.rvm/gems/ruby-1.8.7-p174/gems/hudson-war-1.386/lib/hudson/hudson.war

If you want to unpack it to a particular location then

legolas:~ cowboyd$ hudson.war unpack tmp
jar xvf /Users/cowboyd/.rvm/gems/ruby-1.8.7-p174/gems/hudson-war-1.386/lib/hudson/hudson.war -C tmp/

You can run a test server with it

legolas:~ cowboyd$ hudson.war server --daemon
Forking into background to run as a daemon.
Use --logfile to redirect output to a file
legolas:~ cowboyd$ hudson.war server --kill

Beyond the basics, let’s say you want to do something interesting like compile some java code against the hudson core. You’re gonna need a classpath, and you can get it with:

legolas:~ cowboyd$ hudson.war classpath
/Users/cowboyd/.hudson/wars/1.386/WEB-INF/lib/hudson-core-1.386.jar

You can even compile java code with it directly. Here we compile a trivial class that has a dependency on hudson.

legolas:~ cowboyd$ cat > ImportsHudson.java 
import hudson.model.Hudson;
public class ImportsHudson {
  public static void main(String args[]) {
    System.out.printf("The Hudson Model object's class is %s\n", Hudson.class.getName());
  }
}
legolas:~ cowboyd$ hudson.war javac ImportsHudson.java
legolas:~ cowboyd$ java -cp .:`hudson.war classpath` ImportsHudson
The Hudson Model object's class is hudson.model.Hudson

It’s like RVM for hudson. Wrangling java has never been so fun!

The War Gemmer

Befitting a continuous integration project, the Hudson team releases a new version at least every two weeks, and sometimes even sooner. It can be a bitch to keep up with, so I’ve set up a wargemmer hudson task to periodically poll the update center to see if there is a new release. If there is, it’ll gem it right up and push it to rubygems. No fuss no muss.

Let the war begin!

We must come together to honor the command line

I reap no joy from writing command line interfaces in Ruby, and about 5 minutes before leaving RubyConf this year I realized that I am not alone.

It’s not angst I feel so much as a general “meh”: There’s a fairly decent out-of-the-box way to do it in OptionParser, but the minute you want to introduce commands is when the muddling begins. Sure you can see your way through it, but it’s never robust and you always get smacked in the ass by the edge cases that inevitably come along.

It ain’t for lack of trying

My own abortive attempts not withstanding, I’ve tried in the last year at least 3 different libraries for parsing CLI options in my applications and gems. These are gli, thor and optitron. I have known and loved each of these libraries in its own way, but ultimately all of them left me feeling unsatisfied. I won’t go into my specific dissatisfactions with each here, but suffice it to say that my malaise was shared. The general consensus of the cohort at the bar was that despite the recent innovations and improvements in the space, we’d really have just been better off with ”OptionParser plus commands.”

But we should do better

Better off for sure, but is that still what we want: to just get by?

No. we need a way to write command lines that feels like judo: deploying minimal force to effortlessly sling mountains of code.

I say we can have our cake and eat it too. I say our command lines can be easy. I say our command lines can be sexy. I say our command lines can be so awesome that they crap rainbows and have herds of unicorns come thundering out of their asses.

That’s why I’m starting the optimal project. It may lead to nothing. It might even lead eventually to some code… that’s not the point right now though. The point is to reach out to everyone who feels strongly about this and ask them to contribute ideas and requirements in order to build a consensus on the next generation of CLI builder that everybody can be proud of.

It certainly will not happen overnight, but for the betterment of all, it’s high time to find the true successor to OptionParser: the one the community sees eventually integrated into the standard library.

Accessing JavaScript Objects From Ruby

Awhile back, I wrote a post on how to access Ruby objects from inside your JavaScript enviroment when using The Ruby Racer. It showed just some of the many ways that you can access Ruby state and call Ruby code from within JavaScript. However, the story doesn’t end there. The Ruby Racer is, after all, a two way bridge, and I thought it would be useful to document some of the ways in which you can access your JavaScript environment from within Ruby.

Let’s start with the Context#eval method, which is used to execute JavaScript code from Ruby. Its the most intuitive way, and what gets used most often for examples.

Given a string, it compiles it as JavaScript source and then executes it inside the context. Nothing crazy there:

V8::Context.new do |cxt|
  cxt.eval('1 + 1') #=> 2
  cxt.eval('foo = {bar: "bar", baz: "baz"}')
  cxt.eval('foo.bar') #=> "bar"
  cxt.eval('foo.baz') #=> "baz"
  cxt.eval('new Object()') #=> [object Object]
end

Among other things, the context captures what functions and objects are defined in the global scope whenever anything gets eval()‘d. So, for example, the Object constructor used in the last line is stored in the context.

As a tool for embedding however, eval() is the most blunt and (often) inefficient method available. For this reason The Ruby Racer, sports an extensive Ruby API for interacting with JavaScript objects which includes accessing properties, calling functions and methods, as well as creating new instances. In fact, I rarely use eval() at all to manipulate a JavaScript context except for loading source files into the interpreter.

The Ruby API consists of V8::Object and all of its subclasses:

V8::Context.new do |cxt|
  cxt.eval('new Object()').class #=> V8::Object
  cxt.eval('(function() {})').class #=> V8::Function
  cxt.eval('new Array()).class #=> V8::Array
end

Objects

The most fundamental operations in dealing with JavaScript objects are the getting and setting of values. Coincidentally, this is also a fundamental concept in Ruby, so it comes as no great surprise that we can re-use those constructs in Ruby that deal with property access to mirror those same operations on their JavaScript counterparts: the []()/[]=() and foo()/foo=() methods:

given the following context:

cxt = V8::Context.new
order = cxt.eval('order = {eggs: "over-easy"}')

we can read the eggs property of our order in several differnt ways. Via hash access (note that both string and symbol are acceptable as key values):

order['eggs'] #=> "over-easy"
order[:eggs] #=> "over-easy"

Values can be set with the hash-style access as well, and changes made from Ruby will be reflected accordingly on the JavaScript side

order['eggs'] = "sunny side up"
cxt.eval('order.eggs') #=> "sunny side up"

For property names that are also valid Ruby method names, you can access them just like you would with Ruby properties declared with attr_reader or attr_accessor. Again, changes made to the object in this way will be reflected in JavaScript:

  order.eggs #=> "sunny side up"
  order.eggs = "scrambled"
  cxt.eval('order.eggs') #=> "scrambled"
end

In the cases where the property name is not a valid Ruby method name, hash-style access is mandatory:

order['Extra $%#@! Mayonaise!'] = true

V8::Object also includes Enumerable and allows you to access all properties of a given object.

order.each do |key, value|
  puts "#{key} -> #{value}"
end

#outputs:
eggs -> scrambled
Extra $%#@! Mayonaise! -> true

As a convenience, V8::Context delegates the hash access functions to the JavaScript object which serves as its global scope.

order == cxt['order'] #=> true
order == cxt.scope['order'] #=> true

Arrays

Not much to see here, but before you move along: A V8::Array is like every other V8::Object except it has a length property, and it enumerates over the items stored at indices instead of the key, value pairs of its properties.

array = cxt.eval('["green", "red", "golden"]')
array.length #=> 3
array.map {|color| "#{color} slumbers"} #=> ['green slumbers', 'red slumbers', 'golden slumbers']

Functions

Consider the classic “Circle” example from every object oriented playbook you’ll ever read. In JavaScript, the way to implement the circle “class” is with a constructor function.

circle = cxt.eval<<-JS
  function Circle(radius) {
    this.radius = radius
    this.area = function() {
      return this.radius * this.radius * Math.PI
    }
    this.circumference = function() {
      return 2 * Math.PI * this.radius
    }
  }
  new Circle(5)
JS

Now that we have the Circle constructor defined, and we have a reference to an instance of it in Ruby, we can call its methods just as though it were a normal Ruby object. Of course, only we know that under the covers the implementation is actually in JavaScript:

circle.class            #=> V8::Object
circle.radius           #=> 5
circle.area()           #=> 25Π
circle.circumference()  #=> 10Π

In JavaScript, methods are just object properties that happen to be functions. Therefore, you can get a reference to the actual function value just as you could from JavaScript simply by accessing the property by name. The resulting value is an instance of V8::Function.

area =                circle['area']
circumference =       circle['circumference']
area.class            #=> V8::Function
circumference.class   #=> V8::Function

Once you have a reference to a V8::Function, there are two ways to call the underlying JavaScript code (Actually there are three, but the third will be covered in the next section). These are the call() and methodcall() methods. To understand the difference between these two methods, it helps to understand how JavaScript functions themselves are invoked. In the event, there is the option to pass an object which will serve as the implicit invocant or this value. If no this is provided, the function will use the global scope in its place. Take for example the following JavaScript

var circle = new Circle(5)
var area = circle.area  //=> [object Function]
//no invocant, corresponds to ruby call()
area()  //=> NaN, there is no global 'radius'.
//call with invocant, corresponds ruby methodcall()
area.apply(circle)  //=> 25Π

The same code in Ruby:

area = circle['area']
area.class              #=> V8::Function
area.call()             #=> NaN
area.methodcall(circle) #=> 25Π

Because of this mechanism, JavaScript method invocation is much more flexible than Ruby in the sense that virtually any object can be used as the invocant of any function provided it has the requisite properties to satisfy the function’s requirements:

area.methodcall(:radius => 5) #=> 25Π
other_circle = OpenStruct.new
other_circle.radius = 10
area.methodcall(other_circle) #=> 100Π

A very powerful construct indeed.

Constructors

But wait, there’s more! Any JavaScript function can be either invoked normally, or, combined with the new keyword, as a constructor. It’s what we used in the original eval() block to create the instance of Circle whose methods we were messing about with.

That was not quite necessary since we can invoke functions as constructors from Ruby too. This is done with the new method of V8::Function.

Circle = cxt['Circle']
circle2 = Circle.new(3)
circle2.radius        #=> 3
circle2.area          #=> 9Π
circle2.circumference #=> 6Π

Interestingly, when used as a constructor, a JavaScript function is almost completely indistinguishable from a Ruby class. This apparent duality between classes and functions is behind the decision to represent Ruby classes reflected into JavaScript as functions

The driving goal in all of these cases is the ability to author intuitive, tightly bound JavaScript, which necessarily doesn’t involve evaluating strings to get what you want done. In closing, if there is something missing from this API you think would be useful, I would love to hear about it

Accessing Ruby Objects From V8

The upcoming version of The Ruby Racer will see some significant changes to the way Ruby objects are reflected into the javascript runtime and how you the programmer can control the way that those ruby objects appear.

Most of these changes were precipitated from the experience of integrating The Ruby Racer with the Rednode and RDom projects. Both efforts have run up against the shortcomings in the way Ruby objects are embedded into JavaScript, and as a result have both had a large hand in crafting the improvements which I’m about to describe. Many thanks got to Aslak Hellesoy and Sven Fuchs for helping to flush out all the various issues.

In picking the default strategy, there were three principles that I wanted to adhere to

Do what I mean

The JavaScript object representing your Ruby object should behave as closely as what you intend with the Ruby object itself. In other words, properties should be properties. Methods should be methods. Public properties and methods should be public, while private properties and methods should be private.
No new syntax required

Ruby has a rich and flexible syntax for describing how objects behave. Instead of introducing a new syntax by way of a DSL describing how those objects behave inside a javascript runtime, I wanted to borrow/re-use as much of the Ruby syntax as possible. This may prove to be a difficult goal to achieve going forward given that the two runtimes do differ in many significant ways. Nevertheless, I want to see how far plain old ruby semantics can take us.
Safe by default

JavaScript has been used since its earliest beginnings to provide a eval-safe sandbox in which anonymous code can be executed without posing any danger to the host system. To enable this type of development, one of the design goals of The Ruby Racer is for developers to be have the freedom to build their JavaScript environments confident that the framework won’t introduce silently introduce dangerous security holes. Unsafe behavior must be explicitly included into the default JavaScript runtime.

Basic Access

To call a Ruby object from javascript, assign a name to it in the context. The following code assigns a new instance of the class Foo to the name f.

class Foo
  def foo(count)
    "foo" * count
  end
end
cxt = V8::Context.new
f = Foo.new
cxt['f'] = f

We can now access this object from within JavaScript

cxt.eval('f.foo(2)') #=> "foofoo"

Ruby procs and methods can also be directly embedded into javascript and will behave just like a function

cxt['foo'] = proc {|times| f.foo(times)}
cxt['footoo] = f.method(:foo)

cxt.eval('foo(3)') #=> "foofoofoo"
cxt.eval('footoo(1)') #=> "foo"

Properties vs. Methods

In JavaScript, everything is just a property on an object. Methods are just properties that happen to be functions. In Ruby on the other hand, properties and methods are indistinguishable. A property is just a method without any arguments, so The Ruby Racer re-uses this convention for reflecting Ruby objects into JavaScript. That is to say if a Ruby method takes no arguments, then it will look like a JavaScript property.

class Random
  def number
    (Kernel.rand * 100).to_i
  end
end

cxt['random'] = Random.new
cxt.eval('random.number') #=> 80
cxt.eval('random.number') #=> 77

Property assignment

Properties aren’t always read only. When you want to assign to them, The Ruby Racer uses standard Ruby assigment methods

class Doubler
  def number
    @number ||= 2
  end
  
  def number=(value)
    @number = value * 2
  end
end

cxt['doubler'] = Doubler.new
cxt.eval('doubler.number = 5')
cxt.eval('doubler.number') #=> 10

Which mean for standard run of the mill Ruby properties, you can use plain old attr_accessor, just like Matz intended

class NotFancy
  attr_accessor :pants
end

plain = NotFancy.new
cxt['plain'] = plain
cxt.eval('plain.pants = "blue"')
puts plain.pants #=> "blue"
plain.pants = "green"
cxt.eval('plain.pants') #=> "green"

Methods with No Arguments

There are some cases however, where you want a zero-argument Ruby method to not look like a property. Most of the time these represent definite actions that can be invoked (this.dequeue(), stack.clear(), etc…) as opposed to state that can be read (e.g. array.length). For this case, make a getter that returns a function.

class Mouth
  def sayHello()
    proc do
      "Hello"
    end
  end
end

cxt['mouth'] = Mouth.new
cxt.eval('mouth.sayHello') #=> [object Function]
cxt.eval('mouth.sayHello()') #=> "Hello"

Dynamically Declaring Properties

A highly meta-programming enabled language like Ruby means that methods are often generated by code and not by the programmer. If you’re going to generate methods that will be called from JavaScript, then it is important that you be able to control whether they appear as properties or functions. By default, Ruby Proc objects have an arity of -1 (meaning variadic arguments)which means that JavaScript will see them as a function and not a property.

class Foo
  self.send(:define_method, :bar) do
    "bar!"
  end
end
cxt['foo'] = Foo.new
cxt.eval('foo.bar') #=> [object Function]
cxt.eval('foo.bar()) #=> bar!

To make the method appear as a property, you must explicitly tell ruby that the block takes no arguments by declaring it with “empty pipes”

class Foo
  self.send(:define_method, :baz) do || #<--- empty pipes
    "baz!"
  end
end

cxt.eval('foo.baz') #=> baz!

Intercepting Property Access

What happens if you want to implement properties so dynamic that you won’t know if they are defined until you actually try to access them? This is actually very common in Ruby. If it cannot find a method for particular object, it will invoke its method_missing() method and allow it to provide a suitable implementation at the very moment of dispatch!

This usually happens when you use information encoded in the property name as part of the property definition as in ActiveRecord finders such as find_by_job_description.

The Ruby Racer provides a similar mechanism to allow Ruby objects to define properties at access time. When you attempt to access a property of a Ruby object from JavaScript, it will first look to see if there is a pre-defined property. If not, it will then check to see if the object has defined a Ruby “hash access” method [](). If so, it will invoke that method with the missing property name in order that the object can provide any implementation.

Let’s say for example that you wanted to dynamically define properties that returned a list of process for a given user.

class Processes
  def [](name)
    `ps -U #{name} -o pid`.split("\n")
  end
end

cxt['processes'] = Processes.new
cxt.eval('processes.cowboyd') #=> ["205", "209", "210", .....]
cxt.eval('processes.root') #=> ["1", "10", "11", "12", .....]
cxt.eval('processes.no_such_user') #=> []

In a similar fashion, you can use the Ruby hash setter method []=() in order to intercept property setting

Embedding Classes

In may come about that you want to embed an actual Ruby Class object into your JavaScript enviroment. JavaScript does not have the concept of classes, so we have to choose the best representation we can. The most natural way to do this is by treating a Ruby class as a constructor function. It behaves just like any other object embedded into ruby except that by default the Ruby Racer treats them as constructor functions. You can call the constructor in order to create instances of your ruby class and use them from JavaScript

class Point
  attr_reader :x,:y
  def initialize(x, y)
    @x, @y = x, y
  end
  
  def to_s
    "(#{x},#{y})"
  end
end

cxt['Point'] = Point
cxt.eval('new Point(1,2)') #=> (1,2)

Where it Looks for Methods/Properties

Finally, when The Ruby Racer is trying to find a method or property on an object, it does not search all of the object’s public methods, but only a subset. This is because there are many potentially dangerous methods on every single Ruby object (such as eval()), and the Ruby Racer should tend towards safety. For that reason, The Ruby Racer provides access to all methods in a class’s list of ancestors up to but not including Object.

This includes any included or extend modules by an object’s class and superclasses (except those included by Object and above). e.g.

module A
  def a 
    "a" 
  end
end
class B
  include A
  def b 
    "b"
  end
end
class C < B
  def c
    "c"
  end
end
module D
  def d
    "d"
  end
end
class Object
  include D
end

cxt['c'] = C.new
cxt.eval('c.a') #=> a
cxt.eval('c.b') #=> b

cxt.eval('c.d') #=> nil
cxt.eval('c.to_s') #=> nil

That’s about it. The Ruby Racer aims for accessing Ruby objects and methods to be intuitive, simple, and safe so that you can focus on creating your ideal embedding.

Rye Repeat Yourself Enough

Lately, I’ve been practicing the exact opposite of the DRY principle. Yup, you read that right: I repeat myself constantly, and in as many different contexts… including code, database schemas, test-plans and even documentation. The reason: So I won’t have to repeat myself. It’s not that I don’t buy into the DRY principle, or that I’m somehow skeptical about the value its proper application yields. I just find it difficult to achieve in practice.

According to the Pragmatic Programmers who coined the term:

DRY says that every piece of system knowledge should have one authoritative, unambiguous representation. Every piece of knowledge in the development of something should have a single representation. A system’s knowledge is far broader than just its code. It refers to database schemas, test plans, the build system, even documentation.

Essentially it is an empirical measure against which to gauge the proper level of abstraction in your system. If you have not made the right abstractions, then you will be found to be repeating yourself, whereas if you have made the right abstractions, then you won’t. This is because abstractions directly encode the system knowledge that DRY talks about no less than 3 times in its definition.

The benefits of good abstractions are many and well known, and I’m not going to go into them here (there’s always the internet for that), but what I’ve taken particular heed of lately is that abstraction is a double-edged sword that should not be wielded lightly. Get it right, and the previously intractable dissolves to simplicity in an instant… but get it wrong, and a snarl of complexity gridlocks your code base just as fast.

You see, I wasn’t telling the whole truth when I said before that abstractions directly encode system knowledge. It would be more correct to say: Good abstractions directly encode system knowledge. Bad abstractions directly encode system ignorance.

The problem is that the DRY principle, while still very valid and useful, doesn’t help us here because it assumes system knowledge as its input. But really valuable system knowledge isn’t just lying around, it must be carefully constructed with significant amounts of research and analysis. That is to say, it must be learned. And what better way is there to learn something than by doing it over and over again until you know it inside and out? What better way is there than trial and error to really get at the physics of a phenomenon until you can use it to easily analyze it in all its forms? The fact is, you’ve got to repeat yourself over and over enough times until you can say with confidence: This I know to be true.

You can think of it like doing your math homework, or being like Daniel-San: waxing Mr Miyagi’s car again and again until you know karate by instinct. In fact, I’m convinced that there’s no other way to generate real knowledge.

Ironically, I repeat myself a lot these days because I hate repeating myself. I loathe it with an almost pathological passion, and sometimes (more often then I’d like to admit) it plays to my disadvantage. In my obsessive/compulsive rush to not repeat myself I’ll introduce a bad abstraction based on flawed or incomplete system knowledge, and worse still, I won’t know that I took the wrong fork in the road until it’s way late in the game and it’s hard to find the way back home.

That’s why I’m playing it cool lately and making sure that I repeat myself enough to perfect the system knowledge required to keep myself DRY.

In Search Of A Better Autogrowing Textfield

Autogrowing textfields are a nice piece of flash for any web application. They conserve precious screen real-estate, but grow to a height just big enough to accomodate the text the user actually typed without resorting to unsightly things scrollbars.

There are a number of solutions out there, but since I’m using jQuery on my current project, the two that I evaluated were Growfield and Autogrow.

While those two plugins often did what they were supposed to do, they also had tons of quirks and edge cases that prompted me give it a go myself. Specifically, I found that depending on the browser and operating system I would encounter:

eye-jarring jitter wherein the textfield would expand and contract my 1 or 2 pixels with every keystroke</li>
resizing of the textfield to 0 pixels, when there was no text in the box
intermittent loss of focus on some browsers</li>

But most importantly, it seemed that more often than not, the text did not actually grow correctly in any of the real-world environments that I tried in my production application.

As I discovered, the main reason for this was that the technique used was to fill a hidden DIV element with the text from the textfield, and then use the height of that DIV to determine how tall the textfield should be. That works great except for one nasty detail: most browsers use native widgets for text inputs, and those native widgets have their own word-wrap and layout code, so even things like font, line-height and letter spacing being equal, you can still get different word-wrap behavior from OS to OS.

In an attempt to side-step all of these issues, I thought “what if instead of using a DIV element to calculate the height, I could use a native textfield?” After all, if native textfields use a different word-wrap and layout algorithms, can’t I just try and harness those algorithms, whatever they might be? At first this seems perfectly circular i.e.: I don’t know what the ideal height of a textarea should be, so I’ll use the height of a texarea to find out. Of course there’s a twist, and the wonderful little property that makes it all possible is the scrollHeight.

lines, not heights.

Please Stop Using Global Variables In Ruby

I couldn’t sleep this morning. I woke up around 4:30 AM thinking about how code is organized in ruby. I’m sure I will look back on my life with disbelief and a touch of shame at such behavior, but that is (hopefully) decades away.

The problem turning around in my head which eventually caused me to wake was my own inability to come to terms with a prevalent pattern in the ruby community: the global variable. Swirling in my semi-conscious mind was a motley mixture of bewilderment, self-doubt, scorn and a touch of righteous indignation.

On the one hand, there must be something I’m missing. One of the reasons I love ruby-space is that it tends to be filled with talented programmers who work the language to produce code that is powerful while at the same time being easy to understand and use. Surely, their collective conscious is a better guide than my own experience and intuition. I worry that is primarily my own fuddy-duddity that prevents me from accepting global variables as good practice.

To clarify, when I speak about global variables, I’m talking primarily about using global class objects as the repositories for methods and state. There are plenty examples out there of this:

# active record
Post.find(:all)

# amazon s3
S3Object.store('me.jpg', open('headshot.jpg'), 'photos')

# paypal-business gem
Paypal.capture(params)

In each of the preceding examples, there is implicit state stored on the class object. In active record, the database connection information, in S3Object, the AWS access key id and secret key, and finally, in Paypal, the paypal server url and business account name. Because the class object is global, the state on it is effectively global. And that is what sets my spidey sense to tingling.

Not only is this style subject to the all the arguments for why global variables are bad, but it just flat-out makes the API’s themselves less useful. Here’s an example from an actual project.

I had a system which needed to talk to two different SQS queues which were contained in two separate amazon web services accounts. Unfortunately, the standard SQS api uses class variables to store configuration information, and then a set of class methods to access the queue services, so I was forced to overwrite the global configuration parameters every time I wanted to access the queue. Luckily, the application was not multi threaded, or I would have been stuck rolling my own.

I don’t see why:

SQS.access_key_id = 'ACCESS_ID'
SQS.secret_access_key = 'SECRET'
q = SQS.get_queue 'MyQueue'

is any better than:

@sqs = SQS.new :access_key_id => 'ACCESS_ID', :secret_access_key => 'SECRET'
q = @sqs.get_queue 'MyQueue'

(IMHO it’s uglier) and in the second, hypothetical API, you can painlessly, and in a thread-safe manner talk to multiple queues on multiple accounts.

@one = SQS.new :access_key_id => 'AccountOne', :secret_access_key => "AccountOneSecret"
@two = SQS.new :access_key_id => 'AccountTwo', :secret_access_key => "AccountTwoSecret"

But there’s a deeper principle at stake here than allowing multiple instances. It’s about giving power to the programmer, and letting them control the API, and not the other way around.

It seems to me that when you’re designing an API, you want the programmer(to the greatest extent possible) be able to create their own little world over which they have complete control. The should be able to spin up as many instances of your code and use it in ways that perhaps you hadn’t thought of without them having to worry that one world will interfere with another world they’ve created.

If you’ve absolutely got to go global, it’s very little effort to wrap a static/global interface around a single default instance, but not the other way around. Given the similarity in effort, it seems like embedability is the way to go.

Global variables surely have their place in some applications, and I’m not opposed to them on purely ideological grounds, but ruby already has a construct for global variables… it’s called the ’$’ sigil.

In your environment.rb, or wherever you can just put:

#environment.rb
$sqs = SQS.new :access_key_id => 'AccountOne', :secret_access_key => "AccountOneSecret"

#somewhere else
q = $sqs.get_queue "MyQueue"

My suspicion is that Ruby on Rails shares a large portion of the blame for the evolution of this style what with ActiveRecord.establish_connection() and friends. As the highest profile ruby project it’s only natural that people will copy its conventions –for better or for worse. Still, I wish people in the future would break from this particular precedent and if they go global, to do it properly. Give the poor class objects a break!

Patiently Waiting For Javafx Script

About a year ago, SUN announced a “new” platform for developing embeddable applications on the web, and they called this platform JavaFX. In reality, this new platform seems to me less of an innovation and more of a rehabilitation of the applet infrastructure which they allowed to languish over the last decade.

That’s not a criticism, it’s a compliment. I’m happy that they’re going to give applets another go. Heck, I’m even glad they’re calling it JavaFX too. “Applet” always sounded too effete anyway. In any case I’ve always like the applet model, and wished it were a more viable option for web development. Given the success of Flex, which also uses the applet model, other people must feel this way too.

But I digress.

As far as I can tell (I’ve only been able to get my hands on some demos and a little open source compiler) JavaFX involves upgrading some of the more boorish aspects of the java browser plugin. To name a few: faster download and initialization of application and JRE code, tighter integration and communication with the rest of the DOM, modern video and audio codecs…. All those are necessary upgrades to ensure the viability of the platform, but they can hardly be called innovative

That said, one new and curious aspect of JavaFX is that it will (for the most part) not be written in Java. Instead, SUN is promoting a completely new and aptly named scripting language: JavaFX script.

Of all the new features it was the specification of this language that impressed me most. I’m not sure how this little gem of a language managed to come out of SUN, but it’s absolutely loaded with modern language features to make programming in it clear and concise.

I’m talking about

closures
pure functions
lazy evaluation
list comprehensions (that put ruby's to shame)
optional declarative syntax
rolled in query language for searching and modifying data structures
data binding supported at the language level
tons of stuff I'm sure I don't even know about....

Overall it seems like JavaFX script is like 4 separate languages existing in harmony in one neat package, and as I read more and more about it, I kept thinking to myself “This is not just some standard UI scripting language with a little sugar and a few extra hooks for writing GUIs. This is positively avante-garde.

I like the idea that SUN can challenge their developers to think in new ways, but I can also imagine a backlash, or more mildly put, an general aversion in the developer community, since this is not any old language research project, but the centerpiece in SUN’s product in the battle over the browser VM.

So where did this decision come from to go with such a departure from the norm when it came to defining JavaFX script? I like it alot, but I worry that SUN is going to chicken out, and go with something a little less sexy but a little more digestible (at least in the short term) to its current developers as well as the developers of competing platforms like Adobe Flex or Microsoft Silverlight.

When Using Mod Rewrite Remember That The Version Counts

RewriteRule ^archives/(\d+).html http://www.thefrontside.net/map2new.php?$1 [R]

I wanted to match files like archives/000532.html.

But when it came time to deploy, it didn’t work. Turns out the environment I was deploying to was using a different version of Apache. That’s bad practice of course, but aside from that, the little gotcha was that the regular expression interpreter was different in different versions of Apache, and in this case, its behavior differed not only from the newer version but also from the behavior of most regexp engine’s out there. Specifically, the “digit” literal \d is not understood by older versions (which interpret it as a literal “d”).

Instead I had to use [0-9]

RewriteRule ^archives/([0-9]+).html http://www.thefrontside.net/map2new.php?$1 [R]

This is a very specific nugget, but hopefully it will save someone a headache down the road. Just remember, if your mod_rewrite regexp isn’t working, check the specific version of the engine, and make sure your regexp is one that it will understand.

The Chicken And The Vine Trans Global Pair Programming

The Frontside Software is a three person company with "offices" in Michigan, Finland, Massachusetts, and New Jersey. We're don't see each other every day, and we're rarely in the same room, but we still do a significant portion of our development work in pairs. Despite many other competing setups, we still do this with the not-so-new, not-so-exciting, yet extremely flexible and reliable VNC combined with a voip product like skype.

How it works

One of us (the driver) runs a vnc server which transmits everything that is rendered on his display to one or more vnc clients (passengers) being run by the other half of the pair. That way, the client can see everything that goes on while the driver is coding, including his code editor, his web browser, his terminal windows, etc... Meanwhile, you've got real-time audio so that you can talk about the work you're doing as you're doing it.

Is there something better?

Not yet. We've seen some new collaborative coding tools like SubEthaEdit and Gobby come down with some very slick features. Specifically, the updating and syncing of editor state between the two machines is very fast, and effectively coordinates multiple people editing the same document, with as few clashes as possible. The way in which they do this is impressive, but after having given it several abortive attempts as a real solution for remote pair programming, we went back to good old VNC. Here's why:

Lack of editing features: The collaborative editors of today are good at one thing: editing text collaboratively. The problem is that when you're pair-programming, you're not editing text, you're editing code, and code is only a simple sequence of text to a computer. There are lots of editors these days that leverage the semantically rich structure of the documents on which they operate like TextMate, Emacs, Eclipse.... everybody has a favorite, and because your collaborative editor is not your favorite, that means it sucks ;-)
Lack of environment: Of course, there are a scant few collaborative editing plug-ins for existing IDEs which would seem to address this problem, but adding on another layer, development is about more than just coding. It's about browsing documentation, running servers, invoking build scripts from the command line, and about a million other tiny tasks. In effect, your actual IDE is not just one application, it's your whole computer, and if the only thing being shared is a single app, then it cuts your pair out of a lot of important context. With VNC, everybody sees what's going on all the time. They can see not only the code, but also the running program.
Editing the same document at the same time isn't really helpful anyway: If you've done much pair-programming, then you realize that the real value doesn't come from having two sets of fingers on the keyboard at the same time. In fact quite the opposite: Both participants are following the thread of development, but one of them is freed entirely from the act of coding so that they can think about high-level architectural issues, or lookup api docs, or google for resources --all in parallel. Having both people pounding on the keyboard actually hinders this dynamic.
VNC is cross platform: This is a biggie. There are vnc clients and servers for Windows, OSX, and Linux (we code on all three), and they all interoperate with each other. That's a pretty hard feature to top, especially for single-platform apps like SubEthaEdit and Gobby which run on OSX and Linux respectively.

About the only drawback to pair programming with VNC is that there can be some fairly significant lag (between 1-4 seconds) depending on network load on either end, but even so, when it comes to actually developing collaboratively, VNC and Skype are our daily tools of choice.

VNC Clients/servers

OSX client: Chicken of the VNC
OSX server: Vine Server
Windows/Linux client/server: http://www.tightvnc.com/download.html

Taming The Rhino Making Mozillas Javascript Command Line A Little Less Brutish

I recently described how to use the one of the freely available shells as a great way to explore your javascript runtime. There are two implementations of the javascript interpreter sponsored by the Mozilla project, Spider Monkey an interpreter implemented in C, and Rhino, an interpreter implemented in Java. With respect to the javascript runtime itself, these two implementations are almost identical, and so what works in one, will generally work in the other. They diverge, however, when it comes to embedding objects and functions that are implemented in a language other than javascript. Naturally, Spidermonkey is better suited for embedding objects implemented in C, while Rhino excels at embedding objects implemented in Java.

What exactly is "embedding" an object implemented in Java? If you don't know what this means already, I could try to describe it to you, but why not show it live and in the flesh with the Rhino command line? That's the exploratory technique that I find so valuable.

cowboyd@subzero:~$ java -classpath js.jar org.mozilla.javascript.tools.shell.Main 
Rhino 1.5 release 5 2004 03 25
js> var javaString = new java.lang.String("Hello World")
js> javaString.hashCode()
-862545276
js> javaString.startsWith("Hello")
true
js> javaString.startsWith("World")
false
js> var jsString = new String("Hello World")
js> jsString.startsWith("Hello")
js: "<stdin>", line 5: uncaught JavaScript exception: TypeError: startsWith is not a function. (<stdin>; line 5)
js>

As you can see, javaString is a reference to an actual java.lang.String object, and has access to all the methods of that class, many of which are not contained in the native javascript String.

Embedding goes two ways. Not only can you instantiate and use java objects from javascript, but you can also pass in javascript objects as parameters to java methods. You can even extend objects and implement interfaces in javascript! Once again, the command line shows this in action. In this example, we’ll implement the java.lang.Runnable interface in javascript

cowboyd@subzero:~$ java -classpath js.jar org.mozilla.javascript.tools.shell.Main 
Rhino 1.5 release 5 2004 03 25
js> var impl = new Object()
js> impl.run = function() {print("Yeah that's right, you better run!")}

function () {
    print("Yeah that's right, you better run!");
}

js> var runnable = new java.lang.Runnable(impl)
js> var thread = new java.lang.Thread(runnable)
js> thread.run()
Yeah that's right, you better run!
js>

This is all well and good, but that's quite a bit of code to get right the first time! Chances are, if you're just starting out with the Rhino command line, you're not going to have as much luck, especially if you're using it as a tool to learn javascript in the first place, and unfortunately, this is where Rhino comes up way short. Rather than have a forgiving CLI, Rhino punishes you for every syntactic and semantic error that you make by not collecting command history. Even worse, hitting the up and down arrows result in bizarre character literals output directly to the prompt. You can't even correct a mistake that you've made in the current line you're typing without backspacing all the way to the error, and then re-typing from that point on.

In this example, I'd like to arrow-left so that I can correct my misspelling of "java.lang"

cowboyd@subzero:~$ java -classpath js.jar org.mozilla.javascript.tools.shell.Main 
Rhino 1.5 release 5 2004 03 25
js> var r = new java.lng.Runnable(^[[D^[[D^[[D^[[D

Ugh! Or what happens if the shell didn't take my last command because it was slightly bogus? I'd like to retrieve the command with the up-arrow, edit it a little bit in-place and then try again because after all, it was only slightly bogus. Watch me try and recover from this minor syntax error...

cowboyd@subzero:~$ java -classpath /usr/share/java/js.jar org.mozilla.javascript.tools.shell.Main
Rhino 1.5 release 5 2004 03 25
js> var f = function() {print("oops I forgot to close these parens"}
js: "<stdin>", line 29: missing ) after argument list
js: var f = function() {print("oops I forgot to close these parens"}
js: ...............................................................^
js: "<stdin>", line 29: missing } after function body
js: var f = function() {print("oops I forgot to close these parens"}
js: ...............................................................^
js: "<stdin>", line 29: Compilation produced 2 syntax errors.
js>
js> //I know. Up-arrow to the rescue!
js> ^[[A^[[A^[[A^[[A^[[A^[[A //drat, foiled again!

These problems are in particularly nasty contraposition to the technique of exploration via the shell which I advocate because the the cost for failure is so expensive. Indeed, what is so wonderful about most modern shells is that the cost for a syntax error is so small. For some reason, the implementors of the Rhino CLI decided to implement their shell with the typical functionality circa 1962.

It's not a problem for me though, thanks to one of my favorite unsung java libraries, JLine. JLine hits a super sweet spot in that it takes somewhere around 0 effort to add loads of standard functionality to your command line interfaces. It seems that no one in the java world bothers with a decent CLI; tragic in my opinion, but probably because it's considered well-understood, non-trivial and therefore tedious. With JLine, building that CLI comes at around 0 cost. What's really cool about JLine is that a program doesn't even need to be written with it. It can transparently intercept the console input for any java program and seamlessly splice on any and all functionality you'd expect from a hot shell: in-place editing, command history, you name it. In a word: perfect for a beast like Rhino.

cowboyd@subzero:~$ java -classpath js.jar:jline.jar  jline.ConsoleRunner org.mozilla.javascript.tools.shell.Main
Rhino 1.5 release 5 2004 03 25
js> prnt("oops let me try that again")
js: "<stdin>", line 1: uncaught JavaScript exception: ReferenceError: "prnt" is not defined. (<stdin>; line 1)
js> print("oops let me try that again")
oops let me try that again
js> //trust me, that was easy. Just like it should have been in the first place.
js>

JLine truly is a healing salve for your chafing CLI woes. Did I mention that it's cross-platform?

While the barrier to entry is extremely low, the path to upgrade is but a mild upward slope. If you do end up wanting nice extras such as TAB-completion or custom key bindings, it has a simple configuration mechanism, and Java API to make the pain of writing custom code as minimal as possible. But that's another bedtime story altogether.

I hope you enjoy JLine with Rhino, or with any other impolite java command lines you may use!

Learning Javascript From The Command Line

In the Drunk and Retired Podcast, episode 59 I spoke about learning your way around javascript the language independently from the browser, and how you can use the command line tools that come with the various javascript engines to interactively explore the javascript runtime.

When most folks think about javascript, they think about scripts that they embed into their web pages, but the truth is that it is a general-purpose programming language that has absolutely nothing to do with HTML. In fact, the javascript runtime is so orthogonal to other web browser functionality, that mozilla offers the javascript interpreter that it uses in Firefox and friends as a completely separate download. It's available as both .deb or .rpm package, and just to show it: Here's the wonderful hello world program, as entered into the shell.

cowboyd@subzero:~$ js
js> alert('hello world')
1: ReferenceError: alert is not defined
js>

OK, so I boobie-trapped that example in an attempt to beat the point I've been making to death. It's an error because `alert()` isn't actually part of javascript. In the context with which we're familiar(DHTML), it's a function that's defined by the browser. Of course, it just so happens that every browser implements `alert()` to behave in almost exactly the same way, but the function itself has nothing to do with the javascript core. Implementing our own version of alert is simple enough though.

js> var alert = function(message) { print(message)}
js> alert('hello world')
hello world
js>

Personally, I love the command line because it let's you dig your fingers deep into the computer's brain and, by pushing its buttons directly, see what's going to work and what isn't. Every time I have a question about how the javascript interpreter is going to behave, I don't look up the spec, or write something into my programs that I'm not sure how it will work. Instead, I fire up my trusty interpreter to discover empirically how the system works. Need to know if a RegExp is going to match? Don't guess. Ask the interpreter.

js> "foo".match(/bar/)
null
js> "foo".match(/oo/)
oo
js> "foo".match(/oo$/)
oo
js>

Wonder what the built-in "constructor" property of an object is? The runtime can tell you. It's his business after all.

js> function A() {}
js> var a = new A()
js> var o = new Object()
js> a.constructor

function A() {
}

js> o.constructor

function Object() {
    [native code]
}

js> o.constructor == Object
true
js> a.constructor == A
true
js>

Sure, you could write an in-browser script to do all this and throw output at yourself in the form of alerts, but the beauty of the javascript command line is that you can collapse the whole edit-save-reload-alert scripting cycle into a single step; type in the next line and see what happens. It's that super-tight feedback which let's you learn that much faster.

Live Stylesheets

Even after working with DTML for over a year, I'm still constantly astounded by how dynamic it actually is. The most important thing to keep in the back of your head as a DHTML programmer is that the same mechanisms used by the browser to build an HTML document at load time are available to your in-page scripts. So, really, the static loading of your pages when you access a url is just the HTML parser invoking the same methods on the same objects that are available to your javascript code.

Take this static HTML code:



This is my div

The same result can be achieved dynamically:

Nothing mind-blowing there; indeed, this technique is the cornerstone of DHTML. But it takes awhile to let it sink in to the point where using it is one of your first strategies to solving a problem. Specifically, you can use it not only to dynamically change layout-related HTML elements (span, div, ul, table, et al...), but also to create/modify behavioral elements(link, script, meta, etc...)

That said, I was recently trying to create a CSS stylesheet at runtime, and enable it so that its rules would be active in the page. At first, I was trying to use the W3C DOM CSS interface, the idea being to create a stylesheet object, add a bunch of rules to it corresponding to the rules that I wanted, and then put it into the stylesheets array. Unfortunately, doing this in a cross-platform way is borderline impossible. After pounding my head against that for awhile and getting nowhere, I figured, "why not just create the whole thing as a top-level html element, and let the browser just take it away from there?" The basic strategy is this: create a style DOM Element dynamically, add text content to it representing the actual stylesheet, and then pop it into the HTML DOM. With a few caveats, it works like a charm. The browser parses the text as CSS, and links in into the live cascade.

I mentioned caveats.. well, of course there are the obligatory cross-browser compatibility issues to be aware of:

In Internet Explorer, you can't just append a Text Node to a Style element, you have to use some trickery to get it to work.
KHTML & Safari do not honor style elements that are not contained in the HEAD element of the document.
While IE and Mozilla always create a HEAD element for you if one doesn't exist, Opera and KHTML do not.
While Opera will honor a dynamically created HEAD element, KHTML will not. (That's a bug in KHTML as far as I'm concerned as it violates the DHTML principle that I've been talking about.)

All in all, they're nothing to worry about and do not stand in the way of the fundamental technique. I've created a demo page showing this code in action. As a nice side effect, it works as quick way of playing around with CSS properties and how they effect the styling of elements.

Here is the source snippet implementing the technique I've described here:


   var style = document.createElement('style')
   style.setAttribute('type', 'text/css')

   var cssText = $('cssText').value
   if (style.styleSheet) { //IE only
      style.styleSheet.cssText = cssText
   } else {
      //for some reason this fails in IE.
      var text = document.createTextNode(cssText)
      style.appendChild(text)
   }