November 4, 2009

Android + Rhino = ?

So, what do you get when you cross an android with a rhinoceros? I have no idea, but that's exactly what I've been doing over the past few weeks.

I recently downloaded the Android Scripting Environment (ASE) that I discovered on Google Labs. ASE is a nifty little project that allows you to write scripts for your Android device. At the time, the project supported the following languages:
The problem is that I don't really know any of those languages. So, I started learning a little bit about each of them. I finally decided to go with Ruby since it was the most object-oriented of the bunch, but about half way through the tutorials, I had an epiphany, "What about JavaScript? Everybody knows JavaScript." So I decided to submit a feature request.

I navigated my world-wide web browser over to the project's issue tracker, and, being the good user that I am, did a quick search for the word JavaScript. The query brought back a single, five-month-old issue titled Add JavaScript support. I read through it and noticed that they suggested doing it using an open source Java implementation of JavaScript by Mozilla called Rhino. I didn't think much about it at the time, I just starred the issue (i.e. added my vote to implement it) and went on with my day.

Later that day, I got to thinking: You know, that issue is five months old. They've already implemented five different interpreters. I may not know anything about Rhino or ASE, but I'm a pretty capable developer and could probably get it working in a matter of days if I really tried. So, I asked them if they'd accept a user-contributed patch. They said they'd be glad to, and I got to work on making it happen.

Implementing it was pretty straight forward. On the ASE side, I just needed to add a couple classes: one telling it about the new interpreter, and the other telling it how to start the interpreter. For the Rhino side, I just compiled the jar with debugging off (to save storage space) and ran it through the dx tool (so Android could understand the compiled code). The dx tool took me a little while to figure out because I kept getting an out of memory error when running it, but once I learned that the default stack size was too small, things fell into place fairly quickly. Now, the only other thing I had to do was create a JavaScript proxy class that scripts could use to make JSON-RPC calls to the Android API (via the AndroidFacade). I followed suite with the BeanShell and Ruby implementations and was able to come up with a fairly elegant solution. Now everything was in place, and it was time test.

I immediately started getting errors the first time I went to test it. Using the interactive shell I was able to call methods through the proxy class but wouldn't get any return values, and the executing saved scripts just spewed an ugly stack trace all over the screen. The first problem (no return values) was caused by an ambiguity in the JSON.parse() method to Rhino so, I figured the string was coming form a fairly trusted source and just went with JavaScript's built-in eval() method instead. The next problem was not so straight forward. Investigating the ugly stack trace showed that it was yet another out-of-memory error. I fixed that problem (again by increasing the stack size), but yet another error appeared: Class format not accepted. This threw me off. At first, I thought Dalvik (Android's JVM) was trying to load the JavaScript file directly, but that didn't make sense since other interpreters were working fine. After a few days of meditating and some ritual sacrifices to the programming gods, it hit me: Rhino was compiling the JavaScript file to a Java class file and then handing that class file to Dalvik which had no idea what to do with it (since it only understands Dex files). So I turned off Rhino's optimization (thus causing it to interpret and not compile the scripts), and all was well. The Rhinodroid was complete!

I contributed my code, and waited. It took a few days, but Damon Kohle reviewed it and merged it into the main codebase. About the only change I've noticed was that he set the default buffer sizes (making a couple of debug messages go away). He was really quick to get a new release out so now, thanks to my obsession with coding, users can now choose from JavaScript along with the other languages I mentioned previously.

All in all, this was a very fun experience. I got to learn more about Android and Rhino development, and I even got to use a distributed version control system for the first time. JavaScript is probably the widest known scripting language, and I hope that my contribution can help add to the success of ASE and can be yet another of my little claim-to-fames. But most of all, I'm proud that I can finally say, "I've contributed to an open source Google project!"

16 comments :

Mark said...

They're fortunate to have your contribution. Great job.

Brice said...

Hey, thanks Mark! I'm surprised anyone even reads my silly blog posts. :P I can't believe they'd be that interesting... I mostly just do it to document and share all the nerdy stuff I work on.

MikaelKindborg said...

Many thanks for your great work! Used your Rhino jar from the ASE repository and successfully embedded Rhino in a Java Android app. Great! Now I will add remote programming over a socket. Perhaps make a web server on the Android to which you can post JavaScript code.
Best regards, Micke

Brice said...

Haha, sounds you got a good thing going there, Micke! I'm glad I be a [very small] part of it. :)

David said...

Hi Brice, I'm *really* interested in embedding some javascript into an Android app like what Mikael Klindborg describes in his comment, but I can no longer find your .jar file on the ASE downloads page. Is there a way that I could get it directly from you?

Thanks! and thanks for the great work!

David said...

Hi Brice,

I got a copy of your .jar file afterall. It was always there, but in the rhino extras .zip, so I boneheadly missed it.

But there is a real problem that appear when I try to use the .jar file in an Android java project. I am developing using the Android plugin for Eclipse. And in Eclipse, the .jar file appears to be empty. The problem is that your .jar file has already been converted for use with Android so all the .class files have been replaces with a .dex file. This makes total sense for the ASE repository, since those .jar file will just be copied directly to the phone. But it causes some problems when the .jar file is referenced for use in a java application.

When referencing a library in Eclipse, the Android plugin automatically tries to convert the .class files in the referenced .jar file to the classes.dex file, and this process fails with the error "Error generating final archive: duplicate entry: classes.dex" if the .jar file is already converted. Hence your .jar file ends up looking empty in Eclipse and Eclipse will not complie.

Would it be possible to get an unconverted copy of your .jar file?

I'm really excited to use this stuff because I have some really cool ideas I want to implement and it is a bit silly that the Eclipse plugin for Android can't use .jar files already converted for Android, but I guess that's life. :)

Thanks!

Brice said...

Hi, David. I did not make any modifications to the original rhino code. So you should just be able to combine it's .class files with my .dex files. A couple things to note: 1. You must interpret the javascript since compiling it will output bytecode and Android can only understand dalvik. 2. You'll probably need to increase you application's stack size (if you start hitting stack overflow exceptions). Good luck, Brice.

Jay said...

Hi,

I was wondering if it was possible to generate Dalvik bytecode from
Rhino. If I understand correctly, optmizationMode can be changed to either set it in interpreter mode (-1) or to compile it to bytecode
(1-9). Is there a way to make that possible ? (Maybe
by integrating source from the dx tool into Rhino, or maybe replacing
JVM bytecode in Rhino by Dalvik bytecode (which would possibly require
rewriting the ClassFileWriter.java in Rhino?)

Thanks!
-Jay.

Brice said...

Hi, JP. I don't know a whole lot about the inner-workings of dalvik or rhino, but it sounds like you've got a great idea going. To throw in my two cents: I know there has also been some work done on the dalvik just-in-time compiler which might also contain some valuable pieces.

MikaelKindborg said...

@JP Yes, dynamic byte code generation on Dalvik will be cool when it arrives. Sure people are working on this, but it is not trivial. Best Micke

Jay said...

Thanks Brice, Mikael. I actually tried to integrate dx and Rhino and could generate a valid dex file from Rhino. It seems I'm running into some issues with the classloader structure in Rhino which try to load it as a java class and not a dex, because of which it fails.

Jay said...

Hi Brice, I was able to generate a Java class file, dex it, aapt it and load the jar via DexClassLoader. I'm now running into issues with the JS code itself. It seems that functions and the Liveconnect mechanism don't work (I get a CloneNotSupportedException from the constructor of the converted class), but simple JS code seems to work. I'm sort of at a dead-end as to how to proceed. Do you have any insight as to what might be happening and how I might proceed? Any suggestions are welcome! Thanks again for your help!

Aikeru said...

I know this post is very old now... but I thought I'd mention I really appreciate your work on JS for the ASE. I didn't know any of those languages either. :)

Brice Lambson said...

Thanks Aikeru, I had a lot of fun with this project. I'm glad people are still enjoying it.

teebir said...

Thanks Brice! I have a question though... I tried just including the dex'd jar and Eclipse didn't like it. So I just took the js.jar from R3, and included it as an external jar in Eclipse. I was able to perform a simple script.

What does this mean? Do I still need the dex'd jar?

Moroni Granja said...

Brince, thanks for this post. You have saved me hours of trial and error. I just needed to do setOptimizationLevel(-1); on the context to be able to use Rhino on android! Thanks a bunch!