JXCore is a fork of Node.js that introduces multi-threading support and a Javascript engine abstraction layer to plug in different Javascript engines. They have support for iOS using Spidermonkey and Android using v8 and Spidermonkey. Below I provide a FAQ about who they are and what they can do.
Table of Contents
1 What is JXCore?
JXCore is an open source project under an MIT license at Github. It’s created by a company called Nubisa. What Nubisa has done is take Node.js (mostly 0.10.36) and forked it. But the purpose of their fork is to add in three features that they hope they will eventually get to give back to Node.js.
The first feature is multi-threading support. This allows them to run separate parallel instances of node.js at the thread level. Typically to do that in Node.js one has to run multiple processes. By using threads instead the cost of running parallel instances goes down and of course inter-instance communication can potentially be done much more cheaply.
The second feature is an abstraction layer for the Javascript engine underlying Node.js. Today Node.js is so tightly tied to v8 that it’s appropriate to think of Node.js just as a v8 extension. What Nubisa has done is cut the knot tying Node.js and v8 together and inserted instead a set of C level macros that let one wrap any sufficiently powerful Javascript engine underneath Node.js. Right now Nubisa has a plugin for Firefox’s SpiderMonkey as well as for v8.
The third feature is packaging. Nubisa can take a whole group of Node.js related files and put them together into a single executable. This makes deploying and moving them around much easier. They also have technologies to make it difficult to reverse engineer the source code.
Because of how JXCore is written (more detail below) anything that runs in Node 0.10.36 should more or less run in JXCore. Where there are differences it’s mostly due to issues of running on mobile platforms like Android and iOS.
2 Who is Nubisa anyway?
Near as I can tell they are a small self funded start up split between Boston and Poland. They have been around for 2 years now. Apparently they always wanted to target the mobile market but pivoted and decided to first target dense node.js hosting via their multi-threading support. Since each thread is an isolated instance of node.js this means that ISVs can use JXCore to host node.js more densely than is apparently possible with Node.js’s native process level isolation model.
3 You said SpiderMonkey and v8, but what about Javascript Core?
It would seem fairly obvious that if you want to run Node.js on iOS and you have a pluggable scripting layer then shouldn’t you plug in Javascript Core (JSC)? The answer would be yes but as v8Like found out there are limitations to JSC that make it not be a great fit for Node.js. To be fair the bugs don’t appear to be huge but they also aren’t fixed in the version of JSC that ships in iOS. So if you want to use JSC you would have to build your own version and ship that. This brings up two potential issues:
- It may be the case that the iOS terms of use forbid shipping a custom JSC. Nobody is sure if this applies to something like JXCore (as opposed to a web browsing scenario).
- JSC is LGPL. If you dynamically link a LGPL library then all is good. But if you link a LGPL library statically then you are required to release your source code.
So this means that JSC is not a great choice right now for JXCore. But if things change then of course a plugin is always a possibility. That’s the point of the JXCore design. The Javascript engine is a plugin, put in what you want.
4 So we can use v8 on iOS with JXCore?
No! Apple apparently bars JIT’ing so we can’t run v8 straight out. There is however a virtual machine “mode” in v8 where it can apparently output the JIT’d instructions and then interpret them rather than execute them. And yes, I’m told this is as horribly slow as it sounds.
5 So we have to use SpiderMonkey on iOS?
Yes. SpiderMonkey runs an interpreter as its base execution mode. This doesn’t violate iOS’s terms of use and apparently a bunch of games actually ship with SpiderMonkey on iOS. The sad part is that SpiderMonkey’s quite awesome speed (see here) comes from adding in IonMonkey. IonMonkey is a JIT’er that identifies “hot” code in a Javascript program and JITs it. Alas we can’t use IonMonkey on iOS, just the base interpreter. I will release an article soon exploring what this means from a perf perspective.
6 Do we also have to use SpiderMonkey on Android?
No, v8 is fully supported on Android.
7 How much of Node.js’s built in libraries actually work on JXCore?
Node.js's libraries are a core part of its value. So below I look at the state of those libraries in JXCore.
The following libraries just plain work - Buffer, Crypto, Domain, Events, Globals, HTTP, HTTPS, Net, Path, Punycode, Query Strings, Stream, String Decoder, Timers, TLS/SSL, UDP/Datagram, URL, Utilities, VM and ZLIB.
The libraries below work but have some edge case issues, mostly due to the inherent nature of the underlying mobile platforms.
DNS It should work but unfortunately different platforms, especially iOS, have weird behavior around DNS. There is more testing needed here to be sure this works correctly.
File System Some functions supported by node.js just plain don’t work on some platforms. For example, iOS does not have any native support for watchfile. iOS also has its own layout for storage dividing things up into app storage, document storage, etc. iOS also only provides access to a storage sandbox, not the whole drive. JXCore has worked around most of these limitations putting in code that feels more or less natural. In some cases they had to fake output. For example if someone runs stat on one of the node.js files on Android there is no actual file since the data is pulled out of the assets compressed folder. So they return fake stat results to compensate. But in general it should work.
Modules With the exception of native modules this works.
OS Much like file system there are aspects of mobile OS’s that don’t always have equivalents on all mobile platforms but in general JXCore tries to fake them where necessary.
Process Same issue as with File System and OS. Some commands, like cwd, can get very tricky on iOS because of its unique approach to managing files. They work hard to put in “fake” behaviors that do reasonable things.
These next libraries don’t work at all and frankly, shouldn’t. They just don’t make much sense on a mobile platform.
C/C++ Addons This actually might work “as is” on Android for developers who are using the v8 engine plugin. But in general this won’t work because JXCore’s whole point in existence is to provide an engine neutral API. The existing C/C++ Addons in Node.js are completely v8 specific.
Child Processes No. This library creates separate process instances of node.js. That is very heavyweight and probably not appropriate for a mobile platform.
Cluster This is for creating clusters of node.js machines that can share load. Not appropriate for mobile.
Console Mobile app models don’t really support a console. Therefore JXCore uses the console for logging.
Readline As with Console there is really no command line in the mobile execution models so this doesn’t make sense.
REPL As with Console and Readline there is no command line so there is no where to run a REPL from.
TTY No. On Android, since there is no console, they hijack standard in/out for logging. On iOS there really is no equivalent.
And finally, and sadly, the Debugger library just plain doesn’t work. This is a bug though and needs to be fixed. But it’s still pretty impressive that this is the only “total outage” in terms of JXCore’s support for Node.js’s libraries.
8 Will Node.js native add-ons run on JXCore?
No. They won’t. Well, o.k. maybe in some edge cases. Such as Android with the v8 engine. But in general, no. The reason is that Node.js native add-ons are designed to work directly with v8. And JXCore’s whole purpose is to put in an abstraction layer between the node.js code and the Javascript engine, so in general the v8 interfaces aren’t available. To work around this JXCore provides its own abstraction layer for add-ons that they can use to write code that will work with any Javascript engine plugged into JXCore.
In the long run the hope is to enable the JXCore native add-on interface to work with standard Node.js. In that case a native add-on written for JXCore could work both with JXCore and with mainline Node.js. There is also a theoretical possibility of getting something like nan to work with JXCore.
But in general existing native add-ons just plain won’t work “as is” with JXCore.
The more interesting question is - does it matter that existing native add-ons won’t run “as is”? It’s hard to say really. It depends on what you are trying to do. I looked at this question in some detail previously and assuming there isn’t some huge error in my logic it doesn’t seem like native add-ons are all that popular.
Still, I know the JXCore folks are working on porting over the most popular native add-ons.
9 How is the perf?
I have a whole article that explores this. But the summary is that node.js on a modern phone runs around 10x slower than on a modern PC. Which, honestly, isn’t bad!
10 How different is JXCore’s code from node.js? Can they really RI from Node.js?
To help me understand the answer to these questions I took the JXCore code and diffed it with the Node.js code. I was trying to see how different the two code bases are and how readable JXCore’s code is.
My general conclusion is that their code is readable, their changes make sense and other than the changes they made in the src directory I think RI’ing everything else would be fairly straight forward. I do think that RI’ing to the src directory is doable but it requires someone who has an intensely intimate understanding of the JXCore macro layer and JXCore’s functional extensions and even then I bet that for just about anyone but Nubisa’s CTO Oguz Bastemur, it would take a solid two or three weeks to RI to src.
One thing I did notice is that the code is a bit of a Frankenstein’s monster. There is code from 0.10.26, 0.10.29, 0.10.36 and some even from the master branch of Node.js. I talked with the Nubisa folks about this and they explained that they weren’t always happy with some of the changes that Node.js made and so they picked and choose which changes they took. In many cases they felt vindicated in their choices when it turned out that later Node.js would actually roll back a change that Nubisa didn’t like. The Nubisa folks have promised to release a doc on GitHub explaining their choices.
Below I walk through the major directories in the JXCore distribution and explore what the code looked like and how different it is from Node.js.
Root files Most of these files look like v0.10.26 and required some surgery to common.gypi, configure, Makefile and vcbuild.bat. But the changes all makes sense. Mostly adding build targets, removing targets like uploading to node.js’s website, etc. Nothing particularly thrilling.
benchmark Mostly identical, very few changes.
build_scripts These are android/ios utilities provided by JXCore, not part of Node.js.
deps Deps contains a bunch of sub-directories with the major dependencies so what is most interesting is what is in those sub-directories. So I list them individually below.
deps/cares No substantive changes.
deps/http_parser No substantive changes.
deps/npm The key file here is package.json. The only interesting thing is that the file appears to be from v0.10.29. JXCore also added some extra entries to the package.json file included with Node.js.
deps/openssl They are using the same release of opensll as Node.js master. Which makes sense. This is code that must be kept up to date.
deps/sqlite This is included by JXCore and isn’t part of Node.js.
deps/uv The files look like a mishmash of v0.10.26/v0.10.29. But the actual changes are minor. The biggest change is that JXCore really doesn’t like ngx-queue and instead uses queue.h from Ben Noordhuis. This drove most of the, minor, changes in UV. But most of the files are unchanged.
deps/v8 Mostly unchanged with the exception of a few insertions to deal with JXCore’s multi-threading abilities.
deps/Zlib A few minor changes but nothing major.
doc The majority of the files are unchanged other than what appears to be a global search/replace for /node.js/jxcore. A few files are edited where JXCore adds new capabilities.
lib These files look like they are from v0.10.36. The majority of changes are just white space/formatting. I sampled a bunch of the files and what changes there were seemed very minor.
node This is created by JXCore and is where they moved AUTHORS and ChangeLog files from Node.js.
samples This is JXCore code, not node.js code.
src See below
test My main issue is that the tests look like they are from v0.10.26 and I would have preferred they be from v0.10.36. But in any case the tests are there and most files are either not changed or have very mechanical changes like converting NODE_SET_METHOD to JS_METHOD_SET and such. I have to admit that at some point I stopped individually checking all the changed files and just spot checked them because there are literally hundreds of them. But in general the changes were really minor and RI’ing should be pretty easy. Also note that JXCore has added literally 100s of additional test files above and beyond Node.js’s to cover their functionality. Happiness is a large test base.
tools The majority of files are unchanged, what few changes there are, are tiny.
The src directory is by far the part of the code with the most changes from Node.js and that makes a lot of sense because this is the code that primarily handles the relationship between Node.js and v8, not to mention threading. These are the things that JXCore has changed the most.
The way JXCore allows for pluggable Javascript engines is by using a whole set of macros. In jxcore.h you can see the macros that wrap engines as a whole. To see the various type macros take a look inside src/jx/Proxy/Mozilla/Moztypes.h and src/jx/Proxy/V8/V8Types.h.
It’s also worth pointing out that JXCore uses a slightly different file layout than node.js 0.10.36 uses. Specifically a ton of *wrap.* files which live in /src in Node.js are placed in the wrappers sub-directory in JXCore. Similarly the node_win32_* files were moved to src/platform/win and v8_typed_array* and v8abbr.h ended up in JX/Proxy/V8.
In terms of the files I’d least like to own RI’ing my choices would be node.cc, node.js, node_counters.cc/.h, and node_dtrace.cc/.h.
But the bottom line is that even with the heavy surgery performed on the previously listed files I could still see someone with a deep understanding of the macro layer being able to do the RI. The majority of the changes are fairly mechanical. In many cases the JXCore folks choose macro names that mirror the v8 specific names and so make it easier to understand what needs to be changed. I’m not saying that an RI from mainline Node.js would be easy or fun, but it does appear doable.
11 What about RI’ing from Node 0.12.x?
According to the Nubisa folks JXCore was originally written against 0.11.x. So all the big v8 changes in 0.12.x are actually already supported by JXCore. Later on Nubisa decided to move JXCore back to 0.10.x because that is where their customers were. Nevertheless, the hooks for the major changes in 0.12.x are still there. So it should be possible to move to 0.12.x.
What does the acronym “RI” mean?
It stands for reverse integrate. It means that you have two code forks and one is getting updated faster than the other. So when you move changes from the “faster” fork back to the “slower” fork this is usually called a reverse integration since you are integrating from the “future” back to the “past”.
Hi Yaron!
I’m Ori and I was wondering if you can help me figuring something out. I looked at the jxcore-cordova plugin, and I was very interested in it, especially the part where it registers JavaScript functions and call them from Java code.
I tried to follow the footsteps of the sample, and also registered JavaScript function using my own defineEventCB function (I copied it from the plugin). Then I call callCBString from Java, but I get:
06-14 00:38:37.863 989-1002/? W/jxcore-log﹕ !!! js_ReportOverRecursed called !!!
The thing is, I call defineEventCB from thread 1, but callCBString from thread 2 (I can’t control the threads creations. it a system generated flow).
If I call these 2 functions from the same thread, everything works fine. Any idea how to do this?
Thanks,
Ori
Sadly I haven’t had much of a chance to use the Cordova plugin with Java yet. Jukka from my team has been driving that. Funny enough next week I’ll have to drive that code so I may or may not figure out what the problem is then. But my suggestion is to head on over to https://github.com/jxcore/jxcore/issues and file an issue. The JXCore folks are really good about responding to developer issues and will give you a quick response. Bonus points, btw, if you post your code somewhere so they can easily see it.
Very nicely written. i found that active development on JxCore has been stopped – http://www.nubisa.com/nubisa-halting-active-development-on-jxcore-platform/ Does it mean it wont support latest versions of NodeJS included as part of it? In that case, I think I should not go with JxCore for any new projects.. Whats your view on it?
Right now nobody I am aware of is doing any really active development on JXcore. So that means there is no one to fix bugs, there is no one to upgrade to new versions of Node, etc.
The Thali project is still using it, see here for details. But that is because we really have no where else to go. At this point we either need to build a new community around JXcore to keep it going or we need to find an alternative.
The Node foundation seems to be at least mulling over the idea of supporting node on Android. This is possible because v8 runs happily on Android. But this then begs the question – what about iOS?