As I’ve previously discussed I want to get node.js running on Android, iOS and WinRT. But to make that happen we need to understand the node.js ecosystem and that includes native add-ons and node-gyp. So I created a node package, node-gyp-counter, to heuristically determine how frequent node-gyp usage is in the node.js world. If my numbers are right then less than 2% of downloads of packages in 4/2016 involved node-gyp in any way. Of that 2%, just 10 packages account for 76% of node-gyp root package downloads and only 5 of those are relevant to mobile.
[Updated with figures from 5/24/2016]
1 Disclaimer
The code I used is untested, unreliable and probably conceptually incorrect. Seriously. You have been warned!
2 What I found
A tiny bit of terminology before I continue:
node-gyp root package A package that directly uses node-gyp
node-gyp package A package that is either a node-gyp root package or has one or more node-gyp root packages somewhere in its dependency tree.
| Count | Description |
| 279,196 | Total packages in node.js when I ran my query |
| 3172 | Total number of node-gyp root packages |
| 28,863 | Total number of node-gyp packages |
| 10.3% | Percentage of node.js packages that are node-gyp packages |
| 4,230,547,186 | Total downloads of node.js packages for 4/2016 |
| 71,735,382 | Total downloads of node-gyp packages |
| 1.7% | Percentage of node.js package downloads that are node-gyp packages |
Assuming my data isn’t completely wacked this argues that node-gyp is a tiny part of the node.js ecosystem. Only 1.7% of downloads in 4/2016 used node-gyp in any way, shape or form.
Next up I calculated the set of node-gyp packages that accounted for 80% of node-gyp package downloads in 4/2016. This returned 180 packages. I then calculated the union of the node-gyp root packages used by those 180 packages. The result had 64 entries (see A↓). In other words, if we support 64 node-gyp root packages we can support 80% of all packages that use node-gyp in node.js land. I went through those packages and I’d argue that only 41 of those are relevant to smart phones. Heck, if you look at the actual numbers the top 10 packages account for 76% of node-gyp related downloads and only 5 of those are relevant to mobile. So with literally 5 packages one could support almost 80% of all node-gyp downloads on mobile!
My take away is that if node.js ran on smart phones with no ability to run native add-ons we would still add huge value to the world supporting literally 98.3% of what people actually use node.js for. If we just figured out how to directly support 5 packages (possibly by building them straight into our mobile node.js images rather than using node-gyp to build and link) we would literally cover 98.7% of all node.js uses.
A Node-gyp root packages that account for 80% of node-gyp package use ordered by popularity
| Package Name | Description | Relevant to mobile? |
| ws | Websocket client/server/etc. | Yes |
| fsevents | Provides access to OS/X FSEvents. It’s used by Chokidar and Karma amongst others. It’s mostly used for build related stuff. | No |
| NPM | Well, um.. NPM. We don’t actually run NPM on mobile. We run it during build time when we create a mobile app. | No |
| node-sass | Node bindings to libsass, a stylesheet preprocessor. This is used at build time and so shouldn’t be needed on mobile. | No |
| dtrace-provider | Provide deep tracing into node.js. Cool but only runs on OS/X and Solaris systems. | No |
| phantomjs | A headless version of Webkit’s Javascript API. Used in node.js land for testing apps that have to run on Webkit. | No |
| phantomjs-prebuilt | The prebuild version of phantom-js | No |
| bufferutil | Used by WS, it’s actually optional I believe | Yes |
| utf-8-validate | Also used by WS, also optional I believe | Yes |
| fibers | Adds fibers to node.js. | Yes |
| kerberos | Kerberos library. | Yes |
| thread-sleep | Puts node to sleep for a specified period of time unless an event occurs | Yes |
| v8-profiler | Provides cpu profiling of v8. | Yes |
| v8-debug | Used by node-inspector. | Yes |
| sse4_crc32 | Accelerates crc32 calculations on Intel processors only | No |
| buffertools | Utilities to manipulate node.js buffers | Yes |
| ttf2woff2 | A node.js wrapper around Google’s WOFF2 file font format | Yes |
| contextify | A potentially cooler way to spin off separate v8 contexts. Seems primarily to be used by jsdom. | Yes |
| hiredis | A client for redis. | Yes |
| bcrypt | Used to store passwords in a way that is expensive to attack, so provides security if a password file is lost. | Yes |
| sqlite3 | Client for sqlite3 the ubiquitous mini-database. | Yes |
| ws-pure | Ironically this is a pure node.js implementation of WS except it has optional native dependencies | Yes |
| prebuild | Manages pre-built binaries, this is a build time tool, not run time. | No |
| bunyan-syslog | Sends bunyan records to a syslog server | Yes |
| heapdump | Captures a v8 heap dump | Yes |
| node-expat | Wrapper around expat XML parser | Yes |
| websocket | A mostly (but not entirely, there’s some C++ code, which is why it’s in this list) javascript implementation of Websockets. | Yes |
| grpc | Wrapper for Google’s gRPC framework | Yes |
| microtime | Provides current time to a microsecond accuracy. | Yes |
| chromedriver | This is a driver for the Selenium build time framework | No |
| webdriver-http-sync | A sync implementation of the WebDriver protocol used for testing from a PC | No |
| appium-chromedriver | A wrapper for the Chrome Driver used by Appium which uses the WebDriver protocol from a PC to a mobile device | No |
| serialport | Accessing local serial ports, I’m saying this isn’t mobile because mobile devices don’t have serial ports! | No |
| leveldown | Google’s name/value pair high perf database | Yes |
| weak | Allows you to get notified before an object is garbage collected. This is primarily used by dnode which is essentially dcom for node. | Yes |
| runas | Run command synchronously with admin privilege, not even workable unless the phone is rooted | No |
| ursa | Wrapper for RSA operations in OpenSSL | Yes |
| phantomjs-ext | An X86 specific version of the PhantomJS test environment | No |
| pre-commit | Makes sure your node.js code passes tests before committing, this is for development not run time | No |
| snappy | Bindings for Google’s snappy compression library | Yes |
| iconv | Proof that Unicode was a good idea, this library outputs to a bewildering array of country specific text encodings | Yes |
| pathwatcher | Watches for changes in files and directories | Yes |
| usage | Looks up cpu and memory usage on *nix’s. | Yes |
| deasync | Makes async Node.js calls into sync | Yes |
| canvas | Emulates canvas HTML element. I believe it’s primarily used for build and testing scenarios. | No |
| execSync | Executes shell commands synchronously, only meant for development, not production | No |
| bignum | Exposes the big int functionality in OpenSSL | Yes |
| sha3 | Implements SHA3 in C | Yes |
| secp256k1 | Implements ecdsa’s secp256k1 capability | Yes |
| memwatch-next | Detects memory leaks in node | Yes |
| node-zopfli | Trades off time for slightly better compression than gzip | Yes |
| gc-stats | Provides garbage collection stats for V8 | Yes |
| zmq | Provides binding to ZeroMQ | Yes |
| libxmljs | Binds to libxml | Yes |
| node-syslog | Provides interoperability with syslog deamon on unix systems. | No |
| protagonist | Supports specifying Web APIs using a high level language called apiblueprint | No |
| ref | Mostly used by FFI I’m going to argue it’s relevant to smart phones as it helps integrate with C code which runs fine on smart phones. | No |
| marker-index | Operations on text buffers | Yes |
| bson-ext | A binary JSON parser for MongoDB | Yes |
| nodegit | Git support for node | Yes |
| oniguruma | Native bindings to the oniguruma regular expression library | Yes |
| husky | Checks for bad commits | No |
| mmmagic | A binding to libmagic that detects content types of files | Yes |
| appium-selendroid-driver | Interface to Selendroid Android test system | No |
I am currently using node-webkit on osx and a chrome app with browserify. I want to run my nodejs app on mobiles, iOS and Android to be specific. Of all the native modules I only need the net and dgram (udp) modules. Do you know a project that can help?
About the only project I know that claims to ship iOS and Android with node is JXCore but they use a completely different API to talk to the Javascript engine so native add-ons will generally just not work. They claim that moving native add-ons over to their platform isn’t that difficult but I haven’t tried it. Beyond that I really don’t know of any shipping projects that can help. On iOS there are projects like node.app (which, near as I can tell, isn’t really node.js) and v8like (which is node.js but can’t run on stock JavaScriptCore so you can’t actually ship it) but neither is really going to solve your problem. On Android there is anode but it’s using an ancient fork of Node.js.
So unfortunately the current situation is a real mess on mobile. I’m hoping to help make this all better but it’s going to take some time.