Thali’s mission is to create an open source/open standard peer to peer web. This means both local peer to peer (e.g. you and I are standing next to each other, can our devices talk directly even if there is no Internet?) as well as remote peer to peer (e.g. you and I are both on the Internet but aren’t near each other). We have outlined a series of stories (or milestones or sprints or whatever your favorite term is) that lead us where we want to go. Today we released story 0 which contains almost all of the core technologies we need to make Thali fly! There is a ton of work to do but at least now we have a foundation to stand on.
What’s in the box?
Story 0 provides the basic pieces needed to let Android phones sync to other local Android phones and iPhones sync to other local iPhones. Unfortunately local synching between Android and iPhones is a nightmare (see here) that we will tackle later.
At the bottom of our tech stack on Android is Wi-Fi Direct Service Discovery which we use for finding other Android phones running Thali software and Bluetooth which we use for high bandwidth data exchange between Android phones. See here for more on these choices. One thing we have already learned (I’ll be writing an article on this soon enough) is that Wi-Fi Direct Service Discovery is seriously broken. We will be replacing it in future releases.
In the case of iOS we use Bluetooth Low Energy (BLE) for discovery and the multi-peer connectivity framework for high bandwidth data exchange. See here for more details on our choices. The main lesson we learned here is that running BLE in the background on iOS is a serious nightmare. See here for details.
Above both of these discovery and high bandwidth local radio stacks we run node.js or a variant of it called JXcore. Yes, we are running Node.js on the phone! JXcore is a fork (in the good sense) of Node.js that supports running Node.js on Android and iOS (as well as Linux, Mac and Win32). JXcore’s magic comes in two parts. The first part is that they put in an abstraction layer so they can run Javascript engines other than just V8. Right now they support spider monkey and V8. The second part is that they support being run as a Cordova Plugin that includes a “mobile” library which provides basic functionality that someone writing a node.js app on a mobile device needs. We investigated (see here and here) JXcore in depth and liked what we saw. With JXcore we can write the bulk of our logic in Javascript and use it across both Android and iOS. To make this work we use JXcore’s native API layer to expose the native radio stacks and we put over it a standard Node.js API, see here for the spec.
The end result is that we have a node.js level API that can tell the Node.js application when a peer is in the area and let the application open a TCP/IP connection to that peer. In fact we even put in a mux layer (a big shout out to the multiplex folks whose code makes this work) so that node.js clients can open as many TCP/IP connections as they want to the peer and we will multiplex them all down to a single connection for transmission over the air.
Right now the layer that allows for discovery and connectivity is not broken out on its own but eventually it will be so those who just want to have discovery and connectivity but not the rest of Thali can use it stand alone.
On top of that discovery/connectivity layer we built something we call the replication manager. This is a thin layer of code around the ever amazing PouchDB. PouchDB is a Javascript implementation of the CouchDB HTTP based multi-master synchronization protocol. It lets us automatically sync between peers and get them into consistent states.
Wrapped around all of this is Cordova. In fact our main project is actually just a Cordova plugin. We use Cordova for a number of reasons. First, it lets us our customers provide a consistent UX that can work on both Android and iOS. Second, it provides us with a framework with which to manage our native JXcore dependencies and get them built into apps. Eventually we’ll support using Thali’s infrastructure independently of Cordova but for now you have to use Cordova to use Thali.
Can I try it out?
Heck yeah! Probably the easiest way to play with it is to grab two Android phones or two iPhones and head on over to our sample, the postcard app. The instructions for installing it are right on the ReadMe. The app itself is more of a proof of concept than a real app. It just uses all the stuff I talked about in the previous section. It still has lots of challenges including (on Android) a totally unreliable discovery process and a horrible UX that amongst other things requires hitting a refresh button to see if new records came in from the other device. Yuck. There are also various UX hiccups, especially on iOS. The good news is that we do have a new version of the postcard app coming down the pipe (you can see its current state here) that will hopefully suck a lot less.
If you want to see the Thali Cordova plugin head over here, we even have docs. But please keep in mind that right now we are at the start of our journey.
What’s next?
We are just writing a few tests to finish up story 0.0 which just adds support for generating public keys and PKCS12 files. Then we head directly into story 0.0.1 which handles identity exchange. Then we intend to head over to story 0.8 which is where we will introduce a new discovery system for Android that hopefully won’t suck as much. At that point we will probably start making much louder noises about the project. But I sort of hope we also get story 0.6 done which handles finally activating TLS for all of our connections. See here for issues on Thali, TLS and node.
But if you look at our stories page you will see that we actually have a ton more work to do. Just to get local peer to peer working we need to bring in ACLs, notifications, Quotas, the BLE Hub and Android/iOS interop not to mention securing the connection between the Cordova webview and Node.js as well as life cycle issues (e.g. apps starting, going into the background, low battery, etc.) and more.
And then of course we want to go to Internet based functionality including supporting discovery over local Wi-Fi APs (read: mDNS/SSDP) as well as running over the open Internet which due to firewalls means bringing in fun things like Tor and in the case of iOS some kind of file sharing infrastructure (hey OneDrive! Although DropBox and others should work fine, it will be pluggable).
And that is just V1. There is also our hope to get to V2. That includes fun things like enabling a phone to host more than one Thali application, mesh based store and forward support, personal device mesh’s, running on desktop and embedded OS’s and one of my personal favorite features, project Pot Luck. Pot Luck is meant to provide an ad-hoc DHT to allow people to create aggregate services using whatever devices are around. I also muse about adding some kind of Paxos support as well.
So lots of work to do!
How do I learn more and Get Involved?
To learn more it’s probably best to walk around our website. Our architectural overview is probably the best big picture view. If you are interested then let us know! Our contribution page has tons of way to contact us including our mailing list, Twitter and git issues. We also have a blog and backlog.