Accelerating the Open IOT ecosystem
On Nov 22, I am co-moderating an event the event Accelerating the Open Source IOT ecosystem. The event brings together thought leaders and practitioners who are passionate about Open source and IOT.
Here, we are primarily speaking of Royalty free, non proprietary, Open source software for the Internet of Things. Of course, that does not exclude other software paradigms – which are a part of the ecosystem.
In this longish blog post, I will discuss how the webinos project fits in with Open source IOT especially in the context of its role for node.js
I have been leading the webinos IOT hub efforts – the blog comprises of insights and contributions from others at webinos especially Dr Paddy Byers, Dr Nick Allott, Dave Raggett(W3C) and Giuseppe la Torre
It’s hard to describe webinos .. and I once jokingly said applying a Star Trek analogy that ‘webinos boldly takes node.js where no node.js has gone before!’
So, I will use the paradigm of node.js to explain these ideas
Node.js and webinos
Essentially, in webinos we embed an agent into a device that allows them to be part of the Personal Zone of devices managed by a person. The agent is implemented with Node.js and it enables secure mutual authentication of devices in the Zone. Thus, webinos extends the traditional web runtime with a suite of APIs for discovery, messaging etc.
The analogy of an email server is applicable here. Like an email server, messages are stored in the ‘cloud’ but can be accessed by local devices. But webinos also adds distributed functionality i.e. services owned by one person can be shared with others (under policy limitations). In an IOT sense, that means a sensor owned by one user can be discovered and shared with another user
Webinos has the following characteristics:
- Privacy enabling i.e. which helps users in re-establishing control over your devices and personal data.
webinos can be applied to many industries and applications and is initially focussed on four specific areas or gateways: TV, Automotive, Health and Home Automation Gateways. Note that – this blog and discussion relates only to the Home automation/ IOT areas of webinos.
The description of webinos (non-proprietary, cross-device, secure, distributed platform which helps in re-establishing control over your devices and personal data) sounds daunting but in practice, it means :
a) Devices you own can be translated into a service that can be discovered and shared with others (based on policy settings) and
b) Similarly, devices owned by others can be discovered by you as a service and can be accessed (again subject to policy approval).
This has implications for IOT/Home automation/Smart cities
Consider a Smart city scenario:
One department of the city has deployed pollution sensors and temperature sensors. Another department of the city wants access to the same real time information. Indeed, considering Open Data principles, it could be any person – for example developers running a hackathon. In this scenario, the department which owns the sensors can grant access to the sensors to third parties based on Policy scenarios. Indeed, these sensors could become ‘discoverable’ and could be accessed by any third party as needed.
This is achieved through three ways:
- Open technologies(specifically node.js)
- Implementation of Personal zones and
- The webinos Dashboard
Significance of Node.js
(this section – acknowledgements to Dr Paddy Byers)
Most important of all, though, is the ability to build apps using external modules – not built in to the core – provided by third parties. There is a very active ecosystem of developers of these node “modules” which gives you access to a massive catalogue of libraries and frameworks. By having this structure, node can concentrate on maintaining a focussed, high performance, stable and common core, and the module ecosystem can provide huge diversity in libraries and frameworks. Unlike other environments – say Ruby with Rails – there isn’t a single framework architecture that becomes an encumbrance or constrains how things are built. There is diversity in the ecosystem and it isn’t held back by centralised coordination or the need for a single view on how things are done.
Node was one of the first projects whose community engagement was fuelled by Github and that mindset – free, decentralised, and open – has been the core ethos of the developer community for node’s core, the module ecosystem and end-user developers. Although node is now owned by Joyent which has its own commercial mission, node remains open and sees contributions from many individuals and organisations.
node is primarily used for building the “front end” for web sites (i.e. the part that directly handles incoming requests and sends responses). Some organisations use it just for the front end but many sites are built top-to-bottom with node.
Node has a number of key advantages.
1) The principal advantage is scalability. node is based on JS; it is event-driven and single-threaded. While this might at first sight seem to be a disadvantage – running an inherently parallel service on a single-threaded runtime – it turns out to be its key advantage. The reason is that the cost of handling each new request, and in particular the cost of each outstanding request, is very small compared with systems that spawn threads or processes to handle each request. Each request is handled – processing the request, resolving the request path and parameters, triggering database or other IO – but then instead of waiting the system then returns to the idle state ready to handle a new request. The resources occupied by the pending request are simply a few objects and buffers, so many thousands of requests can then be pending on a single server. Secondly, state is easier to share between requests, which minimises the state that needs to be persisted somewhere. A single server can therefore handle tens of thousands of connections and concurrent requests.
2) The next key feature is its accessibility. node is small at its core – which means a small learning curve to get started – but has a rich ecosystem of modules that enable you to add functionality quickly. The openness of the platform and modules, the support available from the community, and the sheer diversity of things being created, mean that you’re rarely on your own when trying something new. If you look through the various testimonials on the nodejs.org site you see multiple organisations using node to power their mobile apps or mobile sites. There are several reasons why it is well-suited to this.
3) Suitability for mobile apps – First, these mobile backends – whether serving html or APIs – require huge scale. Any mobile app with even modest adoption can generate hundreds or thousands of requests a second. node allows these services to scale to this level much more readily and cheaply than with competing platforms. Many organisations, even though they have an existing backend for their mainstream website, will take a “clean sheet” approach to building their mobile platform and node is then a natural choice.
4) Further, mobile apps are increasingly dependent on realtime connections where data can be pushed from the server to the device (egg with long polling or websocket connections), rather than being solely conventional sites or http APIs. node provides ready support for realtime connections (either directly or with helpers such as socket.io) and realtime push-dependent systems can be built far more easily than would be possible with Rails or PHP, say. LinkedIn, for example, have built their entire mobile backend in node and you can see other examples on the node.js site
5) You can also run node on the mobile itself. node is inherently portable – V8 supports multiple CPU architectures and Chrome itself obviously runs on ARM and MIPS and other CPUs as well as x86. node’s footprint on the OS API surface is small – networking, filesystem and events essentially – so it makes it readily portable to multiple environments. There is a port of node to Android and a framework that allows you to build Android apps with node, and there is also an experimental port to iOS.
As devices grow ever more connected, they will increasingly be simultaneously both clients and servers for network services. That doesn’t necessarily mean they will be serving web apps, but your phone has a wide range of data sources that are interesting to exploit – location, camera, proximity via Bluetooth, say, as well as the personal information in contacts, etc. node is a framework that allows you to create servers very quickly for all sorts of functionality.
You would think performance is an issue, but it’s not; modern devices are so powerful that they have plenty of processing power for the kinds of services you would think of. Anywhere you can run a browser you can also run node.
Having services that are always on is an issue for battery life. There needs to be a way of ensuring that an idle service is really idle and doesn’t drain the battery.
Intermittent network connectivity is obviously an issue – services won’t always be reachable.
Webinos and node.js
Webinos has gone further than most other projects in exploring node.js in different platforms. Specifically, it is addressing two separate issues:
a) How to expose device functionality as network-accessible services, and
b) How to create a portable application environment based on JS.
These have implications for IOT
The main technical contribution of Webinos has been that of privacy and access control for services exposed by a device such as a phone or car or TV. Webinos has the idea that a “personal cloud” can be augmented by devices and the services they can each expose; and has created a framework for access to those services, both peer-to-peer and via the cloud.
This is similar to a distributed “plug and play” for personal services; it’s not just about enabling discovery and access, but enabling the owner of the device to give access selectively and to set policies for access. Webinos addresses the range of trust scenarios on which that access might be based – social network relationships, physical proximity, etc.
Webinos is itself built with node and you can download the specifications from the webinos site and from github for webinos
An overview of webinos technology
Within this context now, it’s easier to understand the significance of webinos for IOT
- Today companies provide services, but require centralization of personal data over which you have little control, making it hard to switch companies
- Personal Zones provide an architecture for reclaiming control
- You decide what/when to share with 3rd parties
- This facilitates intent based smart search
- Your data is managed within your zone, by the services you install
- This works well for IoT devices
webinos Personal Zone Hub (PZH)
The Personal Zone is a conceptual construct, that is implemented on a distributed basis from a single Personal Zone Hub (PZH) and multiple Personal Zone Proxy (PZP)s
The critical functions that a Personal Zone hub provides are:
- An fixed entity to which all requests and messages can be sent to and routed on – a personal postbox as it were
- A fixed entity on the web through which requests and messages can be issued, for security and optimisation reasons.
- An authoritative master copy of a number or critical data elements that are to synced between Personal Zone Proxy (PZP)s and Personal Zone Hub (PZH), specifically
- Certificates for Personal Zone Hub (PZH), Personal Zone Hub (PZH) mutual authentication
- All policy rules, for distributed policy enforcement
- All relevant context data
- The functions therefore that a Personal Zone Hub (PZH) can support are
- User authentication service
- Personal Zone Proxy (PZP) secure session creation for transport of messages and synchronisation
- A webinos service host: a Personal Zone Hub (PZH) can host directly Services/APIs that other applications can make use of.
- Context sync: the Personal Zone Hub (PZH) should act as the master repository for all context data
webinos Personal Zone Proxy (PZP)
- The webinos Personal zone satellite proxy, acts in place of the Personal Zone hub, when there is no internet access to the central server.
- In order to act in its place, certain information needs to be synchronised between the satellites and the central hub.
- This information has already been listed above.
- The Personal Zone Proxy (PZP) fulfils most, if not all of the above functions described above, when there is not Personal Zone Hub (PZH) access
- In addition to the Personal Zone Hub (PZH) proxy function, the Personal Zone Proxy (PZP) is responsible for all discovery using local hardware based bearers (bluetooth, zigbee , NFC etc)
- Unlike the PZH, the PZH does not issue certificates and identities.
- For optimisation reasons PZPs are capable of talking directly PZP-PZP, without routing messages through the PZH
- A webinos application runs “on device” (where that device could also be internet addressable i.e. a server).
- A webinos application is packaged, as per packaging specifications, and executes within the WRT.
- A webinos application has its access to security sensitive capabilities, mediated by the active policy.
- A webinos application can expose some or all of its capability as a webinos service
A webinos service is a collection of functions and events, that are accessible by an webinos application
An webinos service must take note of the following parts of the webinos specifications
- Discovery: a service must be discoverable and be able to describe itself to the application in accordance with the discovery specification
- Messaging : a service must be able to receive and respond to incoming RPC messages
One of the critical innovations of webinos, is the virtual overlay network that allows different applications and services to talk to each other over many different interconnect technologies. Not only are the interconnect technologies for local messaging, there are three different scenarios in which this communication can take place. These are highlighted in the diagram below.
Connecting to a full smart device, that hosts both a PZP (therefore can host native APIs presented as services) and a WRT (so can host webinos applications exposing webinos services)
- Connecting to a dumb device, it hosts a PZP but not a WRT. This means that it can expose only native APIs, not webinos applications
- Connecting to a super-dumb device, it hosts neither a PZP nor a WRT, but can expose webinos services – if the client PZP hosts a customised driver
Two other aspects complete the webinos vision – the microPZP and the Dashboard
MicroPZP is an implementation of the PZP when the device is too low spec to deploy a full PZP. A device supporting a MicroPZP has a target to 2mb device range
(acknowledgements to Giuseppe la Torre for this section)
The dashboard brings it all together for the user. In the near future, our houses will be “populated” of several “smart objects” which can be remotely controlled by users. Some efforts will be necessary to create a common platform for the “physical object virtualization”. Webinos provides support for the IoT domain, defining and implementing APIs for generic sensors and actuators.
Webinos provides drivers for Arduino boards, OBD electronic control units, ANT health sensors.
One of the most important feature we will expect from the iot ecosystem is the physical mashup.
The webinos home controller is a web application which, relying on the webinos platform,
allows users to
i) Create customizable UI to display information from user’s sensors. Using the drag&drop paradigm the user can create its own user interface with charts, gauges, text label, and so on. And display information about all the sensors which belong to his personal zone. This UI can be saved and then displayed in each kind of user’s device (TV, in-car, tablet). This part of the app could be easily extended, recently improved with the possibility to display user’s position (a webinos service) on a Google map.
ii) Add “logic” among the the smart objects by means the definition of rules – This is a good example of physical mashup: sensors and actuators (but theoretically each type of webinos services) can be used together to create logic rules of type: if CONDITION then TRIGGER.
Using the drag&drop paradigm, user can move on the UI
-) Input elements (sensors, user input textfields)
-) Condition elements (<,>,AND,OR)
-) Output elements (actuators)
An important webinos feature which has been integrated into the home controller application is theExplorer.
The explorer is a common interface for webinos applications which allows them to get access to the services exposed by user’s devices inside the personal zone.
In the case of home controller app, the explorer allows users to pick services (sensors or actuators) among those inside his personal zone or those belonging to a friend’s personal zone.
The webinos project takes node.js to different platforms – including IOT. But it does a lot more. We will be discussing these platforms (webinos, OSIOT, hypercat, web of things etc) at the accelerating the open source IOT ecosystem event
A big picture of webinos is as below
Many thanks to Webinos project esp. Dr Nick Allott, Dr Paddy Byers, Dave Raggett(W3C) and Giuseppe la Torre