Task 1.1: The SwarmOS

[ Jones, Kubiatowitz, Lee, Rabaey, Wawrzynek ]

The SwarmOS is the collection of essential services that make swarm applications possible, trustable, robust, and efficient. This task will develop the core components of the SwarmOS that will serve as a distributed executive and resource manager for TerraSwarm applications. The SwarmOS will mediate the needs of applications for services and clusters of resources where resource clusters may be, for example, a portion of a processor's resources, or a slice of bandwidth. The SwarmOS must be distributed, resource aware, governed by service-level contracts, capable of restricting admission and of guaranteeing access to critical services. It must function in a heterogeneous network, where multiple technologies are combined and where connectivity may be disrupted or only available sporadically. The SwarmOS will provide a cross-platform implementation of a QoS-aware communication and archiving service called the global data plane (GDP), location-aware routing and caching services, service discovery, and resource brokerage.

A key goal is to create the "BSD for the Swarm," an open-source, well documented, thoroughly tested, and bulletproof suite of core SwarmOS utilities. The task includes leveraging efforts to provide an open and free certificate authority for the web, applying similar mechanisms instead to scalable key management in swarm devices. A second direction concerns how to provide sufficient structure and metadata for streams on the GDP to enable effective use of learning techniques. This leverages the extensive experience of the TerraSwarm machine learning team with the structure of large data sets and the formal contracts work of theme 2. We envision self-describing data sets that codify their structure in the language of contracts, and machine learning algorithms that adapt and are able to aggregate even data coming from disparate sources with diverse formats and content. A third direction concerns resource management, specifically resolving contention for shared resources such as network bandwidth. Again, the language of contracts provides a potential formal framework for this work.