The Micro Web: Putting the 'Web' back into 'Web 2.0'! ----------------------------------------------------- Longer Treatment Duncan Cragg ------------------------------------- Web 2.0 in the form of MySpace, Flickr, YouTube, Google Mail, blogs, tagging, etc., is providing real value supporting some basic human needs to meet, share, collaborate, create, explore and play. Web 2.0 is even showing ambitions towards becoming an 'interactive hyperdata Web': its value comes from technologies and approaches that are two-way, interactive and data-driven: Ajax, Web Feeds, JSON, Web APIs, Microformats, tags, identity and profile support and self-publishing tools. Further, at least with some of these approaches (specifically Microformats, OpenID, Atom feeds and Protocol, JSON, etc.), Web 2.0 can still be compatible in technology and architecture with the core Web on which it rides. ------------------------------------------------------------------- However, there is one area where Web 2.0 often breaks the Web, resulting in 'unscalable walled-garden' sites: Ajax scripting. Specialist client-side Javascripts can break the ubiquitous, generic, declarative Web-client model. Hand-crafted, imperative, thick clients are usually tied to specific application servers (through function-call APIs), can't be indexed by crawlers, miss the benefits of the Web's cacheing architecture and can break the browsing experience (bookmarking, linking, history). To round off their problems, they are hard to make cross-browser compatible and they add to page load times. As a result, we are frequently subjected to a fragmented user experience where there's an application for everything, but nothing is integrated. Our Web 2.0 Email, IM, Blog and Feed reader tools barely interoperate. Web 2.0 is no longer something we can engineer declaratively to generic clients. It won't interoperate like the Web does without URIs and standard content types instead of the proliferation of APIs and hard-coding. And Web 2.0 won't scale like the Web does without proper cacheing, URI partitioning and simple stacks. Web 2.0 is drifting apart from the core interoperability and scalability of the standards-based, open and declarative Web. Finally, a declarative hyperdata Web, interactive or not, built on such fragmented, imperative foundations is unlikely to have a secure, open and interoperable future. ------------------------------------------------------------------- So how can we deliver the Web 2.0 benefits and technology without losing the benefits of the Web Way? How can we accomodate the Web 2.0 drivers to create an open and declarative platform that is better aligned with the scalable and interoperable Web that spawned Web 2.0 in the first place? I suggest we model the two-way, interactive data Web on the current, largely one-way, static document Web. We should build the hyperdata Web on the hypermedia Web. I propose the 'Micro Web' (or the 'ยต-Web'). The Micro Web is a set of Web technologies, standards and conventions on which to build a better, Web-conformant Web 2.0. The Micro Web is made from several open Web ingredients mixed together with some well-applied Web 2.0 technologies and approaches. The Micro Web is an interactive hyperdata Web. At the heart of the Micro Web is a universal Javascript which effectively extends the one-way, static document browser into a two-way, interactive data portal. This browser extension delivers 'Declarative Ajax' or 'Declarative Comet'. Versions of it would be available with infinite cacheability at an address like: http://highly-available.org/micro-web-1.2.js This script takes two steps from the one-way, static hypermedia Web to a two-way, interactive hyperdata Web that delivers a better-behaved Web 2.0 platform. The steps are not unexpected: the first is to move from hypermedia to hyperdata. The second is to add the two-way interactivity. ------------------------------------------------------------------- Step one is the move to a data Web of actual data resources. This hyperdata data Web is compatible with, and can seamlessly grow from, the existing hypermedia Web. It is also 'Web 2.0 compatible', with many resources using Web 2.0 data approaches such as Microformats and JSON. The Micro Web's resources are called MicroResources. MicroResources form the Micro Web by linking together through their URIs. A MicroResource is a small chunk of data in a standardised format. A typical MicroResource will be 1Kb in size. Hyperdata has a finer grain than hypermedia. Examples of MicroResources are Microformats, RDF, Atom entries, Web Feeds, user profiles, calendars, forms, conference slots, CD catalogue entries, photos with captions, lists of URIs given some tag, widgets, stock items, etc. They may be encoded in XML, YAML, JSON, HTML, XHTML or even CSV. MicroResources may be static, they may be rarely changed or they may be very highly dynamic. The latter, dynamic ones appear to be internally animated and some will want to interact with each other in mashups. How will the Web of MicroResources be seen in the browser? The universal 'micro-web.js' script assembles a view on some slice of the Micro Web. Any MicroResource type it sees will have a consistent rendering according to the standard content type it uses. For example, a date may be rendered as a calendar widget, a Geo Microformat as a Google map. Lists and aggregates of things may have expand and collapse or paging buttons. Indeed, entire web pages, documents and web sites may be compiled from many nested and interlinked MicroResources. The difference between this approach and rendering a GUI graph, and the main difference to XForms, is that each MicroResource has its own URI in the graph, and can come from any server - the 'model' is the whole Micro Web! Further, the content types, rendering and fine interactions are standardised, not customisable as in XForms. Also, the Micro Web can be JSON as well as XML. Finally, the whole visible frame is a Micro Web view, not just a separate form section in a larger document. Breaking up client views this way has advantages, some of which have been noted in the past by Mark Nottingham (see hInclude) and others (see CSI): each chunk in the page can be given different cacheability, speeding up page load times. Also, being loaded dynamically means page components can be fetched on demand (e.g. when a containing resource is opened in the middle of the page). ------------------------------------------------------------------- That's the static part moved from hypermedia to hyperdata. Step two is to move from static to dynamic, with two-way interactivity. There are two directions we need to consider: actions from the user onto remote MicroResources, and updates on those animated MicroResources to be notified to the user. If it's about anything, Web 2.0 is about the user - about their content and interaction. The standard Web and standard browser sees the user as essentially a consumer, to be tracked on a site-by-site basis with cookies. The user in this model is out-of-band: not part of the Web. The Micro Web, however, adopts a more user-oriented and user-empowering approach: to create and animate an actual User MicroResource within the browser. The User MicroResource may hold their OpenID, their hCard, and even what they're 'saying' right now. It may be one of many personas that the User wants to adopt in different contexts. It may have a list of links to other MicroResources the User is 'holding' at any time. This active User MicroResource represents the current view and the actions of the user on and within the Micro Web. It could be seen as representing both the View and the Controller over the Micro Web's Model, in the MVC pattern. ------------------------------------------------------------------- So, what interaction model is used in this 'distributed MVC'? What does a User MicroResource action look like and how does the User's view get updated when a MicroResource changes? The answer, of course, is to use the REST style. In particular, to implement this MVC using a Distributed Observer Pattern in a RESTful way. The Micro Web uses a more constrained form of the REST style, where it's just about state transfer by pull or push: pull being traditional GET, push being state update notification through POST. As noted above, the URI and standard content-types containing URIs remain at the core of the model. Logically (ignoring what pipes go where!), the User MicroResource can observe, say, a stock-price MicroResource. Or a clock MicroResource. Now, being an animated thing, the clock can observe the User in return. It may present a switch for AM/PM vs 24-hour times, for example, and respond to the User acting on it to flip it over. Changing MicroResources being watched by a User are updated spontaneously like widgets. A User will subscribe to a selection of such MicroResources, and the updates of those subscribed MicroResources will be pushed into the browser for the User to see. The User then acts on or affects MicroResources by being themselves observed. ------------------------------------------------------------------- So MicroResources are not inert like most Web Resources - they are active. But MicroResources are not objects - at least not in the traditional sense: they only operate in this Observer Pattern mode - watching or subscribing to each other - and they don't have any methods, just mutual state transfer. In the Web 2.0 world, we have to see resources more as dynamic, animated, interacting data than as inert media. MicroResources feel like objects, but objects are not the Web Way - instead, see MicroResources as rather unusual objects that have public data and no special methods! To (RESTfully) interact from within one server host to another, or even within a single application process, a MicroResource can GET the state of one of its peer MicroResources in the Micro Web. It can also proactively POST its own state to a peer. Further, a MicroResource can subscribe to a peer, in the style of the Observer Pattern, and receive notification when the target changes, via POST. This notification can be either the fact of the change (with an Etag) or, for most, 1Kb-ish MicroResources, the whole new state. If cacheing is definitely required, the former approach is needed as it allows the new state to be re-GET-ed via the cache. Observer-Pattern Subscriptions are out of the scope of REST, and are handled in a header on the MicroResource, which lists incoming and outgoing subscriptions. This pattern, if used within an application server, would be best implemented SEDA-style, with event queues. Once running this Observer Pattern inside a server, the next logical step is to animate MicroResources with declarative rules triggered by peer MicroResource state change events - in the manner of spreadsheet programming. A Micro Web mashup is programmed in this declarative way, with remote, cached-in MicroResources. This simple MicroResource interaction model or Distributed Observer Pattern is powerful enough to power any Web 2.0 site or application. ------------------------------------------------------------------- We don't need to change HTTP to do all this, although we may need some compromises to get around the normal asymmetric, client-server configuration. Polling is one example of this. A small, unconventional addition within HTTP would be the inclusion of a Content-Location: header on a POST request, giving the source URI of the content being POSTed. Of course, a bit more tunnelling of this pattern of interaction would be needed to get into browsers: using the Comet Ajax pattern. There would be a notification pipe left open, receiving a stream of MicroResource change alerts, and reverse- or psuedo-GETs on the User MicroResource. ------------------------------------------------------------------- What specifications need to be written up? Here's a sample: Building the dataweb - how links look in all encodings, including parent/child relationships. Form of MicroResource URIs, use of UTF-8 (mandated!). Full set of standard content types for micro-web.js to understand, render and act on. The precise details of those Observer Pattern headers, including subscription lists and how they may be timed out. Use of GET and POST, Content-Location: and Comet tunnelling. Full spec of the User MicroResource, including hCard, OpenID and actions. ------------------------------------------------------------------- The purpose of my presentation of these ideas at WWW2007 is to kick off a community of interested developers and standards-makers to help me build the Micro Web. In my talk I will list the necessary standards which collectively define the Micro Web. I'll show how we can build a 'Web-compatible Web 2.0' from industry-standard parts. The Micro Web can evolve the scalable, interoperable hypermedia Web into an interactive hyperdata Web, to which the highly scalable and productive declarative programming style naturally applies. -------------------------------------------------------------------