PubSubHubbub for JSON
draft-pubsubhubbub-json-00

Status of this Memo

This document is an Internet-Draft and is NOT offered in accordance with Section 10 of RFC 2026, and the author does not provide the IETF with any rights other than to publish as an Internet-Draft.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on December 7, 2010.

Abstract

PubSubHubbub defines a protocol for subscribing to notifications of new or modified items in an Atom or RSS feed. This specification defines an alternative payload format that can be used to subscribe to JSON-based stream resources.

Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.) [RFC2119].

Table of Contents

1.  Introduction
2.  Determining the Topic URL and Hub URL
3.  Subscribing and Unsubscribing
4.  Publication
    4.1.  New Content Notification
    4.2.  Content Fetch
    4.3.  Content Distribution
5.  Notification Message Format
6.  Discovery Document Format
7.  Comparing Payloads for Equality
8.  Acknowledgements
9.  Normative References
§  Author's Address

1.  Introduction

PubSubHubbub (Fitzpatrick, B., Slatkin, B., and M. Atkins, “PubSubHubbub Core 0.3 Working Draft,” February 2010.) [PUBSUBHUBBUB] defines a protocol for subscribing to notifications of new or modified items in an Atom or RSS feed. This specification defines an alternative payload format that can be used to subscribe to JSON-based stream resources.

This protocol is intended only to provide a basic transport format for shipping JSON data payloads. This specification does not define the format of such data payloads except to say that they must be JSON objects. Other specifications may define specific payload formats to enable specific application use-cases.

This specification provides only notifications that a new item has appeared in the set of all items for the topic. However, applications built on this specification MAY define a payload format that affords the use of this format for more sophisticated data synchronization. A goal of this specification is for a JSON-aware hub to be agnostic to the nature of the payloads it is being asked to deliver, such that a variety of application protocols may use this protocol as a transport without requiring changes to the hub implementation.

2.  Determining the Topic URL and Hub URL

The Topic URL is provided by the publisher to the subscriber via a means outside of the scope of this specification.

A publisher SHOULD make available at the Topic URL a Discovery Document as defined in Section 6 (Discovery Document Format) which allows a subscriber to locate the Hub URLs that will accept subscriptions for the Topic URL.

If the publisher and subscriber have a pre-existing relationship, the Hub URLs MAY be determined by another means such as the publisher's API documentation. In this scenario, the Topic URL MAY be an unresolvable URI that merely serves as an shared identifier for the topic between the subscriber and the publisher. However, in this configuration this protocol provides no means for the subscriber to backfill previous items on subscription or after an outage; the publisher SHOULD provide some other means outside of this specification for the subscriber to perform backfill.

3.  Subscribing and Unsubscribing

The specification re-uses the subscription and notification protocols described in section 6 of [PUBSUBHUBBUB] (Fitzpatrick, B., Slatkin, B., and M. Atkins, “PubSubHubbub Core 0.3 Working Draft,” February 2010.). The Topic URL and Hub URL to use for these protocols are those discovered as defined in Section 2 (Determining the Topic URL and Hub URL).

4.  Publication

This specification defines a variation on the publication protocol defined in section 7 of [PUBSUBHUBBUB] (Fitzpatrick, B., Slatkin, B., and M. Atkins, “PubSubHubbub Core 0.3 Working Draft,” February 2010.), adapted to accomodate payloads in JSON format.

4.1.  New Content Notification

When new content is added to a feed, a notification is sent to the hub by the publisher as defined in section 7.1 of [PUBSUBHUBBUB] (Fitzpatrick, B., Slatkin, B., and M. Atkins, “PubSubHubbub Core 0.3 Working Draft,” February 2010.), except that Content Fetch is performed as defined in Section 4.2 (Content Fetch).

4.2.  Content Fetch

When the hub wishes to retrieve new content for a topic, the hub sends an HTTP [RFC2616] GET request to the topic URL. The hub MUST follow HTTP redirects. The Hub SHOULD use HTTP conditional GET mechanisms in its requests to reduce load on the publisher.

The request SHOULD include a User-Agent header field whose value is as defined in section 7.2 of [PUBSUBHUBBUB] (Fitzpatrick, B., Slatkin, B., and M. Atkins, “PubSubHubbub Core 0.3 Working Draft,” February 2010.).

The response to the Content Fetch MUST contain a Discovery Document as defined in Section 6 (Discovery Document Format). In this case the resulting document MUST contain the items property. The hub MUST determine whether any of the payload objects are new since the last Content Fetch, comparing for equality as defined in Section 7 (Comparing Payloads for Equality), and then notify all of the subscribers of this topic about the new objects as defined in Section 4.3 (Content Distribution).

4.3.  Content Distribution

A hub notifies a subscriber of a selection of new objects using a POST request to the subscriber's registered callback URL as described in section 7.3 of [PUBSUBHUBBUB] (Fitzpatrick, B., Slatkin, B., and M. Atkins, “PubSubHubbub Core 0.3 Working Draft,” February 2010.), except that the Content-Type header field value MUST be application/json and its entity body MUST be a Notification Message as described in Section 5 (Notification Message Format).

The request signing requirements in section 7.4 of [PUBSUBHUBBUB] (Fitzpatrick, B., Slatkin, B., and M. Atkins, “PubSubHubbub Core 0.3 Working Draft,” February 2010.) also apply to this request.

When a subscriber provides the same callback URL and secret for multiple subscriptions, a hub MAY combine payloads pertaining to several subscriptions into a single Content Distribution request, identifying the subscription's corresponding Topic URL in each Notification Packet. Notifications for several subscriptions with differing secrets MUST NOT be combined in this way, since the notification signature applies to the entire message payload.

5.  Notification Message Format

This section defines an envelope that is used to deliver a sequence of data payloads in the form of a JSON text.

The root of the JSON text is a JSON object containing a single property called items. The value of this property is a JSON array. This JSON array contains as its items a JSON object representing a Notification Packet for each payload in this message. Each of these Notification Packet objects contains a property called topic whose value is a JSON string containing the Topic URL to which this payload was published, and a property called payload whose value is the JSON object representing the payload.

Application-layer specifications building on this specification MAY define additional properties inside the Notification Message root object. Implementations MUST ignore properties that they do not understand.

6.  Discovery Document Format

This section defines a message format that can be used by the publisher to discover the locations of its the available hubs for the topic and optionally to obtain a backlog of payload items in order to "catch up" on payload objects for which notifications were not received for some reason. This document format is also used to communicate new items from publisher to hub in the traditional loosely-coupled publisher and hub scenario. This document is a JSON text.

The root of the Discovery Document is a JSON object containing a property called hubs. The value of this property is a JSON array. Each of the items in the JSON array is a JSON string containing the URL of a hub as defined in section 2 of [PUBSUBHUBBUB] (Fitzpatrick, B., Slatkin, B., and M. Atkins, “PubSubHubbub Core 0.3 Working Draft,” February 2010.). All of the hubs in this list MUST be hubs that are able to accept subscriptions for the topic represented by the discovery document and are able to provide hub services for JSON-based stream resources.

The root object of the Discovery Document SHOULD also contain a property called items. If present, its value is a JSON array of JSON objects each representing payloads recently added for the topic.

7.  Comparing Payloads for Equality

Two Payloads are considered to be equal if they are byte-for-byte equal after serializing to a JSON text, where both Payloads were serialized using identical serialization settings with regard to pretty-printing and character encoding. Implementations MAY use an appropriate hashing algorithm on the resulting JSON text to reduce comparison and storage overhead, but the chosen hashing algorithm MUST have a low likelyhood of collision within a set of recent payloads of a given topic URL.

8.  Acknowledgements

This document builds upon the work of Brad Fitzpatrick and Brett Slatkin in defining the PubSubHubbub subscription mechanism, and an initial notification format suitable for communicating changes to syndication feeds. It also takes inspiration from XMPP, a work of the IETF XMPP working group in the form of the "smarts at the edges" design that this specification affords.

9. Normative References

Author's Address