Labs/SocialAPI
What would a "Social API for the Browser" look like?
There are many social actions that we perform in the browser, today, using social networking services. We might want to:
- know which of our contacts are available - either immediately, for chat, audio, or video; or "later", for message delivery and sharing.
- start, or join, conversations with our contacts
- see updates and links from our contacts. For some users and times, we'll want to see updates as soon as possible; at other times, we'll want to wait until we have time to reflect and process.
- share updates and links with some, or all, of our contacts
- recommend, or express support, for things that we discover on the net
- receive recommendations, or engage in communications about, activities that we're doing on the net right now. This could be immediate - "watch this with me" - or it could be delayed - "this is my favorite movie on the site", or "it's your turn".
By integrating social services into the browser, we can make these interactions faster, easier, and richer. How do we translate these use cases into a general platform for social interaction, which supports all of our use cases, is secure, robust, and safe, and provides end-user customizability and opportunities for innovative service development?
A proposal: the "Social Platform Provider API"
We propose that the browser maintains a list of "social platform providers". Each of these can be defined by a small text file, distributed through the web, and discoverable through normal page markup techniques. This file could be, in fact, an Open Web Application Manifest - which contains the name, icon, and service data needed for the browser to communicate with the web property.
Each manifest contains several URLs, which, taken together, allow the browser and the service provider to work together to provide social experiences in the browser. These addresses are loaded into the browser in a slightly different contexts than a normal web page, and have some "special powers," reflecting the deeper relationship users have with a social platform, and assuming a higher level of trust by the user. The service provider can update an always-visible screen area, request notification for "pushed" events, and open windows that are "pinned" to a single web domain. Because of these special powers, it's important that the user is always in control of which social platform providers get loaded, and can switch them on and off whenever he or she likes.
The Social API includes both "back-end" integration, which is largely invisible to the user, and "front-end" integrations, which display new user interface elements to the user. We describe both here.
A persistent provider Worker
At service-startup time (which will often be at browser-startup time), a JavaScript file is retrieved from the social platform provider, and installed into a persistent, always-running execution sandbox. This is a "headless" connection from the browser to the service provider; it has no window, and exists purely to coordinate messages between the browser and the service provider. The implementation would be very close to the W3C Shared Worker proposal.
The Worker has access to a limited subset of browser functionality. It can open persistent network connections (XMLHttpRequests and WebSockets) to the server. It can access data stored for the service provider's domain through IndexedDB. It can create Notification objects and dispatch them to the browser, asking for the user's attention (more about that, below). It can register for notification of "user idle" events, indicating that the user has not interacted with the browser for a while, or has returned after an idle, to let the service update the user's "presence" status. It runs with the same cookies that the service provider's site would normally have. It is a central "switchboard" for all the other pieces described below, providing coordination and communication.
Note that it can *not* observe all of the user's activity, see 3rd party cookies, or access any browser-level data. It exists to provide an "always-on" connection to the service provider which is subject to the same sandboxing of data and control that exists in a normal web page context.
The "Social Provider List"
The browser has a list of social service providers, and understands that some of these are "active", while others are "inactive" - meaning that their Worker is running, or not. This list is made visible to the user through an easy-to-access element - for example, a toolbar button on desktop browsers, or a swipe-visible side panel on mobiles.
The list of social providers is user-configurable, using a discovery and configuration system that would be part of the browser's user controls. Sites could advertise their ability to provide social features through discoverable markup, and directories could maintain lists of sites that provide these functions.
The "Social Provider Sidebar"
The browser will create distinctive visual areas where a page from an "active" social provider is loaded and displayed. The corresponding workers are "visible"; the other active providers are "hidden".
On desktop browsers and tablets in horizontal orientations, we propose that visible providers be loaded in a sidebar - a vertical strip of screen space, positioned to the side of the main content region - and that the browser knows how to display it "maximized" (as tall as the window), or "minimized" (shrunk to the height of the location bar, so that the content window is unchanged). Switching between "maximized", "minimized", and "hidden" should be a simple, single-click-or-gesture action. The browser will provide JavaScript or CSS properties that allow the content to lay itself out responsively to the size of the region it is given. On mobile browsers, the social provider sidebar might be normally hidden, but visible with a swipe gesture.
This provides a space in which the social provider can provide whatever experience it likes - a list of updates, current games, available friends, notifications of high-priority events, or more. When "minimized", the provider has a smaller space, which is still enough to display a small number of ambient notifications, to get the user's attention. The space is updated independently of the user's normal browsing activity, and is "pinned" to the service provider's domain - it cannot navigate to other sites. It sends messages to, and receives updates from, the Worker, which allows it to handle real-time updates and messages from the service provider.
The sidebar can open new tabs by calling the window.open method, just like any other web page. Like any other web page, these new tabs are subject to popup control logic - a user click is required to open a new tab. The new tabs are opened into the current window on which the sidebar is displayed. This allows a service provider to recommend an interesting piece of web content, which the user can immediately visit and start interacting with.
"Service-Pinned Windows"
To handle interactions that are initiated from a service sidebar that require more persistence and screen space than just a sidebar, we propose a new feature of the window.open function, which creates a browser element which is "pinned" to the service provider's domain. This can be used for text, audio, or video chats, or other non-modal activities which are larger than a browser window. Service-pinned windows have access to the Worker, and can share messages with it to push and receive updates.
The browser could be in a themed "tearoff" window with no navigation bar and a simplified menu, or it could be "docked" to a browser window. On mobiles, depending on the platform, these windows could follow the user around computationally or visually, representing ongoing social interactions.
The browser implements a new command, which is made visible through a button in the navigation bar (but perhaps also in menus, keyboard shortcuts, etc.), to "recommend" a piece of web content. Under the covers, this is implemented by sending an event to the Worker containing the URL of the object the user has recommended - which could be the current page, but could also be an embedded object such as a picture. The ability to handle recommendations is advertised by the service provider in its manifest, along with whatever icon and language the provider uses for this operation.
Alternatively (or additionally), the service provider can signal its ability to handle "Share" operations, which require more interaction with the user than a "Recommend". When a "Share" operation is initiated (perhaps by another button, or a click-and-hold on Recommend, a second click following the Recommend, or a context menu action), a small floating panel containing a frame rendering the service provider's "Share" URL is rendered above the current browser window. An event is dispatched to the frame indicating what object the user shared and the frame can update itself dynamically using JavaScript or AJAX techniques if it wishes. When the "Share" frame is done interacting with the user, it calls window.close to indicate that it is done, and the panel is closed by the browser.
Previous prototyping work in Mozilla Labs has included Mozilla F1, which demonstrated how this function could work. The feature described here is compatible with, and builds on, the Firefox Share work that is ongoing in Firefox.
Notifications
A Notification constructor is made available to the Worker. The Worker can specify an icon, a type, a title, and a message for each Notification, as well as event handlers that will be used to let the Worker know when the user interacts with it. The browser will use these Notification objects to get the user's attention, through a system provided by the browser that is completely under the user's control. They could be queued for delivery when the user asks, or on the New Tab page, or they could be delivered to an operating-system level system that displays an unobtrusive floating element. The browser can enforce rate, source, and type limiting. (for more, see bug 573588)
Contact Access API
The Worker may optionally provide a "Contacts" API, which responds to events delivered from the browser to the Worker. This allows the browser to retrieve a list of contact data, including names, photos, service endpoints, and more, which can be made available through the WebContacts API to user-approved content. By using this API to access multiple social services, trusted web applications can create "person-centric" advanced contact management systems and social applications. (For more, see bug 674720).
Interactions with Multiple Service Providers
Many users interact with multiple service providers in the course of their day - either because they use multiple Internet-scale social platform providers, or in the course of work, school, or an interest-based community.
The Social API allows the user to keep multiple providers "active" at one point, which means that notifications, chat/audio/video conversations, recommendations and shares, and contact access are active to all of them simultaneously. Multiple service windows can coexist, and each browser window could display different sidebars. In addition, the "visible" sidebar of a particular window might be switched using a simple UI element (toolbar button, or side-swipe gesture).
It is the responsibility of the browser to aggregate notifications from multiple services, and to render them according to the user's priorities.
Interactions with Content
So far, the Social API that has been described above is isolated from the user's browsing activity. The user can trigger messages to the social provider by clicking a recommendation/share button, and can set their browsing focus to a URL provided by a social provider when he or she clicks on a notification or an item in the sidebar, but no other information flows are described.
With the user's understanding and consent, the browser can take a more active role in the interactions between content and social systems. It could, for example, notice the presence of markup in the page requesting a "social browsing" experience. If the user has activated a social browsing feature, the browser could send a signal to one or more of the user's active platform providers, with a "the user is currently looking at..." message, at a user-specified level of granularity (whole-site, or individual URL). The social provider could use this to provide specific content recommendations in a sidebar, or to suggest immediate social interactions.
This could, for example, be used with a HTML5 game application. When the user loads the site, the browser could send a signal to the social provider, which immediately could display a list of which of the user's friends are currently playing - or would be interested in playing a turn-based game, if they're not online right now.
Site developers could also benefit from awareness that the user's browser has already provided social browsing behavior, allowing them to stop using valuable page space for behavior that is duplicated by the browser. A set of JavaScript APIs, for example, could indicate to a site that the user is already browsing socially and has share functionality built into the browser, so the site should focus on its displaying content. Getting this interaction right will require careful attention to the privacy and security characteristics of these APIs, as well as the interactions and visual design of the sidebar.
Get involved!
irc / newsgroup
If you would like to contribute to this ongoing investigation, join us on IRC (irc.mozilla.org #socialdev) or our public newsgroup!
github source
blog posts
- 2012-03-15 Experimenting with Social Features in Firefox
- 2012-07-06 Bringing Social to Firefox
- ...
- hCard - how to publish your contact info on the web, useful for social discovery
- IndieWebCamp - the social API should work with independent "providers", e.g. people that have chosen to own their identity and content on the web with their own domain.
- WebActions - social actions on the web that work with/without JS/APIs.
