Platform/Features/WebRTC
Status
| WebRTC (formerly Video (and Audio) Conferencing) | |
| Stage | Planning |
| Status | In progress |
| Release target | Phase 1 - TBD (first release planned this year, 2012) |
| Health | OK |
| Status note | We have moved forward with development, even though the spec is still being hashed out. Our initial release may be prefixed. |
Team
| Product manager | Maire Reavy |
| Directly Responsible Individual | Maire Reavy |
| Lead engineer | Randell Jesup |
| Security lead | Christoph Diehl |
| Privacy lead | Sid Stamm |
| Localization lead | ` |
| Accessibility lead | TBD |
| QA lead | Jason Smith |
| UX lead | Jennifer Morrow |
| Product marketing lead | TBD |
| Operations lead | ` |
| Additional members | Cisco Team (Enda Mannion, Suhas Nandakumar, Ethan Hunt), Eric Rescorla, Rob O'Callahan (MediaStreams piece), Ralph Giles, Tim Terriberry, Adam Roach, Jan-Ivar Bruaroey |
Open issues/risks
- VP8 is the only open, modern video codec under consideration. There is heavy debate going on in the IETF working group about what video codec should be mandated by the spec. Some are arguing that H.264 should be selected because they prefer the patent licensing situation there (MPEG-LA) and more mature technology. Without a common video codec, the spec could fail to gain acceptance and adoption.
- Overall system latency (delay) might be an issue; if it is architectural changes may be needed. Testing will need to be done.
- Privacy UI/UX for this feature is a challenge since this involves camera and microphone data. Security will be difficult due to the wish for applications to be able to provide useful interfaces, in conflict with the need to guarantee users are safe with regards to access to their camera and microphones. This design is being discussed with the security team.
- Good congestion control is essential for real-time video/audio communication. The algorithm for this is still being developed. We can’t make our first release until we have a good algorithm implemented in the Firefox code.
Stage 1: Definition
1. Feature overview
This feature will allow people using the web platform to include video/audio conferencing and associated data as part of their websites and applications. The main attributes include:
- Using the web as the setup end point for a "call." This is different than other calling systems, which use SIP, XMPP or other signaling systems.
- Allowing sites access to video camera feed for other purposes than a call, such as graphically manipulating the video feed or allowing users to record video
- Once capabilities are exchanged, a point-to-point connection is established between browsers when possible. This includes NAT traversal.
- Video is displayed as a primary object in the browser and can be part of content.
- Data can be exchanged along with the video and audio.
2. Users & use cases
Phase 1
We will support applications that do the following:
- Call a friend using a social API like Facebook. From the chat/online window the user can click on a person and start a video call with them in the browser. The video/audio/data should be securely transmitted.
- Video conferencing / sharing slides from inside an app. The user calls someone from inside an app and is able to share files, images, slides, etc through the P2P connection as he/she is talking via the data channel.
- Games that integrate video, audio and data sharing. For example a chess game where you see the video and hear the audio of your opponent and play the game live using WebRTC’s data channels.
Note: Basic Interoperation between Firefox's WebRTC code and Chrome's WebRTC code will be supported as part of phase 1. This should include video, audio, and data channels interop if Chrome supports data channels in time.
The following is a nice-to-have use case that we may or may not support in phase 1. If we don't support it in phase 1, we will support it in phase 2:
- Remote assistance/co-pilot - you "look over the shoulder" of someone as they browse (optionally controlling the mouse/keyboard), while also talking with them over audio or audio+video.
Phase 2
- Support for Android Mobile devices.
- Probably support for B2G Mobile devices (TBD)
For reference only:
3. Dependencies
- Integration of the newly released WebRTC code.
- Work product from IETF and W3 working groups
- Design of getUserMedia from Media Task Force
- Integration of Rob O'Callahan's Media API work.
4. Requirements
Must Have:
- Security design review
- Security implementation review
- The ability to make a call from a web page using a machine's built-in mic and camera.
- Should be able to work with most (80-90%) of NAT setups across all geographies. Note that certain configurations will require a relay.
- Support for royalty-free voice-friendly audio codecs. Opus strongly preferred.
- Support for royalty-free video codecs
- Audio/Video/Data must be secure (note that no security is absolute)
- A Web API that's been sent to the public working groups for feedback.
- A Web API and transport for data that accompanies the audio and video.
- Congestion control (Automatic bandwidth adjustment) required to be in and debugged prior to Firefox's WebRTC code's fire release
- Ability to add/remove video from an active call and prior to a call being placed or received
Nice to Have:
- Video resolution switching
- If hardware support exists for encoding & decoding VP8, support for video on Mobile.
Non-goals
We are not trying to support anything other than the items mentioned in this document. This page is primarily focused on what's required to ship the first phase of the feature.
Stage 2: Design
5. Functional specification
- Audio codecs planned: G.711 & Opus. (Although royalty/license-free, we have no plans to support iLBC, iSAC and G.722)
- Video codecs planned: VP8
API Docs:
6. User experience design
Latest spec can be viewed here: people.mozilla.com/~jboriss/specs/webrtc_spec.png
Stage 3: Planning
7. Implementation plan
We created our first public demo at IETF 83 (Paris). We were able to demo video/audio calls and working data channels. We need to add signaling (JSEP) support and PeerConnection support. We’re creating a project planning map of work to be done and we have a WebRTC tracking bug, bug 665909
8. Reviews
Security review
It's important to realize that this is the first step in a plan to bring video to the platform. From both a security and privacy standpoint this means we'll need to be reviewing it with an eye towards an eye towards future use cases as well as what's scoped in this one page.
Privacy review
`
Localization review
`
Accessibility
`
Quality Assurance review
`
Operations review
`
Stage 4: Development
9. Implementation
`
Stage 5: Release
10. Landing criteria
`
Feature details
| Priority | P2 |
| Rank | 999 |
| Theme / Goal | Connect |
| Roadmap | Gecko |
| Secondary roadmap | Platform |
| Feature list | Platform |
| Project | ` |
| Engineering team | Media |
Team status notes
| status | notes | |
| Products | ` | This is not part of Kilimanjaro, but there is a strong desire to get the first phase released before the end of this year. |
| Engineering | ` | ` |
| Security | Reviews to start soon | ` |
| Privacy | ` | ` |
| Localization | ` | ` |
| Accessibility | ` | ` |
| Quality assurance | ` | ` |
| User experience | Organizing meetings to discuss | ` |
| Product marketing | ` | ` |
| Operations | ` | ` |
