Platform/Features/MPAPI

From MozillaWiki
Jump to: navigation, search
Please use "Edit with form" above to edit this page.

Status

Media Plugin API (MPAPI)
Stage Planning
Status In progress
Release target Firefox 15 or 16 (TBD) - Mobile only, by the end of Q2
Health OK
Status note Most of the design has been completed by Andreas. Media is finishing off the design and planning the implementation.

Team

Product manager Maire Reavy
Directly Responsible Individual Maire Reavy
Lead engineer Rob O'Callahan (formerly Andreas Gal)
Security lead TBD
Privacy lead `
Localization lead `
Accessibility lead `
QA lead TBD
UX lead `
Product marketing lead `
Operations lead `
Additional members Chris Double, Chris Pearce, Edwin Flores

Open issues/risks

TBD

Stage 1: Definition

1. Feature overview

This feature gives gecko the ability to use a device's hardware decoder to play back encoded video or audio (in particular H.264, AAC, and MP3) on that device. We are targeting Mobile for the first release.

We may use this same approach to support certain codecs (like MP3) on desktop; desktop support is still TBD.

2. Users & use cases

  • App wishes to playback MP3 file on user's mobile device
  • App wishes to playback H.264 on user's mobile device (note: the H.264 video may use AAC to encode the audio track of the video)

3. Dependencies

  • The user's device must have the codec needed for playback already installed, or else this feature will appear not to work.

4. Requirements

  • Playback performance on a device that has the necessary codecs installed must be snappy and appear seamless. (If anyone needs more detail like how quickly playback should begin or what qualifies as seamless, please contact Maire.)
  • Audio/Video synchronization must be maintained

Non-goals

  • Playback will not work if the user's device does not have the codec already installed.
  • The user will not be prompted to download a codec in order to play a video/audio clip, if the codec is not available for playback.

Stage 2: Design

5. Functional specification

Bug 714408

6. User experience design

`

Stage 3: Planning

7. Implementation plan

Phase 1 : Mobile

1) Review, polish and land existing patches for using system codecs on Android/B2G (bug 714408). Those patches create an "MPAPI" interface into which various codec libraries can be plugged. These libraries will be wrappers around libstagefright or other Android system codec interfaces --- different libraries customized for different Android versions and/or devices as required. Currently these patches only work on devices where we can get the decoded YUV data in main memory (MPAPI currently depends on this). Chris Double is currently working on this. Some contributors are also involved.

2) Following on from #1, extend MPAPI to handle cases where we don't get YUV data. The frames might be made available only as some kind of overlay. To integrate this into Gecko as best we can, we'll need to modify the layers system to work with those overlays. The exact details of this depend on exactly what the hardware and APIs can do; needs investigation. Not sure who'll work on this yet, but Chris D will be involved and I'll probably help.

3 (below) is not part of the product, but will land to help contributors:

3) Review and land latest contributed GStreamer patch (bug 422540), which is quite good. We currently have no plans to ship this in a Mozilla product, but contributors want it so it might as well be in the tree as they're using it. This patch was contributed by Alessandro Decina. Review and landing will be done by Chris Double. UPDATE: Patch landed on 4/19/2012.

Phase 2 : Desktop (still TBD if we do this)

If we decide to support MP3 on desktop via MPAPI, #4 (below) explains the plan we would follow:

4) Investigate MP3 support using system codecs on desktop platforms.

  • On Windows (XP and up) we can use DirectShow. Chris Pearce is going to look into this.
  • On Mac we can use some Mac APIs. Edwin Flores is going to look into this.
  • On Linux we could use libav (when MP3 codec is available) directly, or possibly via GStreamer (see #3). Not sure who's going to work on that. The currently plan is to use these system APIs directly in dedicated codec backends rather than wrap them in some other abstraction such as MPAPI, but this may be revisited.

8. Reviews

Security review

`

Privacy review

`

Localization review

`

Accessibility

`

Quality Assurance review

`

Operations review

`

Stage 4: Development

9. Implementation

`

Stage 5: Release

10. Landing criteria

`


Feature details

Priority P1
Rank 999
Theme / Goal Experience
Roadmap Firefox Mobile
Secondary roadmap Platform
Feature list Mobile
Project Apps
Engineering team Media

Team status notes

  status notes
Products ` `
Engineering ` `
Security sec-review-needed bug 749221
Privacy ` `
Localization ` `
Accessibility ` `
Quality assurance ` `
User experience ` `
Product marketing ` `
Operations ` `