Mobile/Fennec TestDev

What is TestDev and Fennec

Fennec TestDev is a part of the Fennec project aimed at bringing automation to Fennec specific areas (such as panning, tabs, bookmarks). Automation will be using existing automation frameworks that Firefox uses, but new test cases that are specific to Fennec.

Have a Question?

Not quite sure how to get involved? Have some great ideas? Maybe something isn't clear or you have a better suggestion? Feel free to contact me and I can point you in the right direction!

Contact Info:

• email: jmaher at mozilla dot com
• irc: jmaher

Areas to focus on

Tabs

• Tracked in: bug 485855 - zad (got the basics done!)
• Recommended harness: (browser-chrome)

The tabs UI is new and unique to Fennec. We have different limitations and display issues to work with. This project is UI automation to cover the basic tab functionality, UI handling of the tabs and browser interaction.

Bookmarks

• Tracked in: bug 486490 - happyhans (waiting on review)
• Recommended harness: (browser-chrome)

Just like the tabs, the Bookmark UI in Fennec is new and unique to Fennec. A lot of work has been done to make this mobile and small screen friendly. We have most of the functionality of the Firefox bookmarks, but a different user flow. This project entails writing UI automation to cover the bookmark usage scenarios, layout, functionality, and edge cases.

Preferences

• Tracked in: bug 489158
• Recommended harness: (browser-chrome)

In Fennec we have a limited amount of Preferences and options as compared to Firefox. These require us to navigate the UI to display the preferences pane and turn on/off the preferences. It is a simple UI compared to Firefox. This project utilizes the UI to automate the viewing, editing, and testing of the preferences.

Panning / Zoom

• Tracked in: bug 489160
• Recommended harness: (browser-chrome)

Fennec has a whole new UI to accommodate the stylus method of input rather than a mouse. With this UI, we allow panning (dragging the screen around) as well as double click to zoom in/out. There are also special pan states to view the tab pane and config pane. This creates a whole set of interesting test cases since this is the core navigation method for the mobile browser.

This project focuses heavily on UI automation and will most likely be done in browser-chrome, but could end up utilizing other tools or test methods. This is a challenging project since there is potential for a lot of test cases and harder to solve problems like verifying the pan or zoom is accurate.

Download Manager

• Tracked in: bug 489161
• Recommended harness: (browser-chrome)

Download Manager just had a complete overhaul and is ready for some automation. This has the same functionality as you would expect from a download manager on the desktop, except it is mobile friendly. Developing the automation will require downloads of files and UI automation to get to the download manager and verify the display of the data.

Awesome bar / Search providers / History

• Tracked in: bug 489163
• Recommended harness: (browser-chrome)

As all of the other Fennec projects, this involves testing the awesome bar which has similar functionality to Firefox. There are existing unittests which cover the query and return values in the awesomebar. For this project we need to test it from a UI perspective, specifically the unique Fennec features.

Tel protocol and integrated phone support

• • Verify support of the tel: protocol
  • Verify we send the correct phone number and handle error conditions

Notification bar prompts

• • Test Password prompt (remember, yes, no)
  • Test Geolocation prompt (remember, yes, no)

Hardkeys on N810

• • Verify Full screen button
  • Verify dpad navigation (up, down, left, right, center)
  • Verify ESC key

Hardkeys on HTC Touch Pro

• • Verify screen rotation
  • Others?

NSPR Unit Tests

NSPR is the core piece of porting that we need to do for each new platform. We need to ensure we have a solid fully comprehensive test suite which covers all the necessary api's that we rely on.

In porting to Symbian there is discussion around doing this: bug 432430 bug 501172

These live in the source tree here: http://mxr.mozilla.org/mozilla-central/source/nsprpub/pr/tests/

Actions todo:

• verify these run on existing platforms
• verify these are reporting accurately
• verify we have comprehensive coverage with these tests (??)

Performance

Taras has come up with a great start on a [Fennec performance story]. From a QA perspective we need to outline the tests that make sense on mobile (not just a general web browser on a desktop). This project will also require taking what Taras creates for performance testing and working to get it integrated into talos!

Here are my initial thoughts on performance scenarios we should be measuring:

• Application launch time
• Time to load a webpage - do this on various pages(difficult since we are dealing with live pages on the internet) - Tp
• Time to pan to the bottom of a webpage (including the number of hiccups and false starts to the pan) - do this on various pages - tPan
• Time to display the tab and config panes
• Time to browse back one page in history (including snapping the config pane, clicking the back button)
• Time to enter a URL with a large awesome bar history
• Time to close Fennec
• Time to open a new tab
• Time to open a new tab and load a page
• Time to Zoom in/out on a page (test with multiple pages of various content/length) - tZoom

A further portion of this project is to take the performance tests and expand those into stress tests. I can see these types of tests (measure time, memory usage, ???):

• Open a 3 new tabs and load pages in them, then close. Repeat 5000 times.
• Pan up and down a long page, zoom in every 3rd pan, out after the following pan 5000 times.
• browse back and forth through a history of 1000 pages using the navigation buttons
• exercise the URL bar for 1000 different pages

Other TestDev related todo's

• Follow up on existing Unittest bugs and get resolution (either in workarounds, or fixes)
  • Mochitest bug 473558
  • Chrome bug 473562
  • Reftest bug 473564
  • XPCShell TBD

References

• Mochitest
• Browser-chrome
• Mochitest-chrome
• Reftest

• Fennec Automation
• Building Fennec