B2G/QA/Gaia Test Plan/Crash

From MozillaWiki
< B2G‎ | QA‎ | Gaia Test Plan
Jump to: navigation, search

General Information

Area Details
Team:
  • Developers:
  • Designer:
  • QA: Naoki Hirata, Marcia Knous

Test Plan

Design Documentation/Background Research

How to force a crash

  • Plugin Crash
    • adb shell ps|grep plugin-container
    • app_0 409 109 55512 16760 ffffffff 40064330 S /system/b2g/plugin-container
    • adb shell kill -11 409
  • b2g OS crash
    • adb shell ps |grep b2g
    • root 109 1 191120 66024 ffffffff 400fa330 S /system/b2g/b2g
    • adb shell kill -11 109
  • browser tab crash

How to check crash:

  • checking crash:
  • root@android:/data/b2g/mozilla/Crash Reports/
  • /data/b2g/mozilla/Crash Reports/pending folder.

Kairo's reports:

Socorro

System Testing

Mobile OS

  • B2G - Firefox OS 18

Mobile Device

Test devices :

  • Leo
  • Inari

Backup test devices:

  • Otoro
  • Unagi

Strategy

  • Exploratory Manual testing using outline; see Test Outline section
  • Regression testing based on Test Cases
  • Eventual automation testing

Signoff Criteria

  • All tef+ blockers are closed
  • All basecamp blockers are closed
  • All smoke tests and basic functional tests are ran without finding any basecamp blockers

Test Cases and Results

  • Test cases will be tracked in MozTrap under "crash reporting"

Bugs

  • bug 761905 - (b2g-crash-reporting) Tracking: Crash reporting for b2g (gonk)
    • Implementation Meta Tracker
  • bug 833574 - (b2g-frontend-crash-reporting) [Meta] Other Crash Reporting Bugs
    • Bugs after implementation Meta Tracker

Test Outline

Note : This test out line is a brain storm of all possible conditions to consider for testing. The scope of the testing will be narrowed from this list based on design information, risks of breakage, etc.

Positive Smoke/Alpha Tests

UI

  • General
    • FTE Crash settings
      • Send
      • Don't send
    • OS Crash
      • Confirm System Crash Message
      • Confirm "Always send" is checked by default
      • Confirm "Don't Send" is inactive when "Always Send" is checked
      • Confirm What's in a Crash Report link
      • Subsequent OS crashes
    • App Crash
      • Confirm System Crash Message
      • Confirm "Always send" is checked by default
      • Confirm "Don't Send" is inactive when "Always Send" is checked
      • Confirm What's in a Crash Report link
      • Subsequent App crashes
    • Settings - send versus do not send
      • Pref on "Never send a report" in the Settings Preferences
      • Pref on "Always send a report" in the Settings Preferences
      • Pref on "Ask me when a crash occurs" in the Settings Preferences
    • Content Crash
      • Browser content crash
  • about:Crashes

Functional

  • UI Based Functionality
    • FTE Crash settings
      • Send
      • Don't send
    • OS Crash
      • Confirm System Crash Message
      • Confirm "Always send" is checked by default
      • Confirm "Don't Send" is inactive when "Always Send" is checked
      • Confirm What's in a Crash Report link
      • Subsequent OS crashes
    • App Crash
      • Confirm System Crash Message
      • Confirm "Always send" is checked by default
      • Confirm "Don't Send" is inactive when "Always Send" is checked
      • Confirm What's in a Crash Report link
      • Subsequent App crashes
    • Settings - send versus do not send
      • Pref on "Never send a report" in the Settings Preferences
      • Pref on "Always send a report" in the Settings Preferences
      • Pref on "Ask me when a crash occurs" in the Settings Preferences
    • Content Crash
      • Browser content crash

Beta Conditional

  • Virtual Conditions
  • Content Conditions
    • check Pending/Sent in /data/b2g/mozilla/Crash Reports/
  • Physical Conditions
    • While Charged
    • While not Charged
    • Power Consumption
    • Performance
      • Speed of functions
      • Processor utilization
      • Memory utilization
        • Memory leaks?
        • Memory efficiency
    • Storage
      • Disk
        • near full SD card and crash
        • full SD card and crash
  • Boundary testing
    • How many submissions can be sent at a time?
    • Sending of pending submissions
  • Negative (Tests to Break) Tests
  • Network Conditions
    • WiFi
      • Send only through wifi
      • Secured Wifi
        • WPA
        • Portal
      • Open Wifi
    • Mobile Data
      • 3G
      • 4G?
    • No connections
  • Race Conditions
    • Battery
      • Battery Low
      • No Battery
    • Calls
      • Incoming Phone Call
  • Device/Platform Specific Issues
    • Portrait/Landscape Orientation
    • Speaker(sound and crash)
    • Phone call and crash
  • Localization
    • Language/Dialog check
    • Date Format (Euro/Japanese/Chinese)
    • Time Format (Chinese/Japanese)
    • Change of IME on Mobile OS
  • Help?

Usability

  • Usability
  • Accessibility (508 compliance)