infostretch - mobile testing: practical tips, tricks and techniques

Practical Tips, Tricks, and Techniques of Mobile Testing

Damanpreet SinghQuality Assurance Technical Lead

InfoStretch Corporation

Agenda

• Introduction• Range of Mobile Devices• Types of Mobile Apps• Importance of Mobile App Testing• Choice of Mobile Devices• Q A

Mobile Phones Landscape

World’s average birth rate is 371,000 babies per day

Average 377,900 iPhone sold per day in Jan 2012

Average 700,000 Android Phones activated per day in Jan 2012

App Count

60k 70k

400k 600k

App Store Facts

Apple AppStore: 25 Billion downloadsAndroid MarketPlace: > 11 Billion

downloadsRecord 1 Billion apps downloaded in last

week of December 2011 on iOS and Android

Blackberry App World: > 2 Billion downloads

Range of Mobile Devices

Andr

oid • Total: 224

• Phones: 192• Tablets: 32 Ap

ple • Total: 10

• Phones: 5• Tablets: 3• Apple TVs: 2

OS Landscape - Android

Courtesy: wikimedia.org

Version Supported Portable iOS Devices

3.1.3 iPhone (original); iPod Touch (1st generation)

4.2.1 iPhone 3G; iPod Touch (2nd generation)

5.1 iPhone 3GS, iPhone 4 GSM/CDMA, iPhone 4S; iPod Touch (3rd and 4th generation); iPad, iPad 2, iPad (3rd generation)

Version Supported Apple TV

5.1 Apple TV (2nd & 3rd generation)

OS Landscape - iOS

Courtesy: wikipedia.org

Inception Design Development SubmissionTesting &Validation

App Store

App to Market Lifecycle

App Types

•Auto Loan Calculators•Credit Card / Debt Calculators

Utilities Apps

•PopEater•Gossip Grub

Entertainment Apps

•Angry Bird•Totemo

Games Apps

•NPR News•AP Mobile

News Apps

•Evernote•YPMobile

Productivity Apps

•WordRoll•Periodic Table

Search Tool Apps

•Facebook•Twitter

Social Networking

•F1 LIVE RACING•NBA Game Time Lite

Sports

•GPS Status•Wikitude

Travel

•Ancient Greece •Obit

Weather

• Diet Calendar• iPump & Fitness Builder

Fitness

• WhatsApp• Call & SMS Filter

Call and SMS

• Kindle Reader• Crimson City

eBook

• BigOven• In the Kitchen

Cooking Recipe

Test Categories

• Security

• Functional

• Usability

• Network Compliance

• User Interface

• Stability

• Profanity

• Copyright & Trademark

• Performance

Common App Failures

• Security OPT-IN/OPT-OUT

• Functional INCORRECT RULES (e.g. CHESS)

• Usability BAD USER EXPERIENCE

• Network Compliance CONT. N/W CONNECTIVITY

• User Interface AMATEUR GRAPHICS

• Stability APP CRASHES

• Profanity OBJECTIONABLE CONTENT

• Copyright & Trademark COPYRIGHTED SOUND TRACK

• Performance >75% CPU

Common OS Failures

• App won’t install• App crashes• Resolution out of proportion• Poor response to Touch, Tap, Pinch, etc.

Tool Set

• Security WIRESHARK, LOOKOUT, ETC.

• Functional FUNCTIONAL REQUIREMENTS

• Usability RANK BASED (1-10)

• Network Compliance HTTP SCOOP, PASSMARK PERFORMANCE

• User Interface RANK BASED (1-10)

• Stability MONKEY, QUICKKEYS

• Profanity CUSTOM SCRIPT FOR SCANNING

• Copyright & Trademark SOUNDHOUND

• Performance SYSTEM PANEL, ACTIVITY MONITOR TOUCH

Device vs Test Coverage

• #Android Phones = ~190• #iOS Phones = 5• #Ave. Test Cases per device = 60• Total Test Cases = 190*5*60 = 57000

Device Coverage

• Android:

Courtesy: http://www.esphoneblog.com

Device Coverage

Courtesy: http://www.wikipedia.org

Tips

• Simple Design• Intuitive Navigation• On-Demand Network Access• Opt-In/Opt-Out Implemented• EULA

Tricks

• KeynoteDeviceAnywhere• Tools: Wireshark, Lookout, System Panel,

Activity Monitor Touch, etc.• Smoke Testing: Functional, Performance, N/W

Compliance

Example (1)

• Security

• Performance

• N/W Compliance

• Usability

Example (2)

• Usability

• Performance

• UI

• Stability

Focus Areas

• Usability, Performance, Functionality

Utilities Apps

• Usability, Stability, Copyright & Trademark

Entertainment Apps

• Usability, Performance, UI

Games Apps

• UI, Stability, Functionality

News Apps

• Usability, UI, Performance

Productivity Apps

• Functionality, Security, Stability

Search Tool Apps

• Usability, UI, Network Compliance

Social Networking

• Usability, UI, Functionality

Sports

• Usability, Security, Stability

Travel

• Usability, Functionality, Stability

Weather

• Usability, Performance, Functionality

Fitness

• Usability, UI, Security

Call and SMS

• UI, Usability, Stability

eBook

• Copyright & Trademark, UI, Usability

Cooking Recipe

Q/A