belfast selenium meetup
TRANSCRIPT
Mobile Grid SetupBelfast Selenium Meetup
October 15, 2015
Mobile Cloud Test Services * personal favorites
• Sauce Labs *
• Test Object *
• Amazon Test Cloud (was Appthwack *)
• Google Test Lab (was Appurify)
• Perfecto Mobile *
• Testdroid
• Xamarin
• TouchTest
• Keynote (was Device Anywhere)
• And many more…
The Cloud• The Pros:
• It’s the present and the future of automated testing.
• Ease of setup and integration.
• Save on costs of buying mobile devices.
• Some services provide you with good analytics and reporting.
• Some services integrate well with cloud CI services. (e.g. TravisCI)
• You expose your app to a much larger range of devices and OS’s.
• The Cons:
• It can be difficult to narrow down issues.
• Is it my network?
• Is it their network or somewhere in-between?
• Is there a problem with my test script?
• Is there a problem with my test framework?
• Is there a bug with my language bindings?
• Is my app at fault?
• The costs can be significant if running full-time CI.
• Tests could take an incredible amount of time to complete.
• Latency.
• This could also depend on the tier of service you have.
• Tests could occasionally hang or timeout without any helpful information as to why.
The Mobile Grid• The Pros:
• It’s not that difficult once you know how.
• The awesome Appium developers baked a grid node into the Appium server.
• appium —nodeconfig android.json -p 4000 -bp 2250 —udid emulator-5554 (or device udid)
• In the long run you will save money.
• You have all you're critical test data local for debugging.
• You’re not handcuffed to a cloud test services 24/7 availability.
• Outages.
• Bugs in their software.
• Acquired by another company.
• You have a bit more flexibility capturing data to include in test reports.
• e.g. You have full access to every connected device.
• The Cons:
• It can be difficult to know where to begin.
• Tough finding documentation or examples.
• Takes a bit of time to setup and smoothing out.
• Require some degree of dev-ops knowledge.
• Random device issue. (I’ll cover this later.)
What I’ll demo…• Getting connected devices programmatically.
• Launching Appium nodes programmatically.
• Running tests distributed.
• Running tests in parallel.
• Feeding test data into Allure.
• Generating the Allure report.
• Lets briefly look at the code…
Appium & Hub Server Methods
Rake File
Spec Helper
Spec Example
• Run the examples:
• Distributed: rake android[dist]
• https://www.dropbox.com/s/8md7ihp2i6pc5pw/distributed.mov?dl=0
• Parallel: rake android[parallel]
• https://www.dropbox.com/s/o7f6gb7chq51xya/parallel.mov?dl=0
• Generate the Allure report.
• allure generate report output/allure/*
• allure report open
• https://www.dropbox.com/s/a8g0erkabpa5zny/signup_failure.mov?dl=0
• Wunderlist Mobile Grid
• https://www.dropbox.com/s/y5xsmnxfldguzr0/grid-trimmed.mov?dl=0
The Demo
Wunderlist Mobile Grid
That is great & all but reporting is key!
Challenges• Random adb disconnects.
• Solved this by programmatically rebooting USB hubs before every test run.
• Random WiFi disconnects.
• Possible Solutions:
• Reverse USB tethering.
• WiFi connection apps.
• Using multiple routers on different channels.
• Spreading the devices apart.
• Alternatives to real devices:
• Using emulators with VNC clients. Facebook does this.
• Using Genymotion emulators. It’s the closest software to a real device.
• Using Docker containers with android emulators as nodes to connect to a hub.
• Some devices behave erratically.
• Finding relavent documentation.
• Finding solid examples.
Thank You!
• email: [email protected]
• twitter: @isonic1
• https://github.com/isonic1/selenium-conf-2015