PowerPoint

NCSOFT / SET team

Inven Game Conference

1

2NCSOFT (2004 ~ )SET (Software engineering in test) 2015 ~ Technical director (2007 ~ 2013) (2004 ~ 2005)(2000) PDA , PalmOS 5.x ()(1994) 3.0 , 96 (1991), (1992) , for

ZDNet

2

. . .

15 13 .

3 20 . 40 1 (2.5%)

97.5% 0% .

3

4() 15

: 13 ? : 3 , 10 : ? : 2.5% (, ?) : 0% 97.5% . . ~ : ( ?)

15 13 .

3 20 . 40 1 (2.5%)

97.5% 0% .

4

5

6

QA .

6

7

8

9

9

QC (Quality Control) QA (Quality Assuarance) QC QC (Software test engineering)QC QA

QC QA 10

https://oasis5pm.wordpress.com/2015/03/03/%EB%8B%A4%EB%A5%B8-%ED%9A%8C%EC%82%AC%EB%8A%94-%EC%96%B4%EB%96%BB%EA%B2%8C-qa-%ED%85%8C%EC%8A%A4%ED%8C%85%EC%9D%84-%ED%95%98%EA%B3%A0-%EC%9E%88%EC%9D%84%EA%B9%8C-google-facebook-atlassian/

. ?

10

/ PC ,

, OS (OS ) /

1/211

https://opensignal.com/reports/2014/android-fragmentation/

12

12

https://imseongkang.wordpress.com/2016/04/21/mobilescreensize/https://opensignal.com/reports/2014/android-fragmentation/13

https://opensignal.com/reports/2014/android-fragmentation/

( , ) (: ) , , , QA ) () ?

2/2* : 14

15 ()B MS SET SDET , PC QA 30 QA

16 UI

UI OS

vs

,

,

17

18 50 90% 300 , 150

19 , () ( ) Perfecto Mobile, TestObject, Bitbar (TestDroid),Xamarin Test Cloud Amazon Device Farm, Google cloud test lab

http://www.perfectomobile.com/https://testobject.com/http://testdroid.com/https://www.xamarin.com/test-cloud MShttps://developers.google.com/cloud-test-lab/https://aws.amazon.com/ko/device-farm/CloudMonkey

19

20 PC PC

Appium, Robotium, Calabash, UIAutomator, Espresso

21AppiumRobotiumCalabashEspressoFrankUI automatorRanorexAndroid OOOOXOOiOSOXOXOXOLicenseOpenSourceOpenSource(commercial)OpenSourceOpenSourceOpenSourceOpenSourceCommercialScriptingJAVA, Ruby, python, C# JAVARubyJAVARubyJAVAC#, python, VBIntegrationOOOOOOORecorderOOPartialXOXOModified appXXOOOXO

22 ( + )

Selenium WebDriver API (Java/Javascript/Python/Ruby/C#/) http Appium UIAutomator TCP , , , Android/iOS Appium23

(Java) Appium (JUnit/TestNG) Appium adb Appium UIAutomator API Appium UIAutomator Appium24

Windows / OS XAndroid DeviceAppium Test scriptUI AutomationiOS DriverUI automatorInstrumentationiOS device

Appium server(node.js)iOS scriptAndroid script

HTTPJSON wire protocolCommandCommandUI automator(Android 4.2+)Selendroid(Android 2.3+)Test runnerAppium client API

25

UI Automator / API 17+

26 UI path UI

Appium Server

UI AutomationLibrary

Test Script( )Game ObjectFind ModuleIn-Game Object QueryTestAppreturn objects information

GameObjectInspectorModule Appium ( ) Appium

Appium Server

UI AutomationLibrary

Test Script( )TestApp

27

28 (1/2) UI (Unity3D ) Appium Appium API findElementByGameObjectNamefindElementByGameObjectNameAndTag

29 (2/2) Scene ID ( Unity3D Inspector UI ) ID

30 (1.3x )

31

32 RunCommand(AddCash 1000"); WaitUntilDisable(networking, 10);

RunCommand(AddStamina 10"); WaitUntilDisable(networking, 10);

String staminaText = "UIRoot/HudRoot/PlayerInfoUI/HudRoot/Stamina/Text_Root/Text"; MobileElement me = driver.findElementByGameObjectName(staminaText);

String staminaNum = me.getText().replaceAll("\\[[\\-a-zA-Z0-9]*\\]", ""); String staminas[] = staminaNum.split("/"); int stamina = Integer.parseInt(staminas[0]);

String staminaCharge = "UIRoot/HudRoot/PlayerInfoUI/HudRoot/Stamina/InkBG_Root/BG"; ClickAfterWaitingClickable(staminaCharge, 5);

String buyStamina = "UIRoot/PopupRoot/ShopMiniStaminaDlg/Body/ListViewPanel/CardRoot/Collider"; ClickAfterWaitingClickable(buyStamina, 5);

String changeBtn = "UIRoot/PopupRoot/ShopBuyItemPopup/Content/Btn_Confirm/Enable_Btn"; ClickAfterWaitingClickable(changeBtn, 5); // buy button

WaitUntilDisable(networking, 10);

32

33 Appium PC . UI ID . UI ID

34 ID TIP ID ID ID UI . . .

35 ( ) command(ShowMeTheMoney 1000000)command(SkipTutorial)

36 UI Appium Appium Unity3D UnrealEngine Cocos2D NGUI

Code Commit37CI CodingBuild ReleasePre-build TestReleasePost-build Test

Static Analysis

Test cases

Emulator/Cloud test/ SW Test

Report

ProcessTestPipeline

38Q & ATwitter @jongwonkim@nvil

: .