tips for better ci on android
TRANSCRIPT
CI for Mercari Android App
• Started April 2015
• Purpose
- Build Check each Flavour and BuildType
- Automate app tests
- Continuous deployment
Good!!
• Easy setup with yml • Build production app without
including secret keys into the repo(http://docs.travis-ci.com/user/encrypting-
files/)
• Most of Mercari project run CI on Travis CI
Good!!• Has better features for testing and debugging
• Bit faster than Travis CI (1- 2min for each build)
• Mercari iOS project runs CI on CircleCI
So-so…• Need some tweaks to avoid Out of Memory
• Building production app looked a bit tricky (Haven’t tried yet)
Check Production App Check Development App Run JUnit Tests
Deploy
Let each CI service do different tasks
PID RSS %CPU COMMAND21787 2250024 165 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xmx4g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /usr/local/android-sdk-linux/build-tools/21.1.1/lib/dx.jar com.android.dx.command.Main --dex --force-jumbo --output /home/ubuntu/mercari-android-as/app/build/intermediates/dex/us/dev /home/ubuntu/mercari-android-as/app/build/intermediates/classes-proguard/us/dev/classes.jar9477 1628300 104 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -XX:+HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-2.2.1-all/6dibv5rcnnqlfbq9klf8imrndn/gradle-2.2.1/lib/gradle-launcher-2.2.1.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 2.2.1 /home/ubuntu/.gradle/daemon 10800000 b3531fa1-fad3-4599-a284-96a81cc9336b -XX:+HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant9145 279164 4.7 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xms256m -Xmx512m -Dorg.gradle.jvmargs=-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -Dorg.gradle.appname=gradlew -classpath /home/ubuntu/mercari-android-as/gradle/wrapper/gradle-wrapper.jar org.gradle.wrapper.GradleWrapperMain assembleDev -PdisablePreDex
Tip 1: Avoid OOM(CircleCI)
PID RSS %CPU COMMAND21787 2250024 165 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xmx4g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /usr/local/android-sdk-linux/build-tools/21.1.1/lib/dx.jar com.android.dx.command.Main --dex --force-jumbo --output /home/ubuntu/mercari-android-as/app/build/intermediates/dex/us/dev /home/ubuntu/mercari-android-as/app/build/intermediates/classes-proguard/us/dev/classes.jar9477 1628300 104 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -XX:+HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-2.2.1-all/6dibv5rcnnqlfbq9klf8imrndn/gradle-2.2.1/lib/gradle-launcher-2.2.1.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 2.2.1 /home/ubuntu/.gradle/daemon 10800000 b3531fa1-fad3-4599-a284-96a81cc9336b -XX:+HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant9145 279164 4.7 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xms256m -Xmx512m -Dorg.gradle.jvmargs=-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -Dorg.gradle.appname=gradlew -classpath /home/ubuntu/mercari-android-as/gradle/wrapper/gradle-wrapper.jar org.gradle.wrapper.GradleWrapperMain assembleDev -PdisablePreDex
Tip 1: Avoid OOM(CircleCI)
JVM is using too much memory!2250024 + 1628300 + 279164 = 4157488KB ≒4GB
PID RSS %CPU COMMAND21787 2250024 165 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xmx4g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /usr/local/android-sdk-linux/build-tools/21.1.1/lib/dx.jar com.android.dx.command.Main --dex --force-jumbo --output /home/ubuntu/mercari-android-as/app/build/intermediates/dex/us/dev /home/ubuntu/mercari-android-as/app/build/intermediates/classes-proguard/us/dev/classes.jar9477 1628300 104 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -XX:+HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-2.2.1-all/6dibv5rcnnqlfbq9klf8imrndn/gradle-2.2.1/lib/gradle-launcher-2.2.1.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 2.2.1 /home/ubuntu/.gradle/daemon 10800000 b3531fa1-fad3-4599-a284-96a81cc9336b -XX:+HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant9145 279164 4.7 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xms256m -Xmx512m -Dorg.gradle.jvmargs=-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -Dorg.gradle.appname=gradlew -classpath /home/ubuntu/mercari-android-as/gradle/wrapper/gradle-wrapper.jar org.gradle.wrapper.GradleWrapperMain assembleDev -PdisablePreDex
Tip 1: Avoid OOM(CircleCI)
Progard + Gradle consumes memory a lot
Tip 1: Avoid OOM(CircleCI)
Tuning tips: • Lower the total memory(JVM,Gradle) size to 4.0G • Adjust Xmx and MaxPermSize • Set timeout (>1000) for GC
Xmx -> Heap space (where object resources are allocated ) MaxPermSize -> Permgen(where class, methods are loaded)
Tip 1: Avoid OOM(CircleCI)
machine: environment: ANDROID_HOME: /usr/local/android-sdk-linux _JAVA_OPTIONS: "-Xms256m -Xmx1280m -XX:MaxPermSize=350m" GRADLE_OPTS: '-Dorg.gradle.jvmargs="-Xmx1536m -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError"' … override: - ./gradlew assembleDev -PdisablePreDex: timeout: 1500
Heap Perm Heap Perm other
JVM(dex) Gradle
4GB
• There is no best practice but I suggest that large memory be allocated to Xmx
Tip 2: Save the JUnit test result(CircleCI)
test: post: - mkdir -p $CIRCLE_TEST_REPORTS/junit/ - find . -type f -regex ".*/test-results/*/.*xml" -exec cp {} $CIRCLE_TEST_REPORTS/junit/ \;
• You can check Test result from ‘Test Failure’
http://qiita.com/tomoima525/items/b4a91d1624aaf0112b3fRef. CircleCIでJUnitの結果を見る
Tip 3: Continuous deployment for specific branches(CircleCI)
• Run shell script for specific branchesdeployment: jp: branch: [/deploy_jp_.*/, /release_.*/] commands: - ./deploy_jp.sh $CIRCLE_BRANCH us: branch: [/deploy_us_.*/, /release_.*/] commands: - ./deploy_us.sh $CIRCLE_BRANCH
RELEASE_MESSAGE=$1
#deploycurl -F "file=@app/apk/mercari_dev_jp.apk" -F "token=${DEPLOY_GATE_API_KEY}" -F "message=${RELEASE_MESSAGE}" -F "distribution_key=${JP_DEV_DIST_KEY}" -F "release_note=${RELEASE_MESSAGE}" https://deploygate.com/api/users/xxxx/apps
deploy_jp.sh
Tip 3: Continuous deployment for specific branches(CircleCI)
• Set variables at `Environment variables`
Tip 4: Add dependencies manually
dependencies: override: - echo y | android update sdk --no-ui --all --filter extra-google-m2repository,extra-android-m2repository,build-tools-23.0.1, android-23
android: components: - build-tools-23.0.1 - android-23 - extra-android-m2repository - extra-google-m2repository
.travis.yml
circle.yml
• The latest packages(ex. Android M) are not installed on the VM
• Add manually with command line
• Tip 1: Check memory usage of JVM to avoid OOM
• Tip 2: Save the JUnit test result on the VM
• Tip 3: Continuous deployment for specific branches
• Tip 4: Add dependencies manually