mobile @ facebook - qconsf 2013

Mobile @ Facebook

Christian Legnitto ([email protected])Mobile release engineering

mailto:[email protected]


Who is this guy?

Who is this guy?

Facebook• Manage Mobile Release Engineering

• I’m hiring

• Run the iOS and Android releases

• Ran Facebook.com web pushes for ~3 months

Who is this guy?

Facebook

Mozilla

• Manage Mobile Release Engineering

• I’m hiring



• Firefox release manager

• Helped design & implement 6 week releases

• Desktop ( 3 platforms ) & mobile ( Android )

• Still a community member

Who is this guy?

Facebook

• Jaguar → Lion updates

• OS X pre-release updates

• Safari, Xcode, Java, iApps

• Build system, scrubber, PM tools

Mozilla

Apple

• Manage Mobile Release Engineering

• I’m hiring



• Firefox release manager

• Helped design & implement 6 week releases

• Desktop ( 3 platforms ) & mobile ( Android )

• Still a community member

874,000,000 MAUs

507,000,000 DAUs

native & web

supports iOS6+

iPads & iPhones

+1M beta testers

+20K alpha

testers

native & web

supports Froyo+

15 apks

How we USED to develop

Mobile core team owned mobile

. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

. . . . . .

Messages team

Events team

Photos team

Android core team

iOS core team

Doesn’t SCALE


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

Messages team

Events team

Photos team

Android core team

iOS core team


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

20

10

30

20

20

Messages team

Events team

Photos team

Android core team

iOS core team


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

20

10

30

20

20

Messages team

Events team

Photos team

Android core team

iOS core team

(numbers made up)


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

20

10

30

20

20

Messages team

Events team

Photos team

Android core team

iOS core team


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

2010

30

20

20

Messages team

Events team

Photos team

Android core team

iOS core team


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

201030

20

20

Messages team

Events team

Photos team

Android core team

iOS core team

60


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

201030 20

20

Messages team

Events team

Photos team

Android core team

iOS core team

60


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

201030 20 20

Messages team

Events team

Photos team

Android core team

iOS core team

60

Doesn’t produce the BEST product


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

Messages team

Events team

Photos team

Android core team

iOS core team


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

Messages team

Events team

Photos team

Android core team

iOS core team

product experts


Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

Messages team

Events team

Photos team

Android core team

iOS core team

product experts platform experts

How we USED to ship

Feature-based releases


Decide on features


Decide on features

Furiously work on them


Decide on features


Test


Decide on features


Test

Ship when all features are done

All features are PRIORITY #1

. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Decide on features

. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Decide on features

1. Emoji 2. Green dot 3. Naming convos

. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Decide on features


1. Reply inline 2. Birthday reminder 3. Gifts integration

. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Decide on features



1. Automatic upload 2. Tag suggestions 3. Change cover photo

. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

. . .

Decide on features




. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

. . .

Decide on features




1. Emoji1. Reply inline1. Automatic upload

. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

. . .

Decide on features





2. Green dot2. Birthday reminder2. Tag suggestions

. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

. . .

Decide on features






3. Naming convos 3. Gifts integration3. Change cover photo

The market and company CHANGES RAPIDLY

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

. . .

Decide on features



7. Naming convos 8. Gifts integration9. Change cover photo

Don’t know when code SHIPS

Can’t judge risk vs rewardFeature-based releases


80% done?


Good enough80% done?

Confuses DEV and RELEASE

Development processCreate the best product possible

→ Focus on improving

Development processCreate the best product possible

→ Focus on improving

Release processShip finished work in a timely manner

→ Focus on not regressing

Took lessons learned from DESKTOP WEB and applied

them to MOBILE

How we develop NOW

Old and busted

. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Android

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS

. . . . . .

Messages team

Events team

Photos team

Android core team

iOS core team

New hotness

. . .

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Desktop Web Android iOS

. . . . . .

Messages team

Events team

Photos team

Android core team

iOS core team

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

Photo picker

Photo albums

Chat

Group messages

Upcoming events

Birthdays

iOS and AndroidEngineers committing mobile code

How we ship NOW

featuresqualityschedule

pick 2featuresqualityschedule

Rule #1Date-based releases

We ship on time


Make usersno worse off


There’s alwaysthe next one


Retreat to safety

4 weeks of development

master

3.5 weeks of stabilization


native mobile

release branch

Automated cut every 4 weeks

RC

1 week of development

trunk

Stabilize

1 week of development

desktop web

Weekly Twice Daily

release branch

Automated cut every 1 week

Release branch


master

3.5 weeks of stabilization


native mobile

release branch

Automated cut every 4 weeks

RC

master

native mobile

release branch

Dogfooding

How we write CODE

GitSource control

>300 engineers

>150,000 files

>100,000 commits

PhabricatorCode review

BuildbotContinuous integration

Facebooki386Builds Facebook

armMessenger

armMessenger

i386 ...

Lint / static analysis regex clang android findbugs ...

Tests OCUnit JUnitsnapshot robolectric ...

https://github.com/facebook/ios-snapshot-test-case



BuildbotContinuous integration

Before diff(optional)

Create diff(mandatory)

Diff updated(mandatory)

Landing queue(optional)

Git commit(mandatory)

Mac minisContinuous integration

Android

Buck - build toolBuilding & running tests

• Fast

• Scalable and extensible

• “buck clean is a bug”

• Android and generic Java support

• Distributed Cassandra build cache

• buckd for background builds

Buck - build toolBuilding & running tests

“Gerrit Code Review recently switched its build process from

Maven to Buck, reducing build time by 67-98% while improving build reproducibility, maintainability and developer happiness.

The Gerrit project migrated itself out of the Maven tarpit into a

better build.” - Shawn Pearce (Google)

https://www.eclipsecon.org/europe2013/code.google.com/p/gerrit/

https://www.eclipsecon.org/europe2013/code.google.com/p/gerrit/

Buck - join us!Building & running tests

Start using Buck:

http://facebook.github.io/buck/

Get help in the discussion group:

https://groups.google.com/forum/#!forum/buck-build

File bugs and submit patches on GitHub:

https://github.com/facebook/buck







xctool - build toolBuilding & running tests

Makes iOS & Mac continuous integration easier.

• CLI tool, runs tests just as Xcode.app would

• Can parallelize test runs

• Can emit JSON objects for every build/test event

• Reporters for Phabricator, JUnit XML

• Attractive, concise output

xctool - “pretty” reporterBuilding & running tests

xctool - “json-stream” reporterBuilding & running tests

xctool - how it worksBuilding & running tests

xcodebuild-shim.dylibxcodebuild

xctool

test-shim.dylibotest

test-shim.dylibxctest

(SenTestingKit)

(XCTest)

Injected

Injected

Injected

xctool - join us!Building & running tests

On GitHub:github.com/facebook/xctool

On Homebrew:brew install xctool

Also available on Travis-CI



atosCrash reports

atoslCrash reports

atoslCrash reports

github.com/facebook/atosl

https://github.com/facebook/atosl

https://github.com/facebook/atosl

Changed how we DEVELOPChanged how we SHIPChanged how we WRITE CODE

Kept our CULTURE

Learn from our experience and use our toolsFacebook is mobile

Links to mentioned awesomeness

Contact me [email protected], https://fb.me/legnitto, @LegNeato

Buck http://github.com/facebook/buck

XCTool http://github.com/facebook/xctool

atosl http://github.com/facebook/atosl

Snapshot Test http://github.com/facebook/ios-snapshot-test-case



https://fb.me/legnitto

https://fb.me/legnitto

http://github.com/facebook/buck






http://github.com/facebook/ios-snapshot-test-case

http://github.com/facebook/ios-snapshot-test-case

Mobile @ Facebook

Christian Legnitto ([email protected])Mobile release engineering



mobile @ facebook - qconsf 2013

Technology