Shake up the Culturewith Automation!

Jul/06/2016Hiroyuki Ito / 伊藤 宏幸

2

Hiroyuki Ito (The Hiro)@hageyahhooYahoo Japan Corporation Automation Coach Agile Coach

Who I Am

3

Speaker of Agile2014

4

Japanese #1 Internet Provider

Web Services -> Smartphone Apps

Aiming to become a Tech Giant

About Yahoo Japan Corporation

5

What isDevOps?

6

CALMS / CLAMS

General Definitions of DevOps

Culture

Automation

Lean

Measurement

Sharing

http://itrevolution.com/devops-culture-part-1/

7

"DevOps" means:

The Hiro Says

Total Optimization

Whole Process Improvement

Based on Technical Foundation

8

Successful DevOps requiresthe cultural changein your organization.

9

VersionControl

CommitStage

AcceptanceTest Stage

ReleaseStage

ManualTest Stage

PerformanceTesting Stage

ArtifactRepository

Humble, Jez, and Farley David, Continuous Delivery, Addison-Wesley, 2010

Check-in

The Hiro’s Scope

10

VersionControl

CommitStage

AcceptanceTest Stage

ReleaseStage

ManualTest Stage

PerformanceTesting Stage

ArtifactRepository

Humble, Jez, and Farley David, Continuous Delivery, Addison-Wesley, 2010

Check-in

The Hiro’s Scope

Cultural Change withTest Automation & CI

11

4 Teams with 3 Products

Our Target

We coaches supported them

Lots of Immature Engineers

Few Automated Tests

Lots of Integration Troubles

12

Agenda

1. Test Automation

3. Evaluation of Measures

2. Continuous Integration

4. Conclusion

13

1. Test Automation

3. Evaluation of Measures

2. Continuous Integration

4. Conclusion

14

Make product teamswrite test scripts voluntarily

Self-running Team

Cross-functional Team

Higher Quality

Higher Productivity

Our Goal

15

Background

16

Prerequisite

PHP

Cloud-based Infrastructure

Brownfield

17

Legacy Code: No test scripts!

Brownfield: Our Common Challenges

Test Automationas a Technical Foundation

Improve design & program

Refactoring

Prevent regressions

18

Lots of Immature Engineers

Engineers

Test Scripts

???

How to WriteTest Scripts

The Importance ofTest Automation

How to TestPrograms & Products

What should I do?

19

Passive Culture

Front-endEngineers

DatabaseEngineers

APIEngineers

Waiting forInstructions

Silos Based onComponent TeamsLeader

20

Concrete Actions

21

Scope of Test Automation

Unit Testing

Acceptance Testing

Performance Testing

Security Testing

Other ility Testing

22

Stages of Adapting to Unit Testing

1. Write simple test scripts

2. Adapt to TDD

3. Use Test Double (Mock)

4. Write characterization test if necessary

5. Refactor production code & architecture

23

Schedule & Actions

Jan Feb Mar

Preparation• Investigate products• Create sample scripts

Execution• Lectures• Workshops• Pair Programming

ExecutionRemove architecturalimpediments

24

A powerful toolto learn

Unit Testing & TDD!

Cyber Dojo: Our Weapon

http://cyber-dojo.org/

25

1. Test Automation

3. Evaluation of Measures

2. Continuous Integration

4. Conclusion

26

Make product teams set up CI processes

Our Goal

Nurture product teams

Avoid integration/deployment troubles

Achieve the target from CTO

Improve product teams

27

Background

28

e.g.) During integration & deployment process

Tons of Failures were There

Needed to avoid these troubles fundamentally

StagingENV

ProductionENV

CONFIG Filefor Staging

Troubles!

29

Voice of CTO

Let’s do CI Companywide!

Top-down Approach

Bottom-up Approach

CTO

30

Concrete Actions

31

Accustomed to CI

WorkingSamples

Team Members Coach

Worked with Samples

Configurationof CI Server

CONFIG File

Did Pair Setting(like Pair Programming)

Team Members Coach

32

Always added Automated Tests

Test! Test! Test!

Pull the real power of CI

Adapt to Test Automation more

Nurture product teams

33

Simplified CI Processes

VersionControl

APServer

CONFIGFile

DBServer

WebServer

???

VersionControl

APServer

CONFIGFile

DBServer

WebServer

34

1. Test Automation

3. Evaluation of Measures

2. Continuous Integration

4. Conclusion

35

1. Results

36

Updated on Mar/31/2016(during about 2 months)

Metrics

Item Quantity

Unit Tests Added/Updated 27

Software Bugs Detected 10

Infrastructure Bugs Detected 24

Prevented Troubles 3

It may be insufficient, but it’s REAL.

37

Self-running Actions by Product Team Members

PO added “writing unit tests” as DoD

Started to write unit tests voluntarily

Established their own CI/CD strategy

Established their own testing rules

38

2. Lessons Learned

39

Coaches should NOT do everything

Role of Coaches

For retaining learning opportunity

For nurturing voluntary actions

Giving hints is sometimes appropriate

Total

40

Should Create Unavoidable Situation

CI:Unavoidable from Test Automation

Top-down Approach:Unavoidable to achieve objectives

Bottom-up Approach:Unavoidable to scale improvements

Total

41

Based on the production code

Sample Test Scripts Are Very Useful

Easy to adapt to Test Automation

Can use them as initial test scripts

Can learn their products & problems

TeamMembers

Coaches

Test Automation

42

Need to Solve Architectural Bottlenecks

Unable to apply mocks

Side effectsby static mocks

Hard to identifyweb components(Selenium WebDriver)

Modified interfacesfor injecting mocks

Introduced Phake(Mock FW for PHP)

Defined locatorsto each component(like “id” or “name”)

Test Automation

43

/** * @test */public function exampleOfTest() {     $sut = new Foo();     $result = $sut->bar('abc');     $this->assertEquals('huga', $result);}

Pair Programming Clarifies Real SkillsTest Automation

44

/** * @test */public function exampleOfTest() {     $sut = Foo::new();     $result = …? /* Cannot write logics */

    $result = Foo::bar('abc');}

Pair Programming Clarifies Real SkillsTest Automation

45

Detected failuresup-front with Automated Tests

Prevented Troubles

Unexpected Environmental Changes

Failures of Merging Branches

Leakages of Merging Branches

CI

46

Detected Misconfigurations on Servers

Only PHP

Package Hell

Don’t need to compileInterpreter

Unused Packages Leakages of Packages

CI

47

Detected Misconfigurations on Servers

Can solve Package Hell

PHP + Automated Test + CI

CI

48

Made team memberslearn a lot based on

automated working processes

Nurtured Product Teams

Fast Feedback

Learn from Failures

Lots of Confidence to Go ahead

CI

49

Technical Foundation

Technology-Driven Development

-> Presentation at Agile2014

CI/CDTest Automation

Teams

Members

Organization

Products

50

3. Next Actions

51

Next Challenges

Skills of other languages

Acceptance Test Automation

Gather other useful metrics

52

Example of Additional Metrics

Test &Test Automation

Code CoverageCyclomatic Complexity# of Bugs Detected# of Members Who Can Write TestsUpdate Times of Test Scripts(Lots of Updates = Valuable Test)

CI/CD Cycle TimeMTBFMTTR# of Releases per Week# of MembersWho Can Operate CI/CD Servers

53

1. Test Automation

3. Evaluation of Measures

2. Continuous Integration

4. Conclusion

54

What isDevOps?

55

"DevOps" means:

The Hiro Says Again

Total Optimization

Whole Process Improvement

Based on Technical Foundation

56

We are challengingthe cultural changein our organization.

57

Test Automation with CIis sufficiently valuable

Adapting to Test Automation & CI in DevOps Context

Iterative & Incremental Improvement

Do Kaizen / 改善

58

Adapting to Automationis a long way

It’s Not Easy to Continue…

It takes 6 monthto do the same thingfor 1 teamin Facebook

Photo by Improve It - Kent Beck no Workshop Mapping XP.(2006) / CC BY-SA 2.0

59

But necessary to continuefor achieving

total optimization

60

Shake up• product teams• working processes• ourselves

for• total optimization• huge success

Always Improve, Always Advance

61

With DevOps!