odd + project control 0.9

Obstacle Driven Development +

Project Control 0.9

http://odd.enterprises/


Testing First?

08/11/2016 ©odd.enterprises 2

Answering a question before its asked is like creating a solution before a test.



ODD Motivation

Life will test every aspect of a products development stages, levels and elements.

So,

• Why do engineering differently?




Background

Ideas of Obstacle Driven Development (ODD) are based on numerous development processes including:


• Control Theory• Scientific Method• ISO V-model• Test Driven Development

• ISO specifications• Requirements analysis spiral• Agile principles• SOLID principles



Control Theory


Control theory applies feedback and feedforwards control.

• Error between reference and plant output compared

• Extensive engineering, mathematic and scientific uses



Project Control

Control theory applied to creating a solution gives a model for control of projects.

• Negative feedback from testing solution

• Feedforward control is creating tests

• Feedback control is solving tests




Traditional Engineering without Tests

Traditional engineering relies heavily on assumptions of engineers.

• Good engineers make good assumptions

• Good and/or bad engineers make bad assumptions




Traditional Engineering + Testing

Testing allows us to identify and correct errors.

• Creating tests after a solution means we design tests according to a solution

• More tests mean we can identify more errors




Assumptions and Debugging 1

If we don’t have efficient testing then debugging is inefficient.

• Assumptions can easily lead to bugs

• Without thorough testing we can’t effectively find bugs

• Automated testing improves efficiency




Assumptions and Debugging 2

Without thorough testing we may have solutions full of bugs.

• May create exponentially more bugs

• Bugs exponentially more difficult and expensive to fix

• What is “Works as Expected”?




ODD Testing


Creating tests first allows design of solution with closed loop control.

• Feedforwards testing allows creation of a negative feedback loop

• Feedback used to determine project progress and control development



ODD Proof 1

𝑂 𝑑 , 𝐸 𝑑 𝑎𝑛𝑑 𝑌 𝑑are Obstacle (input), Error and Output.

• 𝑑 is 2*n bits for status of tests and solutions

• Use bitwise addition and multiplication




ODD Proof 2

𝑂 𝑑 , 𝐸 𝑑 𝑎𝑛𝑑 𝑌 𝑑and outputs are replaced by 2 bits.

• 11 is reference for test and solution to be created

• First loop round gives 𝐸 𝑑 =11 and test created




ODD Proof 3

Complete one loop to create a test before solution.

• 𝐸 𝑑 = 01 as test has been created without solution

• We develop a solution for the test




ODD Proof 4


Complete two loops to create a test and solution.

• 𝐸 𝑑 = 00 as test has been created with solution

• Continue until 𝑛 obstacles are tested and solved



Project Control

Control theory applied to give feedforward and feedback control.

• Feedforwards control is creating a test

• Feedback control is solving a test

• Multiple feedback loops to Analysis




Obstacle Driven Development

Obstacle Driven Development allows testing of all stages.

• Fully testable stages, abstraction levels and elements

• Identify, correct and prevent failure at earliest opportunity




Why Use Obstacle Driven Development?

Traditional engineering tests solutions once they are created.

• Solution tested against often incorrect requirements

• Requirements fixed and not testable

• Errors detected very late

• Errors propagating through stages cost 10x to fix per stage




Scientific Method

Scientific method says we must create tests before solutions.

• Therefore creating solutions before tests is unscientific

• Traditional engineering creates solutions before tests, so is unscientific

• Is a scientific method to engineering required?




Scientific & Engineering Method

Comparing scientific and engineering methods show differences.

• Depends on experience and talent of engineers

• No way of testing requirements

• Customers discover their requirements differ from products




ODD Scientific Method

Scientific method is created through adapting model.

• More testing and feedback than traditional scientific method

• Allows repetition to refine theories

• Learning from failure




ODD Engineering Method

Engineering version of a scientific model.

• Stages, checkpoints and control of ODD shown

• Verification provided by creating tests

• Validation provided by solving tests




ODD Scientific & Engineering Methods


Comparing methods we see structure is identical and wording equivalent.

• Conclusion is the method is scientific

• Each method repeatable to continue refinement

• We can combine the methods for R&D



Extreme Programming

Original inspiration for ODD came from Extreme Programming (XP).

• XP is a software development methodology

• Improves software quality and responsiveness to requirements

• Unit testing of all code

• Avoids programming features until they are needed




Extreme Engineering

From Extreme Programming we built a method for engineering.

• XP ideas applied to stages of software, hardware and embedded

• XP adapted to engineering gives Extreme Engineering




Overkill? 1

Testing every stage, level and element of a development is difficult and expensive.

• Overkill for simple problems

• More efficient the longer and more complex development




Overkill? 2

There are 2 main uses for the method.

• Ideal for complex projects

• Ideal for long lasting projects

• Best for complex and long lasting projects




ODD Principles

Principles inspired by Agile manifesto.

• Over substituted so both can support development

• Stages adapted and models created help facilitate aims

• Focus on linking obstacles to solutions

• Processes and tools which encourage individuals and interactions

• Working software through comprehensive documentation

• Contract negotiation through customer collaboration

• Following a plan which responds to change




Principle 1

Processes and tools which encourage individuals and interactions.

• Identification and solving of problems encouraged

• Individuals empowered to use full ability of process and tools

• Interactions simplified with tests helping communication




Principle 2

Working software through comprehensive documentation.

• Describing products and systems helps create tests and solutions

• Working software designed according to tests

• Comprehensive documentation prevents basic errors




Principle 3

Contract negotiation through customer collaboration.

• Specification helps meet customer requirements

• Behaviours easy to describe and understand

• Creating a specification is cheap, testable and adaptable




Principle 4

Following a plan which responds to change.

• Stages and elements subject to change as required

• No stages frozen and adapted to requirements

• Failing tests may result in modifying previous stages




Test Driven Development 1

Test Driven Development is software development where tests are written before code.

• Creating tests first ensures code tested and testable

• Unit tests ran through test suites for automated testing

• Improved code through early error detection


Write Test

Write Code

Refactor




Tests are written before code.

1. Test is written and ran to observe a fail

2. Simplest code is written to pass the test

3. Code is refactored according to SOLID principles

4. Repeat until coding complete


Write Test

Write Code

Refactor




Development of ODD began with a question when using TDD:

• Where do tests come from?


Write Test

Write Code

Refactor



Behaviour Driven Development

Described as “Test Driven Development done right”.

• Behaviours implemented for testing and design

• Additional stage of thinking about tests

• Increased efficiency over TDD


Write Test

Write Code

Refactor

Think



ODD Behaviour Driven Development

Adapting sequence gives one similar to traffic lights.

1. Red light for behaviour to be coded

2. Amber light when tests are created

3. Green light when code written and tested

4. Blue light when tests are passed

5. Repeat until specification is coded




ODD Process

Process becomes generic to create a new development model.

• Applications to hardware, software and embedded

• Links obstacles with tests for verification and validation

• Refactoring included in Validate Solution




Obstacle Driven Development 1

Process adapted and repeated between four stages in sequence.

• Analysis

• Specification

• Solution

• Production


Specification

Solution

Production

Analysis



Obstacle Driven Development 2

Verification and validation link stages and provide feedback.

• Verification ensures a product is built in the right way

• Validation ensures a product is built right

• Creating and solving tests give verification and validation




ODD Process




ODD for Embedded

Obstacle Driven Development designed for software, hardware and embedded.

• Stages defined to help create physical products

• Benefits of software, hardware and embedded principles

• Suitable for safety critical applications




V-model

V-model engineering for requirements, designs, testing of problem solutions.

• V-models give framework of Obstacle Driven Development

• Development for safety critical design processes

• Described by international standards




International Organisation for Standardisation

Standards and processes provided for requirements, specifications, guidelines or characteristics.

• Ensures products, services and components are fit for purpose

• Sets standards required for state of the art and markets

• ODD designed to be compatible with international standards




ISO V-model

V-model to apply ISO standards for software development.

• Model links various levels of development

• Testing for levels from software up to vehicle tests

• Testing for feedback and ensures each level is verified




Test Driven Development V-model 1

V-models are combined with Test Driven Development.

• Help create tests and integrate systems

• System created according to designs through tests

• Tests help integrate low level code to high level




Test Driven Development V-model 2

We extend and invert V-models so development stages are separated.

• Models consist of V-models with TDD and unit testing

• Inverted V-models allow for separation of stages

• Inverted V-models link with V-models creating further models




Inverted V-models

V-models inverted to link stages and help decomposition.

• V-models help integrate a solution

• Inverted V-models to decompose a creation

• Testing is adapted for decomposition of elements




Extending a Specification 1

• Traditional Problem and Solution Space– Specification a small

interface

• Extended Specification– Specification a

separate stage

49©odd.enterprises08/11/2016



Extending a Specification 2

Extending Specifications for creating new models combined with TDD.

• V-model compatible with Safety Integrity Levels

• TDD processes for V&V of Specification

• Specification to create tests




ODD Verification and Validation 1

Provides full verification and validation through creating and solving tests.

• Verification (“is it built right”) provided by creating tests

• Validation (“built right”) provided by solving tests




ODD Verification and Validation 2

Verification and validation links stages.

• Specification

– Verification and validation

• Solution

– Testing and design

• Production

– Quality assurance and control

• Analysis

– Utilisation and elicitation




Feedforward Processes

Verification is a feedfoward process where tests are created for next stage.

• Verification

• Testing

• Quality assurance

• Utilisation




Feedback Processes

Validation is a feedback process where tests are solved.

• Validation

• Design

• Quality control

• Elicitation




M-Model Comparison

Model links four separate stages of problem domain with unit testing.

• Verification and validation appropriate to each stage

• Extends traditional problem and solution domain

• Unit testing processes links stages




X-model Form

Alternative form is X-model and we concurrently integrate stages.

• Operates same as M-model except all stages are integrated

• Structure is 4 V-models around a central axis

• For developments where time is shorter




Abstraction Levels

Element describes stage and level from material to product and analysis to production.

A Product consists of:

• Systems; which consist of

• Subsystems; which consist of

• Components; which consist of

• Materials




Abstraction Levels

Abstraction levels allow divisions to a stage to organise testing, integration and decomposition.

• Abstraction levels from material to product

• Extendable to company and customer levels

• Integration and decomposition of stages through SRP




System Divisions

We divide stages into abstraction levels and system divisions.

• Linked to equivalent obstacle in previous stage

• Linked to equivalent solution in next stage

• Each single block is an element




Linking Tests 1

Tests link behaviours with solutions through testing and design.

• Solutions designed according to tests from behaviours

• Each solution is a single element of a product

• Unit testing is applied• Test suite created and

ran when changes occur




Linking Tests 2

Testing and design of solutions from behaviours of specification.

• Solution implements 1 or more behaviours

• Tests suite ran for any changes or additions

• Created as with Test Driven Development




ODD Test Suites

Test suites maintain solutions for software and identify errors.

• Test suites contain individual and combined unit tests

• Test suites are intended to be implemented between all stages

• TDD process applied throughout development to create ODD




Linking Solution to Production 1

Solution produced with consistent quality often for large quantities.

• Solution assures and controls quality of related production

• Tests are created for quality assurance

• Tests passed give measure for quality control




Linking Solution to Production 2

Elements of solution create quality assurance tests for production.

• Basic assurance and control from failure rates

• Control ensures failure rate is acceptable

• Assurance will determine an acceptable failure rate




Linking Production to Analysis 1

Linking Production to Analysis ensures product features utilised and elicited.

• Features of product utilised by customers

• Elicitation after utilisation of feature to find requirements

• Verifies current solutions and identifies new obstacles




Linking Production to Analysis 2

Production & Analysis linked with features a customer sees in a product.

• New feature may cover requirement but create another

• Elements and level investigated independently




Requirements Analysis 1

Requirements analysis is an essential stage where we determine what a product should do.

• Requirements analysis is performed in numerous ways:– Spiral model

– Use case analysis

– Safety integrity Levels (SILs)

• Requirements analysis spiral is adapted and integrated




Requirements Analysis 2




Requirements Analysis Adaptions 1

Spiral model is superimposed onto an M-model and adapted.

• Agreed Behaviours substitutes Agreed Requirements

• Quality Assurance equivalent to Testing

• Negotiation similar to Verification

– Continues on next slide




Requirements Analysis Adaptions 2

Spiral model is superimposed ontoan M-model and adapted.

• Verification substituted for Negotiation

• Validation substitutes Evaluation

• Testing substitutes Quality Assurance




ODD Analysis

Requirements analysis combines with Specification in an inverted V-model.

• Tree diagram approach creates situations from events

• Safety Integrity Levels measure and process situations into hazards

• Checkpoints for Product, Consolidated Requirements and Documents




ODD Solution

Solution created from specification through behaviour tests

• Red light for behaviour contained in specification

• Amber light when test for behaviour is created

• Green light when solution designed to pass test




Integration of Solution

Solutions integrated from materials into components, components into subsystems etc.

• Solutions created with bottom-up process

• Each level is integrated into a higher level

• Higher level tests provide integration testing




Solution Control

Using a Specification we create closed loop control to develop solution.

• Creating tests from Specification is feedforwards control

• Solving tests allows a feedback loop with control of creating solution




Testing Spiral for Solution Stage

Solution stage tests Materials first and integrates until Product.

• Continue in circle until all tests passed for abstraction level

• Once tested we spiral outwards to next abstraction level

• Once completed we have a fully tested Solution to a Specification




ODD Production

Production created from solution through quality assurance and control.

• Red light for solution with no quality assurance and control

• Amber light when test to assure quality is created

• Green light when quality control passed




Decomposition of Production

Production decomposed from high level to assure efficiency and high quality levels.

• Production created with a top-down process

• Each level decomposed for efficiency of lower levels

• Efficiency through testing of production quality




Production Control

Using a Solution we create closed loop control to develop Production.

• Creating tests from Solution is feedforwards control

• Solving tests allows a feedback loop with control of Production




Testing Spiral for Production Stage

Production stage tests Product first and decomposes until Materials.

• Continue in circle until all tests are passed for abstraction level


• Effective distribution and control of Production from high level




ODD Analysis

Analysis performed on production features through tests.

• Red light for no utilisation or elicitation of feature

• Amber light when customer utilisation test created

• Green light when customer elicitation test solved




Integration of Analysis

Analysis integrated from events in a situation tree to give complex situations.

• Situations identified from bottom-up

• Situations analysed to identify hazards

• Hazards processed into Safety Integrity Levels (SILs)

• Requirements found from SILs




Analysis Control

With Production we create closed loop control to develop Analysis.

• Creating tests from Production is feedforwards control

• Solving tests allows a feedback loop with control of Analysis




Testing Spiral for Analysis Stage

Analysis stage elicits information on Materials first and integrates to Product.

• Continue in a circle until all tests solved for abstraction level


• Efficient elicitation of customers to all elements and levels




SIL Tree Diagram 1

SIL tree diagram creates situations from events and processes requirements.

• Severity and Controllability added to each event

• Requirements found from SIL ratings using branches

• Allows unit testing approach for situations and requirements




SIL Tree Diagram 2

Adding SILs to a Probability Tree allows requirement identification and processing.

• Branching tree with SIL ratings for events

• SILs found by multiplying along branches of SIL tree

• Each situation given SIL rating to determine requirements




SIL Tree Diagram 3

Processing SIL tree diagram is very similar to a probability tree.

• SIL ratings processed by multiplying probability, severity and controllability

• SIL rating multiplied along branches for each situation

• SIL result between 1 and 4, with 4 being best


Component 1Pass

P(P1)*S(P1)*C(P1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)

Component 1Fail

P(F1)*S(F1)*C(F1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)



SIL Tree Diagram 4

Using a SIL tree diagram gives numerous advantages.

• Branches ensure all expected situations identified

• Extendible and adaptable to new situations by adding events

• Each branch is a situation

• New situations found from adding new events


Component 1Pass

P(P1)*S(P1)*C(P1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)

Component 1Fail

P(F1)*S(F1)*C(F1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)



ODD Specification

Specification created from analysis through requirement tests.

• Red light for a requirement

• Amber light when verification test is created

• Green light when test passed and behaviour validated




Decomposition of Specification

Specification decomposed from high level behaviours to ensure high level performance.

• Specifications created with top-down process

• Level decomposed until all levels specified

• Enables integration testing of a solution




Linking Behaviours to Situations 1

Situations links to Specification by creating and solving tests.

• Situations described by a decision tree

• Diagram capable of failure mode effects

and analysis

• Linked to a behaviour which covers the situation




Linking Behaviours to Situations 2

Branches of SIL tree are situations to be covered by Specification.

• Each situation is tested to ensure coverage by specification

• Ensures situations are covered before

creation of solution• All expected situations

should have an associated behaviour




Testing Spiral for Specification Stage

Specification stage tests Product first and decomposes elements until Materials achieved.

• Continue in a circle until all tests are passed for abstraction level

• Once tested we spiral outwards to the next abstraction level

• Decomposition of Specification from ensures high level features




Obstacle Driven Evolution 1

Spiral model adaptable to model an evolutionary process.

• Dashed arrows represent failure

• Dashed arrows extending outwards for too ambitious

• Dashed arrows contracting for too complacent

• Failure in quadrant indicate invention has failed stage


Note: Electric car invented in 1835




Using the model we can plot why a product or invention failed.

• Was analysis insufficient?

• Was Specification too ambitious or complacent?

• Was Solution created too expensively or too poorly?

• Was Production too high quality or too little?


Note: Electric car invented in 1835



ODD 3D Pyramid Model


Concurrent development of multiple systems requires we layer M-models of the systems.

• Pyramid shaped for increasing number of elements at lower levels

• Observable development progress measurement



ODD Continuous Process

Here we show 2 models which have been linked into a cyclic model.

• Process repeats for continuous improvement

• Further stages may be added

• Proceed clockwise through each stage




ODD Systems Engineering 1

Further stages added as necessary to complete hardware system development linked through tests.

• Safety Integrity Levels and Situation Analysis provide information on hazards

• Supply and Assemble give hardware capabilities





We can divide SIL tress into separate stages for additional error checks.

• Situation Analysis identifies situations

• Safety Integrity Levels process situations into hazards

• Requirements analysis becomes similar to Specification





For hardware we need to ensure that physical elements are supplied and assembled correctly.

• Supply stage handles supply of physical hardware

• Assemble stage handles assembly of physical hardware

• Completed through creating and solving tests





Steps to creating and maintaining a product are converted into an ODD stage with some thought.

• Stages added should ideally come in pairs

• Pairs of stages allow integration and decomposition

• We can add, combine or reorder the stages as required





Further divisions are added to the ODE model so we can demonstrate success or failure of that sector.

• Branches may be added for other vehicle types e.g. trucks, buses

• Ability to investigate past failures and plan future success

• Extendable to other scientific disciplines




ODD OODA

Col. John Boyd’s OODA method has been combined with ODD to produce new models.

• OODA = Observe, Orient, Decide, Act

• Used for military strategy for decades

• Similarity to OODA effectively proves ODD




ODD OODA

OODA methods have been used for many purposes.

• Originally for developing military strategy

• Adapted to Artificial Intelligence, business strategy and legislation

• Adapted to be compatible with ODD




ODDSAP OODA

Further stages are added to provide steps for hardware development.

• Feedback from all stages to Analysis

• Decision included to determine progression

• Outside Information and Unfolding Circumstances are also analysed




Generic Model OBSA

Comparing ODD with OODA resulted in a generic method to describe many things.

• OBSA – Obstacle, Behaviour, Solution, Action




Other Uses

Generic model may be used to create further models.

• Shown is a model for healthcare adapted from ODDSAP OODA

• Other methods for legislation & AI

• Further methods to be identified




ODD Materials

ODD is explained in further presentations.

• Obstacle Driven

Development

• ODD: Requirements Analysis

• ODD: Extending a

Specification

• ODD: Extending TDD

• ODD: Extending V-models

• ODD Is Not Agile or Waterfall

ODD Is Not Agile or

Waterfall

Obstacle Driven Development

ODD: Requirements

Analysis

ODD: Extending a Specification

ODD: Extending V-models

ODD: Extending TDD




http://www.odd.enterprises/obstacle-driven-development.html

http://www.odd.enterprises/odd-requirements-analysis.html

http://www.odd.enterprises/odd-specification.html

http://www.odd.enterprises/odd-extending-tdd.html

http://www.odd.enterprises/odd-extending-v-models.html

http://www.odd.enterprises/odd-is-not-agile.html

Further Information and Questions

Website

Presentations

Facebook

Twitter

Email





http://www.odd.enterprises/odd-presentations.html

https://www.facebook.com/odd.enterprises

https://twitter.com/odd_enterprises

mailto:[email protected]

http://www.odd.enterprises/home.html

http://www.odd.enterprises/home.html



Legal Stuff

ReferencesTest Driven Development for Embedded C

James Grenning, 2011

Test Driven Development

http://en.wikipedia.org/wiki/Test-driven development

Behaviour Driven Development

http://en.wikipedia.org/wiki/Behavior-driven development

Unit Testing

http://en.wikipedia.org/wiki/Unit testing

DisclaimerThe ODD M-model and associated processes are provided by odd.enterprises and may be used for any purpose whatsoever.

The names odd.enterprises and associated logos should not be used in any representation, advertising, publicity or other manner whatsoever to endorse or promote any entity that adopts or uses the model and/or associated processes.

odd.enterprises does not guarantee to provide support, consulting, training or assistance of any kind with regards to the use of the model and/or processes including any updates.

You agree to indemnify odd.enterprises and its affiliates, officers, agents and employees against any claim or demand including reasonable solicitors fees, related to your use, reliance or adoption of the model and/or processes for any purpose whatsoever.

The model is provided by odd.enterprises “as is” and any express or implied warranties, included but not limited to the implied warranties of merchantability and fitness for a particular purpose are expressly disclaimed.

In no event shall odd.enterprises be liable for any damages whatsoever, including but not limited to claims associated with the loss of data or profits, which may result from any action in contract, negligence or other tortious claim that arises out of or in connection with the use or performance of the model.




http://en.wikipedia.org/wiki/Test-driven_development

http://en.wikipedia.org/wiki/Behavior-driven_development

http://en.wikipedia.org/wiki/Unit_testing

odd + project control 0.9

Technology