rapid continuous software engineering - meeting the challenges of modern software development
DESCRIPTION
In recent years Software Engineering (SE) is moving towards a support for a continuous software development; from daily building and agile processes, refactoring, to automatic software deployment. However these methods have shown a number of weakness such as problems with software correctness, performance, and scalability. In addition, focus on small changes leads to limitation of significant innovation and improvement [ref]. On the other hand the well developed and proven SE methods mainly support development from scratch, and provide support for modeling, analysis, verification and validation of the entire system. Such methods ensures correctness, but produce a huge overhead in efforts and resources required for the support, and are becoming obsolete for continuously and rapidly changing systems. This brings new challenges in developing of new SE methods and models for continuous software change, both supporting rapid changes in software systems while guaranteeing system correctness and qualities, and ensuring sustainability in long-term software system evolution. This talk will point to these challenges and propose some possible directions to address them.TRANSCRIPT
Rapid Con*nuous So.ware Engineering
-‐ Mee*ng the challenges of modern so.ware development
Ivica Crnkovic Mälardalen University à Chalmers & Gothenburg University, Sweden
[email protected] hHp://ivica-‐crnkovic.net
RCoSE, June 2, 2014, Hyderabad
Con*nuous SE – some history • 1990…2000 – get it big* • 2000…2005 – get it manageable • 2005 -‐ 2010 -‐ get it simple • 2010 – get it wide • Now: get it sustainable (survivable)
* -‐ func*onality, business, in use
2014-‐06-‐02 2
1990…2000 – big shots – so.ware releases So.ware Major.Minor/Revision – (SW 4.5/11)
User func*onality
*me
1.0 1.1
1.2
2.0
2014-‐06-‐02 3
Ge_ng complex…. maintenance, variants…
User func*onality
*me
1.0
1.1
1.2
2.0
2014-‐06-‐02 6
1.0/1
1.2 1.2/2 1.2/1
1.1/1 1.1/2 1.1/3
2.0-‐variant A
Ge_ng complex…. New roles
User func*onality
*me
1.0
1.1
1.2
2.0
2014-‐06-‐02 7
1.0/1
1.2 1.2/1 1.2/2
1.1/1 1.1/2 1.1/3
Developers
maintainers
Get it manageable..
2014-‐06-‐02 8
!"#$%&'()*+(,-./0%
123%
121%
124%
423%
12351%
124%12454%12451%
12151% 12154% 12156%
42378,$.,(/%9%
Developers Maintainers Architects SCM Manager Quality team
CMM(I) Process Improvements Quality Assurance
So.ware Product Lines
Component-‐based SE Aspects SCM tools MDD
Get is simple!
2014-‐06-‐02 9
User func*onality
Focus on: Code Changes Short *me planning Customers Team-‐working Refactoring Pragma*c (ad hoc) reuse
Things are ge_ng wide
2014-‐06-‐02 10
But: Distributed development Outsourcing, offshoring crowdsourcing, Crowd-‐tes*ng, Eco-‐systems, Services & Cloud Compu*ng COMPETITION HARDER THAN EVER
Challenge
2014-‐06-‐02 11
How to enable rapid changes yet insure improvements on long and short terms? How to be compeGGve and sustainable? How to be rapid and conGnuous?
How to achieve this?
2014-‐06-‐02
a) Let the system improve/adjust itself b) Make it easy for the users to adapt the system c) Let others do the improvements (for you) d) React rapidly on requirements & users’
behavior
e) Ensure correctness and quality f) Ensure innovaGon g) Ensure (increased) outcome
13
SE challenges* • Dealing with con.nuously and rapidly changing systems at run-‐.me: – How to achieve that the system itself maintains the architectural integrity and ensures its quality?
• Dealing with con.nuously and rapidly changing systems at development-‐.me: – How to facilitate developers to effec*vely predict consequences of poten*ally applied changes?
• Suppor.ng and synchronizing development-‐ and run-‐.me: – How to enable maintaining alignment of all lifecycle ar*facts in a con*nuously evolving system?
2014-‐06-‐02 14 * Jan Bosch, Michel Chaudron, Ivica Crnkovic, Patrizio Pelliccione, MaHhias Tichy: Controlled Con*nuous So.ware Engineering, Research proposal SW Research Council
Research objec*ves 1. So=ware architecture for con.nuous experimenta.on
– experimental evalua*on of different SA solu*ons in parallel to the exis*ng “baselined” SA.
2. Managing changes of non-‐func.onal proper.es (NFP) – Rapid analysis of (possible) changes of NFP caused by changes in SA
3. Methods for synchronizing architecture/designs and source code – co-‐evolu*on of both source code and SA.
4. Suppor.ng controlled and reliable adapta.ons – support unplanned adapta*ons in a controlled and reliable way, i.e.,
by ensuring the preserva*on of system invariants.
2014-‐06-‐02 15
Example 1: dependable systems • Dependability Ability of a
system to deliver service that can jus*fiably be trusted (Ability of a system to avoid failures that are more frequent or more severe than is acceptable to user(s)
• Robustness is the persistence of a system’s characteris*c behavior under perturba*ons or condi*ons of uncertainty.
2014-‐06-‐03 DFG 2014 16
Dependability
Attributes
Threats
Means
Availability Reliability Safety Confidentiality Integrity Maintainability
Fault Prevention Fault Tolerance Fault Removal Fault Forecasting
Faults Errors Failures
Resilience vs. dependability vs. robustness.
2014-‐06-‐02 DFG 2014 18
dependable (robust&resistent) systems
System states
• Highly controlled • Operates in a narrow band • Predefined states (“modes”) • Top-‐down design • Challenge: predict all states
caused by the environment
• A broad spectrum of possible equilibrium state • Not necessary all states are predicted • Adap*ve and evolving systems • impact of the system on the environment • Challenge:
• Adapta*on • Op*mal performance in different states
“Resilient systems” Environment
Extended development process
2014-‐06-‐02 DFG 2014 19
1JOSEPH FIKSEL Designing Resilient, Sustainable Systems , Environ. Sci. Technol. 2003, 37, 5330-‐5339
Example 2: Crowdsourcing
2014-‐06-‐02 20
Crowdsourcing the cosmos: Astronomers welcome all to idenGfy star clusters in Andromeda galaxy www.andromedaproject.org.
Crowdsourcing assump*on
2014-‐06-‐02 21
OEM Product company
In-house development
Off-shore development
centre COTS Out-
sourcing Crowd-
sourcing
Amount of accessible expertise
Crowdsourcing -‐ challenges • Provide aHrac*ve calls that will bring the wanted results
• Provide open architectures that enable – a constant flow of innova*on – efficient integra*on of components from both OEM and supplier perspec*ves.
• Efficient organiza*ons and processes – support balancing of openness and IPR management.
• Efficient assessment of quality aHributes for components and systems.
2014-‐06-‐02 22
Integra*on Engine
Component-‐tool
System design tool
Synthesis tool
Refactoring tool
Repository
Integrator
Supplier
External Repository view
CALL
Component development
Conformance checking
Acceptance test
Component Submission
Conclusion: Rapid con*nuous SE
• From Agile to Agile with systema*c SE • Enabling openness and con*nuous change of so.ware (at design and run-‐*me)
• Enabling adaptability, evolu*on • Keep quality • Increase innova*on capabili*es
2014-‐06-‐03 24
Papers @ ICSE 2014 • So.ware Engineering at the Speed of Light: How Developers Stay
Current using TwiHer, Leif Singer, Fernando Figueira Filho, and Margaret-‐Anne Storey
• Mining Behavior Models from User-‐Intensive Web Applica*ons, Carlo Ghezzi, Mauro Pezzè, Michele Sama, and Giordano Tamburrelli
• Automated Design of Self-‐Adap*ve So.ware with Control-‐Theore*cal Formal Guarantees, Antonio Filieri, Henry Hoffmann, and Mar.na Maggio
• Trading Robustness for Maintainability: An Empirical Study of
Evolving C# Programs, Nélio Cacho et al.
2014-‐06-‐02 25
FOSE, Session, Papers @ ICSE 2014 • So.ware Evolu*on and Maintenance, Václav Rajlich
• Session: Automated Bug Detec*on and Repair
• Session: Processes and Agile Development – Automated So.ware Integra*on Flows in Industry: A Mul*ple-‐Case Study, Daniel Ståhl and Jan Bosch
– Evidence-‐Based Decision Making in Lean So.ware Project Management, Brian Fitzgerald, Mariusz Musiał, and Klaas-‐Jan Stol
2014-‐06-‐02 26