hans petter dahle: programvareindustrien og fri programvare - erfaringer fra fou-prosjekter

Norges Forskningsråd støttet COSI og OSIRIS prosjektene

Eureka ∑! 2023 Programme, ITEA projects COSI and OSIRIS 1IKT Norge, 19.05.09

F A M I L I E R

Programvareindustrien og fri programvare –

erfaringer fra FoU-prosjekter

16. april 2009

Hans Petter DahleProsjektleder

IKT-Norge



F A M I L I E R



Del 1

COSI prosjektet

JL-3

ITEA if-04031

Co-development using inner & Open Source in Software Intensive systems

Start: 1.1.2006 (i Norge)Varighet 3 år

Norske aktiviteter:Først og fremst rundt prosesser

COSI GoOpen 20094

JL-4

Prosjektet Norsk COSI

Start: 1.1.2006 (i Norge)

Varighet 3 år

Norske aktiviteter: Fokus på prosesser, for distribuert utvikling Nasjonale demonstratorer

COSI GoOpen 20095

JL-5

the Netherlands

Spain

COSI Consortium

Finland

Sweden

12 partners in 5 countries

unique: 7 subcontractors in Norway

Norway

COSI GoOpen 20096

JL-6

Commodification of software

intra

companyopen cooperation

commodity

differentiatingtechnology life cycle

inter

companies

basic for the business

wasting valuableengineering resources

losingintellectualproperty

technology

COSI GoOpen 20097

JL-7

Problem statement

How to manage the complexity of heterogeneous distributed development

Challenges Larger groups working on the same software Development groups get more distributed Software shifts to commodity 3rd party software is

increasingly used

COSI addressed this issue WP1: Business & community models WP2: Process models WP3: Architecture

……

……

……

……

…………

COSI GoOpen 20098

JL-8

in houseoutsourcingbounded

collaborations

external

opencollaborations

company

control &ownership

knowledge

& cost

sharing

software shift

4 understand & use the shift

3 trust & understanding in open collaborations

2 agile developments in distributed settings

1 distributed concurrent heterogeneous development

COSI goals

COSI GoOpen 20099

JL-9

Leveraging OS opportunities

processprocess

repositoryrepositoryproductproduct

processprocess

repositoryrepository

ClosedClosed

OpenOpen

1 Practices1 Practices

2 OS tools2 OS tools

3 Using OSS3 Using OSS4 OSS products4 OSS products 5 Symbiotic5 Symbiotic

relationshiprelationship4 Opening up4 Opening up

productproduct

COSI GoOpen 200910

JL-10


collaborations

external

opencollaborations

company

control &ownership

knowledge

& cost

sharing

software shift

Achieved COSI Goals

1 Introduced distributed concurrent heterogeneous development

Partners moved to community-style collaboration Applications architectures were restructured with OS

tools and OSS components …

COSI GoOpen 200911

JL-11


collaborations

external

opencollaborations

company

control &ownership

knowledge

& cost

sharing

software shift

Achieved COSI Goals

2 Introduced agile developments in distributed settings

Partners introduced inner-source and collaboration oriented as their way of working

Partners integrated OS processes with SOA & MDD …

COSI GoOpen 200912

JL-12


collaborations

external

opencollaborations

company

control &ownership

knowledge

& cost

sharing

software shift

Achieved COSI Goals

3 Improved trust and understanding of open collaborations

We now know how to build attractive open applications & collaborations

Corporate partners evaluated, introduced & influenced OS tooling, e.g. subversion and GIT

…

COSI GoOpen 200913

JL-13


collaborations

external

opencollaborations

company

control &ownership

knowledge

& cost

sharing

software shift

Achieved COSI Goals

4 Understanding & strategic use of the software shift Policies are established on ISS or OSS use We learned how to contribute & influence communities …

COSI GoOpen 200914

JL-14

Case studies examples

ICT-Norway/eZ Systems eZ Publish Improved community involvement for core product

Improved process in place, but not much experience yet

ICT-Norway/Keymind KeyWatch Selection of open source tools used in own

development

Going open source with own product

COSI GoOpen 200915

JL-15

Case studies examples

ICT-Norway/Linpro OSS component selection Improved selection process

More structured, documented and transparent

Time and effort will be saved during component evaluation and selection

ICT-Norway/Linpro Nomadic Version control Because deltas are smaller, and limited in scope,

conflict resolution is easier to handle

Developers that used SVK are very satisfied with it

Problem concerning off-site development without access to Linpro's Subversion server, was solved

GoOpen 2009 Audun Jensvoll, Keymind Computing16

JL-16

Developing with OS componentsDeveloping with OS componentsSelection, integration and life-Selection, integration and life-

cycle issuescycle issuesKeymind ComputingKeymind Computing


JL-17

Develop with open source Develop with open source components – the workflowcomponents – the workflow

Find candidate componentsFind candidate components

Select best candidateSelect best candidate

Internal validationInternal validation

Integrate the componentIntegrate the component - architecture wrapping- architecture wrapping - domain adoption- domain adoption

Life cycle managementLife cycle management - workaround / bug fix- workaround / bug fix - upgrade - upgrade

Developer, Developer, domain expert, domain expert,

architectarchitect

HarvestHarvest

PreparePrepare

ConsumeConsume

Version control – snap shotVersion control – snap shot

AssessmentAssessment


JL-18

HarvestingHarvesting

Find candidates Find candidates

• Open source host sitesOpen source host sites

• Community mailing listsCommunity mailing lists

• General ”googling”General ”googling”

Assessing componentsAssessing components

• Apply check listApply check list

• Smaller is betterSmaller is better

• Low risk is betterLow risk is better

Short-listing check list: Short-listing check list:

Potential showstoppers: Potential showstoppers: Licence ok ?Licence ok ? Architecture ok ?Architecture ok ?

Community health / trust: Community health / trust: Committed industrial partnerCommitted industrial partner Community diversityCommunity diversity Project health: Project health: Mailing listsMailing lists Bug trackingBug tracking Version controlVersion control DocumentationDocumentation

Component healthComponent health Documented use /referencesDocumented use /references Reputation okReputation ok Domain scopeDomain scope Architecture complianceArchitecture compliance


JL-19

PreparePrepare - Test - Test

Test (usually only the chosen candidate component)Test (usually only the chosen candidate component) Goal: Verify that the component will work in our contextGoal: Verify that the component will work in our context It is expensive to fail this selectionIt is expensive to fail this selection Dependent of component typeDependent of component type

Small components: Small components: make a small prototype and test most important make a small prototype and test most important

functionality functionality make a preliminary integration layer make a preliminary integration layer

(could be empty)(could be empty) patch only if we mustpatch only if we must timeframe: from a couple of hours to a day or twotimeframe: from a couple of hours to a day or two

comp system


JL-20

Prepare - IntegratePrepare - Integrate

Loose coupling: Loose coupling: - Bind to snapshot from available (stable) releaseBind to snapshot from available (stable) release

- Never depend on future releasesNever depend on future releases- Use as isUse as is- Wrapping: glue, integration layer, anticorruption Wrapping: glue, integration layer, anticorruption

layer, access layerlayer, access layer

- 2 aspects of integration: 2 aspects of integration: - Domain - adapt to application’s domain language Domain - adapt to application’s domain language

(adapt to domain model, wrap in service)(adapt to domain model, wrap in service)- TranslatorTranslator- Service providerService provider

- Architecture: Maintainability, scalability, security, Architecture: Maintainability, scalability, security, distribution, etc. distribution, etc.


JL-21

ConsumeConsume

How to ”live with the component”How to ”live with the component”• Component included in development environmentComponent included in development environment

• Preferably packed as libraryPreferably packed as library• Under version control with source code and documentationUnder version control with source code and documentation

• Internal responsible, the component ”uncle”Internal responsible, the component ”uncle”• The developer who introduced the component, becomes The developer who introduced the component, becomes

”internal responsible” (component uncle)”internal responsible” (component uncle)• Follow discussion and developmentFollow discussion and development• Contribute to bug fix if relevantContribute to bug fix if relevant• If patch is applied in our development, update documentation If patch is applied in our development, update documentation

appropriateappropriate

• Stay with the snapshotStay with the snapshot• We don’t always need to upgrade to latest version (lazy We don’t always need to upgrade to latest version (lazy

upgrading)upgrading)

• Upgrade Upgrade • As bug fixAs bug fix• Because of changed requirements to our applicationBecause of changed requirements to our application


JL-22

Challenges / conclusionsChallenges / conclusions

• Process applicable for smaller componentsProcess applicable for smaller components Larger components (frameworks etc) has much more influence on Larger components (frameworks etc) has much more influence on

our application and requires more attention in all phases.our application and requires more attention in all phases.

• Payback to the OSS communitiesPayback to the OSS communities We get a lot of value from OSSWe get a lot of value from OSS Respect licences Respect licences If not committers or core developers, at least contribute as active If not committers or core developers, at least contribute as active

usersusers

• Main value: AvailabilityMain value: Availability For us, availability is main the value of OSSFor us, availability is main the value of OSS Appreciate openness about status, bugs, etcAppreciate openness about status, bugs, etc

• Organizational structureOrganizational structure Dependant of developer competence as “fisher men”Dependant of developer competence as “fisher men” Each component gets an uncle – take responsibility for the included Each component gets an uncle – take responsibility for the included

components. Binds some resourcescomponents. Binds some resources

GoOpen 2009 Martin T. Ranang, Linpro23

JL-23

Revision-Controlled,Revision-Controlled,

Distributed, and NomadicDistributed, and Nomadic

Development Using SVKDevelopment Using SVKLinproLinpro


JL-24

Linpro case studyLinpro case study

Linpro wanted to make it easy for their developers to perform nomadic, distributed, and fine-grained revision-controlled development through the introduction of a set of guidelines and the open-source tool SVK


JL-25

Context: Revision ControlContext: Revision Control

Serious software development, including OSS, use some kind of version control

Also known as source code management (SCM) or software configuration management (SCM)

Subversion has become the de-facto standard for revision control for OSS projects Great improvements over CVS include

atomic commit operations full revision history for renamed/moved files versioning of symbolic links


JL-26

Linpro Baseline ProcessLinpro Baseline Process

Almost all* of Linpro's development is placed under revision control with Subversion (SVN)‏

Based on experience and reasoning, our developers have found that fine-grained deltas, or commits with small changes, reduce the effort needed to handle merge conflicts, if they arise

Consequently, Linpro's developers prefer fine-grained deltas

Linpro strives to keep an up-to-date repository of all non-auto-generated material for every project on one of Linpro's central servers

If a developer is working off-site, the developer will perform commits over a secure network connection, using the svn+ssh protocol


JL-27

Linpro Improved ProcessLinpro Improved Process

Combining the use of the open-source tools Subversion and SVK Linpro accommodated to nomadic, distributed, revision-controlled development

This enabled Linpro's developers to continue doing fine-grained revision controlled development, but without the constraint of having to connection to Linpro's central revision-control repository for every revision-control operation

The improved process is documented in Linpro's contribution to the COSI Library of Assets, “Revision-controlled, Distributed, and Nomadic Development Using SVK”


JL-28

Linpro ResultsLinpro Results

The problem concerning off-site development without access to Linpro's Subversion server, described in the baseline process, was solved

The developers that have used SVK are very satisfied with it

It was a bit difficult to grasp which parts and actions of SVK that interact and are dependent on each other

There is still a risk of running into merge conflicts, but because the deltas are smaller, and limited in scope, the conflict resolution is easier to handle than when merging large commits

Understanding how SVK works was a bit confusing because SVK introduces a conceptual layer “above” Subversion but still shares a lot of functionality and command names with Subversion

A set of documented guidelines: The guidelines are documented in Linpro's contribution to the

COSI Library of Assets

COSI GoOpen 200929

JL-29

Incubator process

• Incubator process addresses the main risk of the OSS projects

• Provides an environment to achieve a stable release and gather a healthy community• Roles (e.g. Sponsor, Champion, Mentor, etc.)• Tool for support for the newborn projects• Workflow with phases and responsible role

• Existing projects’ experience is useful to the nascent projects

• Some structures responsible of making decisions are useful to guarantee the projects are “compatible” with ecosystem• Focus on architecture issues• Avoids duplicating efforts

COSI GoOpen 200930

JL-30

3 ways to use open source

Use Open Source software Integrate open source software in products and

the development environment

Share own source in the open Open own (commodity) software Grow a community around it Use external maintenance and improvements

Practices – inner source Internal development with open source

practices

COSI GoOpen 200931

JL-31

OSS selection - learnings

OSS selection is an important issue for software developers

Work with OSS evaluation and selection processes resulted in different process models

This reflects two common trends in software development: More time is spent on assessing, tailoring and integrating

components Commodification leads to general availability of OSS

components

Linpro established their own selection process

COSI GoOpen 200932

JL-32

OSS Selection – Case Studies Examples

Each company has addressed challenging issues within their organizations, and improved their process accordingly.

The selection processes differ with respect to Selection scope (see next slide) The focus and purpose of the improvement to

the selection process

COSI GoOpen 200933

JL-33

OSS Selection – Case Studies Examples

Different scope: Evaluation of single component vs. Selection from list of components Selection for single use (product/project) vs. Selection for multiple use

(organizational reuse) Selecting a component that will be part of a product vs. selecting a tool that

will be used in software development Different phases are covered (from evaluation only to full lifecycle).

Evaluation criteria are linked to company opportunities and risks

COSI GoOpen 200934

JL-34

OSS selection – Results

Selecting an OSS component needs to address company’s Requirements Purpose Goals

Identify critical selection criteria and collect information to assess them Understand licensing to avoid unwanted restrictions on a

product or service Community assessment is a crucial criteria in most cases

Do not forget to assess the components functionality

COSI GoOpen 200935

JL-35

Journals.Conferences and workshops papers 22 conference papers and 12 workshop papers on

international conferences Academic

• International Conference on Open Source Systems• International Conference on Quality Software • International Conference on Software Maintenance• International Conference on Model Driven Engineering Languages

and Systems • ….

Practitioners• Eclipse Summit Europe • Subversion Conference• Linux Conf Europe• ….

Journals (5) IEEE Software (3) IEEE Transactions on Software Engineering International Journal of Business Innovation and Research

COSI GoOpen 200936

JL-36

Dissemination

Achievements: More than 50 publications by COSI members 4 main events organised Many more events attended Significant internet based dissemination. Web site has

been maintained alive and full of contents

Web sites: Library of assets hosted by Friprog

(Norwegian Open Source Competence Center)• www.friprog.no/Laer-mer/Prosjekter/COSI-Library-of-Assets

External COSI website will be maintained during 3 more years

• www.itea-cosi.org

COSI GoOpen 200937

JL-37

Internet

Library of Assets

Collection of project results that are available for the general public on the internet

Addresses inner and open source related topics

Contains Process descriptions Architecture designs Papers on issues studied during COSI Documented experiences from case studies …

COSI GoOpen 200938

JL-38


collaborations

external

opencollaborations

company

control &ownership

knowledge

& cost

sharing

software shift

4 understand & use the shift

3 trust & understanding in open collaborations

2 agile developments in distributed settings

1 distributed concurrent heterogeneous development

Conclusions – goals reached



F A M I L I E R



Del 2

OSIRIS prosjektet



F A M I L I E R

Prosjektet Norsk OSIRIS

• Open Source tjenesteplattform• Start: 1.7.2005 (i Norge)• Varighet 3 år• Norske aktiviteter:

• Bidrag til felles tjenesteplattform• Nasjonale demonstrator prosjekter



F A M I L I E R

Open Source Infrastructure for Run-time Integration of Services

InternetHome

Gateway

ManagementCenter

GPS

ContentProvider

Service/ContentAggregator

ServiceProvider

MobileAccess

Surveillance

RemoteWork

MultimediaServices

DomoticServices

OSIRIS is an across-domain open source service platform that will provide support for services provisioning, aggregation, delivery, dynamic adaptation to the context and lifecycle management through smooth integration of devices.



F A M I L I E R

An OSIRIS scenario

Architecture: OSIRIS



F A M I L I E R

OSGi



F A M I L I E R

Osiris architecture in a nutshell



F A M I L I E R

Technology demonstrators

e2e demonstrator

Business demonstrators

Financial services Mobile CRM Tax Services

Demonstrators organisation

Norges Forskningsråd, IKT programmet, OSIRIS prosjektet

Eureka ∑! 2023 Programme, ITEA project ip04040, OSIRIS 46IKT Norge, 19.05.09

F A M I L I E R

What the CRM demonstrator does

• Contact and Diary information available on the mobile

• Live replication of CRM data from the central database

• Abstract of a number of the last interactions between his company and a specific customer

• Smart adaptation of services used• Map and route planner depending on price, bandwidth,

accuracy

Eureka ∑! 2023 Programme, ITEA project ip04040, OSIRIS 47ICT Norway 5/19/09

Demo



F A M I L I E R

Mobile CRM service

Internet SuperOffice CRM Client

TellU

CRM DemoPosition

Firewall

SMS

MMS

Call

Control

SuperOfficeCRM server

mobile

network

GSN

UMTS

GSM

Firewall

1

2

4

3

Map provider



F A M I L I E R

Tax Service

OSIRISInternetConnector ex. 1

Service frameprotocol

OSIRISInternetConnector ex. 2

Web Service

Client

Mobilephone (Light

Weight

node)

Digitalinteractive TV

(Light weight

node)

Tax Authority

Web Services

JVM

OSGi

OSIRIS Basic Node

TellU Forms distribution and parsing system.

OSIRIS Profile Extensions



F A M I L I E R Ny e retningslinjer for mobile og DTV

t jenester

1. Nav igasjon og arbeidsfy t

2. Feil3. Søk og spørsmål4. Input / output-

tek nik k er5. Tid6. Tek st og språk7. Stemme og ly d8. Gra fk k9. Tall10. Hjelp og suppor t



F A M I L I E R

Demonstrator

Skjema-motor

mobil

XFormsskjema-

base

• Skjema utvikles med skjemabygger• Output format er XForms XML• Skjema kan inneholde lyd og video• Skjemaer lagres i en skjemabank• Skjemaer vises på mobil av Tellus skjemamotor i hht tilpasset Elmer• Datafangst og validering utføres av Tellus skjemamotor• Resultat XML lagres i et passende repository (Altinn)

Skjemamotor ligger dels på server ogdels på mobiltelefon eller Settop boks.

Det tar kort tid å utvikle et nytt skjema. Tiden er avhengig av kompleksiteten på skjemaet.

Skjema-bygger

Utviklingav skjemaer

XForms

Eureka ∑! 2023 Programme, ITEA project ip04040, OSIRIS 52ICT Norway 5/19/09

Exploration

• SuperOffice • mobile client under development, feature

complete now

• TellU:• Own product Actor Frame published as Open

Source • Part of Telenor’s connected objects platform

• Forms engine used when building services• Focus on universal design for user interface



F A M I L I E R

http://www.itea-osiris.org