Experiences with the Apache Software Foundation

Marlon Pierce, Suresh Marru, Raminder Singh, Gerald Guo, Lahiru

Gunathilake

E-Science Software Sustainability

• The National Science Foundation has recognized that it needs strategies for sustainable software that it funds.– Cyberinfrastructure Software Sustainability and

Reusability: Report from an NSF-funded workshop• C. Stewart, et al.

– Cyberinfrastructure Framework for 21st Century Science and Engineering (CIF21)

– Software Infrastructure for Sustained Innovation (SI2) program

What Are Ingredients for Sustainable Software (Besides Funding)?

• Open source or free licensing– Apache or GPL license families

• Code should use supporting tools such as those from SourceForge, GitHub, etc– Version control software, trouble tickets, mailing lists, IRC, etc.

• Project management philosophy• Meritocracy, benevolent dictator, etc.

• Reproducible build and test• Committed user base• Open community

– Overlooked but key– Process for attracting new developers– Encourage and reward projects for diversifying– Need to go beyond the SourceForge model

Some Organizational Models for Sustainable Academic Software

• Institution-led consortia– Sakai, Kuali, HUBzero

• NSF-funded communities– Globus CDIGS– NCAR

• Non-profit spin-offs– HDF Group

• But these don’t scale down for smaller projects

Problem and Proposed Solution

• Problem Statement: how can we improve the chances of good academic software surviving the departure of its principal developer(s).

• Our proposed solution: Use the Apache Software Foundation– Not model ourselves after it—join it.– Apache will provide the necessary community support and

project lifecycle enforcements to improve the chances that good software flourishes.

• Evaluating this hypothesis is part of our NSF SDCI funding.

Apache Software Foundation Key Elements

• Open community, not just open source– Decisions are made by voting from the project

members– Projects succeed by attracting more contributors

and committers, not just downloads– All discussions and decisions must be done on

public mailing lists.• Successful members can be selected to steer

Apache as a whole.

Apache Project Lifecycle

• Projects are proposed and voted in or out.– Usually this is to publicize a new project– But Open Office was very contentious.

• New projects go through an incubation phase– Champion, mentors from Apache– Code donations– Defined criteria to graduate

• Failure is an option– Incubation projects can fail to graduate– Other projects can be retired, moved to the attic, suspended,

etc (Xalan)

Getting Started

• We have started two Apache Incubators– Rave: OpenSocial and W3C widget portal– Airavata: Scientific Workflows

• First step: make a software disclosure to IURTC.– This was fairly painless– Key is to make sure you get permission from any one

who has contributed to the code.• Second step: find a champion– An established Apache member– Will propose your project as an incubator

Further Steps

• Sign and submit an Individual Contributor License Agreement (CLA)– “You hereby grant to the Foundation and to recipients of

software distributed by the Foundation a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute your contributions and such derivative works.”

• Get someone wearing a tie to sign a Software Grant Agreement and an Corporate CLA.

Last First Steps

• Expunge inappropriately licensed and export restricted code from your code base

• Contribute it.– Basically, do and “svn import” of your code into

your area of the Apache SVN.• Congratulations, you are now an incubator

and must figure out how to graduate– There is no set criteria.– Your champion and mentors help determine this.

• Apache Rave is a joint submission of OneHippo, MITRE, SURFNet, and IU.– Unusual submission

• It’s a generalization of the OGCE Gadget Container• Champion: Ate Douma • Mentors:

– Sylvain Wallez– Upayavira– Ross Gardler– Hadrian Zbarcea

• Apache Airavata consists of the workflow tools developed by the OGCE– More traditional submission

• Champion: Ross Gardler• Mentors:

– Alek Slominski– Ate Douma– Sanjiva Weerawarna– Paul Fremantle– Chris A. Mattmann

• Website design by Barbara Hallock

Apache Connections

• Successful incubator graduation depends on establishing connections with other Apache projects.

• Rave: Shindig, Wookie• Airavata: ODE, Axis2, QPID, OODT

Is It Worth It?

• Early impressions are positive• We’ve been introduced to a number of great

developers.– It is useful to spend time looking at other people’s

codes– And it is useful to have even informal code

reviews, so make sure your code is clean and you have your unit tests in place.

Apache Contributions Aren’t Just Software

• Apache contributors and committers aren’t just code writers.

• Successful communities include– Web developers– Documentation and tutorial writers– Testers, requirements providers, and constructive

complainers • Using Jira and mailing lists

More Information

• Links below are to the Websites– Further links to mailing lists, SVN browsing, Jira

system, Jenkins, Sonar, and other Apache infrastructure

• Apache Rave: http://incubator.apache.org/rave

• Apache Airavata: http://incubator.apache.org/airavata/