let the continuous build embrace your database
DESCRIPTION
JUnit tests should not depend on database state." - "Set up your test data before you run your test." - We figure this just does not always scale. Mocking test data for hundreds of tables may not be suitable and database schemes evolve as the application does. These are common challenges when developing large J2EE systems. This presentation shows practical approaches for project setups and (functional) tests that decided to depend on a database. Developers and build servers may or may not share the database and support for this should be as simple as possible. We give an overview of what proved to be a good setup for an Eclipse / Maven 2 based development and build environment that relies on an evolving relational schema.TRANSCRIPT
Continuous Database BuildAnd: Practical Approaches For DB Based Functional Testing
Stefan Rufer – Netcetera AG
Michael Wernli – Telekurs Card Solutions AG
3020
2
Speakers
> Stefan Rufer
– Studied business IT at the University of Applied Sciences in
Bern
– Senior Software Engineer and Architect at Netcetera
– Main interest: Server side applicat ion development using JEE
> Michael Wernli
– Cert if ied engineer in IT
– Senior Software Engineer at Telekurs Card Solut ions
– As Applicat ion DBA in charge of large- scale cashless payment
t ransact ion system.
3
Why are we here?
> You will see how a cont inuous database build improved our
overall build stability and helped us to focus on business
problems during integrat ion tests.
4
Motivation
> “Who the #@!* dropped the salary column in the build database?!”
– Dude, you didn‘t run the script to add it in the f irst place…
> Did the build just turn red…
– because someone forgot to clean up the build database –
AGAIN?
> svn revert drop-that-table.sql
– Hmm, the table is st ill missing in the DB…?
> ……
5
Motivation
6
AGENDA
> Continuous Build For The Database
> Re- Runnable Functional JUnit Tests
> Eclipse/ Maven Setup For Functional JUnit Tests
> Outlook / References
7
> Continuous Build For The Database
> Re- Runnable Functional JUnit Tests
> Eclipse/ Maven Setup For Functional JUnit Tests
> Outlook / References
8
The Problem
> 100..200 SQL scripts (DDL+ DML) per major release
> Scripts have to be runnable f irst - to- last at all t imes
> 20 developer/ build/ integrat ion databases to keep in sync
Poor Database Administrator!
9
The Idea
> Install database scripts as soon as they are commited to the
subversion repository.
> Run SQL script against reference database. If installat ion
terminates without error, populate to all other databases.
> Store in the database what script was installed in what subversion
(SVN) revision.
10
Basic Setup of a Continuous Database Build
11
Demo Part I
12
Different behaviour of database sources and Java sources during build
13
Demo Part II
14
We need re- runnable (idempotent) database source scripts
15
Delta and Source Scripts
> Most dif f icult is data definit ion
or data modif icat ion scripts.
– delta-scripts: „run once“
– Add idempotency manually
> Packages, t riggers, views can
be reinstalled easily.
– source-scripts: „run
however“
– Idempotent by design
16
Branching for database source scripts is a bit different
17
Demo Part III
18
Quality Checks
> Automated quality checks for
– Primary key constraint names
– Unique key constraint names
– Foreign key constraint names
– Check constraint names
– Index names for all of the above
– Sequence names
> Hooks provided to start quality
checks
19
> Continuous Build For The Database
> Re- Runnable Functional JUnit Tests
> Eclipse/ Maven Setup For Functional JUnit Tests
> Outlook / References
20
JUnit Tests Modifying Database
21
How This All Relates
> JUnit tests that do insert / update/ delete must not leave their
t races in the build database. Auto rollback JUnit class
> Good experience by providing personal database to each
developer.
– But: How to keep it up to date? Cont inuous Database Build.
> Developers want to switch between a personal and a build
database. Eclipse/ Maven Setup
22
The Idea
> AutoRollbackTestCase (extends JUnits class TestCase)
> Datasource and JDBC connect ion managed internally
> Rollback called during tearDown method of test no t races!
23
Undo, Undo!
class ExampleDBTestCase extends AutoRollbackTestCase {
public void testDbInsert() throws SQLException { Statement stmt = getDataSource().getConnection().createStatement();
stmt.executeUpdate("INSERT INTO testtable values ('X')");
ResultSet rs = stmt.executeQuery("SELECT * FROM testtable"); assertTrue("expect one row", rs.next()); assertEquals("X", rs.getString("testrow")); assertFalse("expect only one row", rs.next()); }
}
24
Little Burden – Big Deal
> JDBC insert / update/ delete statements are a crucial part of
enterprise applicat ions. They need to be tested!
> Avoid the burden of cleaning up the database (someone will
forget anyway…)
> Unit tests that roll back at the end offer a simple way to run SQL:
– Verify correct SQL syntax
– Verify database state after update
> Spring offers the possibilit ies of AutoRollbackTestCase (and much
more), check out spring-test.jar /
org.springframework.test
25
> Continuous Build For The Database
> Re- Runnable Functional JUnit Tests
> Eclipse/ Maven Setup For Functional JUnit Tests
> Outlook / References
26
Properties Overloading
> ConfiguredTestCase reads junit .propert ies
> If junit_fritz.propert ies is available for user „fritz“, it is read as
well
Overwrites propert ies in junit .propert ies
> How does this look like?
user=buildpassword=buildurl=jdbc:derby:builddb
example=buildsrv:80
user=integrationpassword=integration
example=localhost:8080
reuse URL
junit.properties junit_fritz.properties
27
KISS – But How?
> Scope: Mult imodule build setup using Maven 2 and Eclipse
> Eclipse and Maven classpath handling dif ferent
> Eclipse: Mult iple projects referencing each other results in a
„global“ classpath.
Where to place database connect ion informat ion for unit tests?
> Maven: Each module needs junit .propert ies in
src/test/resources if we want to avoid test resources in the
main source t ree.
Where to place database connect ion informat ion for unit tests?
28
Eclipse Setup For Functional JUnit Tests
> User adds his one and only junit_USERNAME.propert ies f ile to
„toplevel“ project
> Eclipse uses this for all JUnit runs as it is always on the classpath
User specif ic database connect ion set t ings in Eclipse.
29
Maven Setup For Functional JUnit Tests
> junit .propert ies f ile in each submodule
> junit .propert ies contains variables:
> Values def ined in Maven POM…
…and injected during Maven lifecycle process-test-resources
db.user=${db.user}db.password=${db.password}db.schema=${db.schema}db.url=${db.url}
<db.user>build</db.user> <db.password>build</db.password><db.schema>build</db.schema><db.url>dpas10</db.url>
junit.properties
pom.xml
30
Test- properties: Define once, use everywhere
31
> Continuous Build For The Database
> Re- Runnable Functional JUnit Tests
> Eclipse/ Maven Setup For Functional JUnit Tests
> Outlook / References
32
Outlook
> Continuous Database Script Suite current ly tailored to Oracle
systems
– Consider generic implementat ion or at least ident ify and
isolate specialized points
> Test data management is the next big topic on our radar. As we
have to select sensible stuff f rom 1TB of data we need some
clever ideas…
> Integrate Maven/ Eclipse setup with a tool like MavenIDE.
33
Links / References
> Full source for continuous database build system
http:/ / www.stefanrufer.ch/ dbbuild
> Spring Test inghttp:/ / stat ic.springframework.org/ spring/ docs/ 2.5.x / reference/ test ing.html
34
The End
35
Two things to remember
Store in the database how the database was
built.
Write JUnit tests that roll back.
Stefan Rufer [email protected]
Netcetera AG www.netcetera.ch
Michael Wernli [email protected]
Telekurs Card Solutions www.telekurs.com