Tapahtumanhallinnan pulmakohtia ja ratkaisuja

TJTSE54 - Kehittämismenetelmät ja arkkitehtuurit liiketoiminnassa, kevät 2007

Ville Seppänen <rissepp@jyu.fi>

Kertausta● Transaktio eli tapahtuma● Tietokantatapahtumien tulee täyttää nk. ACID-

vaatimukset– Atomicity = Jakamattomuus– Consistency = Johdonmukaisuus– Isolation = Erityvyys– Durability = Pysyvyys

● Ihanneoloissa samat vaatimukset voitaisiin asettaa kaikenlaisille tapahtumille, mutta käytännössä sitä ei aina voida/kannata toteuttaa

Jakamattomuus● Tapahtuman (T) järjestelmään tekemät

muutokset vahvistetaan (commit) ainoastaan mikäli kaikki T:n operaatiot onnistuvat. Muussa tapauksessa T keskeytetään (abort)

● Virheen tapahtuessa järjestelmään mahdollisesti jo tehdyt muutokset perutaan (roll-back, backward error recovery) siten, että järjestelmän alkuperäinen tila palautetaan (esim. lokitietojen perusteella)

Tapahtuma ja operaatio?● Tapahtuma T = operaatiot t

1, ..., t

n

● T = Siirretään 100€ tililtä A tilille B● t

1 Luetaan A:n saldo AS

● t2 Kirjoitetaan AS = AS - 100

● t3 Luetaan B:n saldo BS

● t4 Kirjoitetaan BS = BS + 100

● T = Myydään tavaraa● t

1 Vastaanotetaan tilaus

● t2 Käsitellään tilaus

● t3 Vähennetään tuotteen varastomäärää

● tn ...

Johdonmukaisuus● T:n järjestelmään tekemien muutosten tulee

vastata kuvatussa ilmiössä tapahtuvia muutoksia ja muutosten tulee tapahtua järjestelmän eheysvaatimusten (integrity constraints) puitteissa

● Useimmissa tapauksissa järjestelmä on väistämättä epäjohdonmukaisessa tilassa jossain T:n välivaiheessa. Eheys tarkistetaan tavallisesti vasta T:n pyytäessä vahvistusta (deferred integrity constraints)

Johdonmukaisuus

State n State n+2State n+1ACID ACID

Read(A)Write(A, (A-10))Read(B)Write(B, (B+10))

Temporary inconsistency

Erityvyys● Järjestelmän samanaikaisesti suorittamat

tapahtumat täytyy eristää toisistaan; ts. ne eivät saa päästä käsiksi epäjohdonmukaisessa tilassa oleviin resursseihin (vrt. edellinen)

● Virheelliset tulokset● Ketjuuntuvat peruutukset (cascading rollback)● Eriytyvyyden suhteen useita eri

lähestymistapoja mm. optimistinen samanaikaisuuden hallinta, aikaleimaus, lukitusmenetelmät

● Sopiva menetelmä tapauskohtaisesti

Eriytyvyys

read(A)read(A)write(A, (A+10))commit

write(A, (A+10))commit

update loss

read(A)write(A, (A+10))

read(A)write(A, (A+10))commit

abort

dirty read

read(A)read(A)write(A, (A+10))commit

read(A)commit

inconsistent read

Pysyvyys● Tapahtuman järjestelmään tekemät muutokset

eivät saa kadota tahattomasti tai tapaturmaisesti

Long living transaction (long running, pitkäkestoinen)

● Has a long duration compared to the majority of other transactions either because– It accesses many database objects– It has lenghty computations– It pauses for inputs from the users (or processes)– Or a combination of these factors– About the lenght of time of execution, not size of a

transaction

Distributed Transaction mngntHajautettu tapahtumanhallinta

tx_being; order_part; withdraw_part; payment;tx_commit;

Inventoryapplication

Billingapplication

DB

DB

Site 1

Site 2

withdraw_partreturn_partstock_level

order_partbillingTransaction

coordinator

DBMSLocal appsResource mngrs

Subtransactions

Kaksivaiheinen vahvistus2-phase commit protocol

● Atomic commitment protocol– An algorithm that ensures that all the processes

involved in a distributed transaction either commit or abort

● Coordinator process– Responsible for controlling the overall atomicity– Controls the 'voting'

● A number of participating processes– Execute parts of a distributed transaction

Kaksivaiheinen vahvistus2-phase commit protocol

● Voting procedure – phase one– The coordinator sends vote requests to all

participants– When a participant receives the vote request it

replies by voting either Yes or No, according to whether it is able to carry out the task

– Participants that voted Yes start waiting for a confirmation message from the coordinator

– Participants that voted No can unilateraly abort

Kaksivaiheinen vahvistus2-phase commit protocol

● Voting procedure – phase two– The coordinator collects all the vote messages– If all the participants voted Yes the coordinator

decides to commit and sends Commit messages to all participants

– Otherwise, the coordinator decides to abort and sends Abort messages to all participants that voted Yes

– According to the received message, a participants decides to commit or abort

Kaksivaiheinen vahvistus2-phase commit protocol

● It is possible that messages may not arrive due to a failure and processes may be waiting forever: a timeout mechanism must be able to interrupt the waiting period

● In addition, the coordinator may attach the list of participants to the vote request message and thereby let the participants to know each other: Cooperative termination protocol

Cooperative termination protocol● If participant p comes across the timeout while

waiting for the Commit or Abort message from the coordinator, it can requrest this from the participant q

● If q has already decided to commit (or abort) it sends a Commit (or Abort) to p

● In the ase that q has not voted yet it can decide to abort and send Abort to p

● If q has voted Yes but has not received the final Commit or Abort request from the coordinator it cannot help p in making the decision

Nested transactions● A mechanism to facilitate transactions in

distributed systems● A tree-like model with parents, children, top-

level (root), and leaves

T0

T3 T4

T2T1

Top-level

Parent

Child

Leaf

Nested transactions● Rules

– A parent can spawn any number of children– Any number of children may be active concurrently– Parent can't access data when its children are alive– A child can inhereit a lock held by any ancestor– On child commit, its lock are inherited (anti-

inheritance) by parent

Nested transactions● Rules, continued

– Commit dependency: parent can commit only after all its children terminate (commit/abort)

– Abort dependency: on parent abort, updates of committed children are undone

– Updates persist only if all ancestors commit● Kotitehtävä: perustele säännöt

Nested transactions

void send_all_salaries() { Transaction tx = new Transaction(); tx.begin(); for(int i=0; i<emplCount; i++) { send_salary(employerAccount, employeeAccount[i], amount[i]); } tx.commit();}

void send_salary(from, to, amount) { Transaction tx = new Transaction(); tx.begin(); from.remove(amount); to.add(amount); tx.commit();}

Nested transactions● Intra-transactional parallelism

– Safe concurrency– Reduced response time

● Intra-transactional recovery control– Finer control on error handling– Improves availability

● System modularity– Composition of separately developed modules

Save points● A check point in a transaction that forces

system to save the state of the running application and return a save-point identifier for future references

● Instead of removing the entire transaction after a failure of single operation (sub-process) backward recovery can return the last valid state of the transaction saved in the save-point reference

Compensation & Saga transactions● Long-living and distribued transactions are not

reasonable to implement as single ACID transactions– Keep resources locked for the long periods of time,

which can significantly delay the commit of other simultaneously executed transactions

– Deadlock frequency grows with the fourth power of the transaction size

Compensation & Saga transactions● Atomic commitment protocols ease the

implementation of ACID transactions in distributed systems but further transactions to become long living

● Backward recovery is expensive and difficult to implement

● The idea of compensating transactions (or processes, operations) was introduced to simulate the transactional properties in such applications

Compensation & Saga transactions● In Saga transaction model a long living

transaction is broken up into a collection of subtransactions that interleave with other transactions– The results of a subtransaction can be made

immediately visible once it has committed

Compensation & Saga transactions● Saga subtransactions can be executed

independently but they must finally form an atomic unit– Should any of subtransactions fail the failed

transaction is aborted and already committed subtransactions are undone

– Removing the committed transaction means that results of subsequent transactions may become inconsistent and cause cascading rollbacks

● Instead of removing the results of failed transaction by using rollback, the idea of compensating transaction is intruduced

Compensation & Saga transactions● Each substransaction is a real transaction in the

sense that it preserver DB consistency● However, they are related to each other and

any partial executions of the saga are undesirable; if they occur the compensation takes place

Compensation & Saga transactions

● Each saga substrabsactions Ti is provided with

a compensating transaction Ci

– If the compensation is not needed saga T1, T

2, ..., T

n

will executed as a sequence of transactions● The compensating transaction undoes, from a

semantic point of view, any of the actions performed by the transaction but doesn't necessarily return the database to the state that existed when the executing of transaction began

Compensation & Saga transactions

● For the transaction T1, T

2, ..., T

n the

compensating transaction C1, C

2, C

n-1 is defined

● In case of compensation the sequence executed is T

1, T

2, ..., T

j, C

j, ..., C

2, C

1; where 0

<= j < n

Compensation & Saga transactions● Contrary to traditional backward recovery

mechanism, which is based on compensating operations automatically deduced from the schedule, semantically compensating transactions may not be automatically generated– Even simple compensating actions (such as + & -

operations) must follow the integrity constraints– Everything cannot be undone

● For example, 'real actions' with 'real consequences': impossible to undo but it may be possible to override the effect with a new action

– Committed acceptable & aborted acceptable termination states: 'business considerations'

Forward recovery● Combining the save-points, compensation and

recovery of failed transaction– Transaction that caused the failure is aborted using

the conventional rollback– Committed transactions are undone in reversed

order using their compensating counterparts until the save-point is found (backward recovery)

– Finally, the transaction is restarted from the location of save-point (forward recovery)

Forward recovery

C GF

B ED IHA

Execution graph (arrow) and compensation graph (dashed arrow)

Forward recovery● When recovering, it's not always feasible or

even possible to complete the transaction by re-executing the original steps again

● However, it's possible that the same objective can be achieved with different ways. Alternative methods of forward recovery should be considered during the process design

Transactional processes● Typical scenario: a top-level process consists of

subcomponents that are logically connected to the higher level process but execute virtually independently

● A global transaction– A collage of all separate transactions that are

included in the process– Often long-living and distributed

● Each global transaction can be divided in a set of local transactions– Can be represented as a tree of hierarchically

ordered tasks (or a graph), which may also include manual operations

Transactional processes

Sales Book Invoice Payment

Send documents

Cancel

Accommodation

Transportation

Preparedocuments

Writebooking

Bookingapp

Bookingapp

DB

DB

Subprocesses Local transactionslocally ACID

Global transaction structure (top-level process)globally ACID (atomic commitment), distributed, long-living

Transactional processes

Sales Book Invoice Payment

Send documents

Cancel

Accommodation

Transportation

Preparedocuments

Writebooking

Bookingapp

Bookingapp

DB

DB

Sub-processesmay execute nestedsaga or some othermodel dependingon the task

Apply save-points on top-level process workflow.Provide with compensating processes. Globally ACID

Recovery mechanism required also on lowerlevels

Local transactions – Atomic commitment DB-level transactionsManaged by DBMS

Kysyttävää?