language integrated stm in c# using the roslyn...

class Account

{

public atomic long Balance { get; set; }

static void Main()

{

va

r salary = new Account(1000);

va

r studentLoa

n = new Account(-50000);

TransferMoney(1000, salary, studentLoan);

}

public Account(long balance)

{

Balance = balance;

}

public static void TransferMoney(long am

ount, Account from, Account to)

{

at

omic

{

from.Subtract(amount);

to.Add(amount);

}

}

public void Add(long amount)

{

at

omic

{

Balance += amount;

}

}

public void Subtract(long amount)

{

at

omic

{

if (Balance - amount >= 0)

{

Balance -= amount;

}

else

{

throw

new Exception(”Insuffic

ient funds!”);

}

}

}

}

Language Integrated STM in C#Using the Roslyn Compiler- An Alternative to Locking

Tobias Ugleholdt Hansen

Andreas Pørtner Karlsen

Kasper Breinholt Laurberg

{

Department of Computer ScienceSelma Lagerløfs Vej 300

DK-9220 Aalborg Øhttp://www.cs.aau.dk

Title:Language Integrated STM in C# Us-ing the Roslyn Compiler - An Alter-native to Locking.

Project Period:Spring Semester 2015

Project Group:dpt109f15

Participants:Tobias Ugleholdt HansenAndreas Pørtner KarlsenKasper Breinholt Laurberg

Supervisor:Lone Leth Thomsen

Page Numbers: 120 + 4 appendices

Date of Completion:June 8, 2015

Abstract:

This master thesis investigateswhether language integrated STM isa valid alternative to locking in C#in terms of usability, and provides ad-ditional benefits compared to library-based STM. To do so, an extensionof C# called AC# was implemented.AC# provides integrated support forSTM, including conditional synchro-nization using the retry and orelseconstructs, and nesting of transac-tions. AC# was implemented by ex-tending the open source Roslyn C#compiler. To power AC# a library-based STM system, based on theTLII algorithm, was implemented.The extended compiler transformsAC# source code to regular C# codewhich utilizes the STM library. Foreach concurrency approach: AC#,library-based STM and locking inC#, four di↵erent concurrency prob-lems, representing di↵erent aspectsof concurrency, were implemented.These implementations were ana-lyzed according to a set of usabilitycharacteristics, facilitating a conclu-sion upon the usability of languageintegrated STM. Our evaluation con-cludes that AC# is a valid alterna-tive to locking, and provides betterusability than library-based STM.

http://www.cs.aau.dk

PrefaceThis report documents the master thesis done by group dpt109f15 at theDepartment of Computer Science at Aalborg University. The thesis waswritten as part of the Computer Science (IT) study program in the spring of2015 at the 10th semester.

The first time an acronym is used it will appear in the format: SoftwareTransactional Memory (STM). Inline quotations and names will appear initalics. The work presented in this report is based on work or results describedin books, articles, video lectures, and research papers from outside sources.The full list of acronyms along with the bibliography, appendix, and summarycan be found at the end of the report.

We would like to give a special thanks to our supervisor Lone Leth Thomsen,from the Department of Computer Science, for her excellent guidance andimmaculate attention to details. She supplied invaluable help throughoutthe project with professionalism and black humour. Her feedback has beenindispensable and has significantly raised the quality of the final result. Herconstructive criticism helped us to narrow down the subject and kept usmotivated and enthusiastic about the project.

The report is structured with dependencies between the chapters, and thefollowing can be used as a reading guide:

• Chapter 1 “Introduction” presents the motivation for choosing the topic,the related work, the scope, and the hypothesis. Lastly the method ofevaluation is presented.

• Chapter 2 “Background Knowledge” establishes the term “locking”,and the key concepts of Software Transactional Memory (STM). Thisknowledge is required in order to understand the remaining work. If thereader is familiar with these areas, this chapter can be skipped.

• Chapter 3 “Roslyn” outlines the structure of the Roslyn compiler, whichenables the integration of STM into C#.

• Chapter 4 “Requirements for AC#” analyze the requirements to theSTM system which executes transactions in AC#.

• Chapter 5 “Design and Integration” describes the decisions related todesigning and integrating AC# based on the requirements.

i

PREFACE ii

• Chapter 6 “STM Implementation” describes the implementation of theSTM system that powers AC#, and is based on the requirements anddesign choices.

• Chapter 7 “Roslyn Extension” describes how Roslyn is extended toencompass language integrated STM, thus being a compiler for AC#.

• Chapter 8 “Evaluation of Characteristics” evaluates AC#, its associatedSTM library and locking in C# according to the evaluation methoddescribed in Section 1.5.

• Based on this evaluation, a conclusion on the hypothesis is made inChapter 9 “Conclusion”.

• To reflect on the decisions made throughout the report, Chapter 10“Reflection” discusses the choices made and their consequences.

• Continuation of the work in the future and its potential is discussed inChapter 11 “Future Work”.

PREFACE iii

Tobias Ugleholdt [email protected]

Andreas Pørtner [email protected]

Kasper Breinholt [email protected]

ContentsPreface i

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . 51.5 Evaluation Method . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Background Knowledge 82.1 Locking in C# . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 STM Key Concepts . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 Roslyn 143.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2 Roslyn Architecture . . . . . . . . . . . . . . . . . . . . . . . . 153.3 Compiler Phases . . . . . . . . . . . . . . . . . . . . . . . . . . 173.4 Syntax Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4 Requirements for AC# 264.1 Tracking Granularity . . . . . . . . . . . . . . . . . . . . . . . . 264.2 Transactions & Variables . . . . . . . . . . . . . . . . . . . . . 284.3 Strong or Weak Atomicity . . . . . . . . . . . . . . . . . . . . . 294.4 Side-e↵ects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.5 Conditional Synchronization . . . . . . . . . . . . . . . . . . . . 334.6 Nesting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.7 Opacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.8 Summary of Requirements . . . . . . . . . . . . . . . . . . . . . 36

5 Design and Integration 375.1 Transactional Blocks & Variables . . . . . . . . . . . . . . . . . 375.2 Transactional Parameters . . . . . . . . . . . . . . . . . . . . . 405.3 Example of AC# . . . . . . . . . . . . . . . . . . . . . . . . . 435.4 Conditional Synchronization . . . . . . . . . . . . . . . . . . . . 455.5 Nesting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.6 Summary of Design . . . . . . . . . . . . . . . . . . . . . . . . . 48

6 STM Implementation 506.1 Implementation Criteria . . . . . . . . . . . . . . . . . . . . . . 506.2 Selection of Algorithm . . . . . . . . . . . . . . . . . . . . . . . 526.3 Library Interface . . . . . . . . . . . . . . . . . . . . . . . . . . 56

iv

CONTENTS v

6.4 Internal Details . . . . . . . . . . . . . . . . . . . . . . . . . . . 606.5 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

7 Roslyn Extension 677.1 Extension Strategy . . . . . . . . . . . . . . . . . . . . . . . . . 677.2 Lexing & Parsing Phases . . . . . . . . . . . . . . . . . . . . . . 697.3 Syntax Tree Transformations . . . . . . . . . . . . . . . . . . . 737.4 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 847.5 Design and Integration Revisited . . . . . . . . . . . . . . . . . 86

8 Evaluation of Characteristics 908.1 Implicit or Explicit Concurrency . . . . . . . . . . . . . . . . . 908.2 Fault Restrictive or Expressive . . . . . . . . . . . . . . . . . . 918.3 Pessimistic or Optimistic . . . . . . . . . . . . . . . . . . . . . . 918.4 Readability & Writability . . . . . . . . . . . . . . . . . . . . . 92

9 Conclusion 1089.1 Problem Statement Questions Revisited . . . . . . . . . . . . . 1099.2 Hypothesis Revisited . . . . . . . . . . . . . . . . . . . . . . . . 109

10 Reflection 11210.1 Preliminary Investigation . . . . . . . . . . . . . . . . . . . . . 11210.2 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11310.3 STM Implementation . . . . . . . . . . . . . . . . . . . . . . . . 11510.4 Roslyn Extension . . . . . . . . . . . . . . . . . . . . . . . . . . 11510.5 C# 6.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

11 Future Work 11811.1 Performance Test . . . . . . . . . . . . . . . . . . . . . . . . . . 11811.2 Integration into CLR . . . . . . . . . . . . . . . . . . . . . . . . 11911.3 Irreversible Actions . . . . . . . . . . . . . . . . . . . . . . . . . 119

Appendix 121

A Roslyn Compiler Call Chain 122

B Concurrency Problems 124B.1 Dining Philosophers . . . . . . . . . . . . . . . . . . . . . . . . 124B.2 The Santa Claus Problem . . . . . . . . . . . . . . . . . . . . . 125B.3 Concurrent Queue . . . . . . . . . . . . . . . . . . . . . . . . . 125B.4 Concurrent HashMap . . . . . . . . . . . . . . . . . . . . . . . 125

C Evaluation Implementations 127C.1 Lock-Based . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127C.2 STM Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142C.3 AC# . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

CONTENTS vi

D Summary 172

List of Acronyms 173

Bibliography 174

1 IntroductionThis chapter describes the motivation behind this project in Section 1.1.Related work is presented in Section 1.2 and the scope of the project isaddressed in Section 1.3. The hypothesis and problem statement questions arepresented in Section 1.4. Finally, the project evaluation method is stated inSection 1.5.

1.1 Motivation

Today the increase in CPU speed comes in the form of additional cores, asopposed to faster clock speed[1]. In order to utilize this increase in speed, manyof the popular sequential programming languages, such as C, C++, Java, andC#, require changes or new additions in order to adapt[2, p. 56]. Our recentstudy[3], analyzed the runtime performance and characteristics of three di↵erentapproaches to concurrent programming: Threads & Locks (TL), STM, andthe Actor model. The study concluded that STM eliminates many of the risksrelated to programming in the context of shared-memory concurrency, notablydeadlocks, leading to simpler reasoning and improved usability. Additionally,STM fits with the thread model used by sequential languages, and can therebybe applied to existing implementations, without requiring major rewrites. Theruntime performance of STM is at a competitive level comparable to that offine-grained locking[3]. The analysis uncovered one major caveat of STM, it isnot orthogonal with side-e↵ects, such as Input/Output (IO) and exceptions[3].

STM has, as of the time of writing, seen only limited o�cial language integra-tion, despite of the advantages STM could provide to sequential languages.Toour knowledge STM has only been introduced as an o�cial built-in languagefeature in Clojure1 and Haskell2. STM has been introduced as a library basedsolution in sequential languages. However to supply features such as static anal-ysis and syntax support require language integration. From the programmerspoint of view, a single feature rarely justifies adopting a new programminglanguage. Especially considering that adopting a new language may requirerewriting existing code. Thus integrating STM into existing languages willbenefit the programmers of these languages.

As of April 2014 and February 2015 Microsoft open sourced the new C#compiler[4] codenamed Roslyn and the Core Common Language Runtime(CLR) [5] respectively. Microsoft thereby facilitates an opportunity to extendthe C# language and compiler as well as its runtime system, thereby openingup for integrating STM into the language.

1http://clojuredocs.org/2https://wiki.haskell.org/Haskell

1

http://clojuredocs.org/

https://wiki.haskell.org/Haskell

CHAPTER 1. INTRODUCTION 2

1.2 Related Work

This section describes related work within the area of STM. In order to learnfrom the approaches taken by others as well as achieving a better understandingof the subject, a number of papers, articles, and other research material ofrelevance has been read. The focus of this section has been on identifyingdi↵erent strategies for language integration of STM as well as di↵erent STMimplementation strategies.

1.2.1 Composable Memory Transactions

In [6] Harris et al. describe their work with integrating STM into Haskell.Haskell is extended with support for an atomic function which takes an STMaction as input and produces an IO action as output[6, p. 51]. Evaluatingthe IO action executes the transaction defined by the atomic function. TheHaskell setting allows the authors to divide the world into STM actions andIO actions[6, p. 51], e↵ectively disallowing IO actions within transactions aswell as only allowing STM actions to be performed inside transactions. TheSTM system is implemented as a C library integrated into the Haskell runtimesystem. The Haskell constructs utilize this library to execute transactions[6,p. 56]. Furthermore, the authors provide a description and implementationof STM constructs for conditional synchronization. The retry statementallows a transaction to block until some condition is met at which point thetransaction is aborted and re-executed[6, p. 52]. The orElse can be used incombination with the retry statement to specify transactional alternatives tobe executed in case the previous alternatives encounter a retry[6, p. 52].

1.2.2 STM for Dynamic-sized Data Structures

In [7], Herlihy et al. describe the Dynamic Software Transactional Mem-ory (DSTM) system. DSTM is a library based STM system aimed at theC++ and Java programming languages[7][p. 92]. DSTM uses transactionalobjects which encapsulate regular objects and provide STM based access andsynchronization[7][p. 9]. Each transactional object contains a record of itscurrent value, old value and a reference to the transaction which createdthe record[7][p. 95]. A Compare-And-Swap (CAS) operation is employed toatomically update the state of a transactional object[7][p. 96]. DSTM is anobstruction-free[8] STM system. Obstruction-freedom guarantees that anythread which runs long enough without encountering a synchronization conflictmakes progress[8][p. 1]. Unlike stronger progress guarantees such as lock-freedom and wait-freedom, obstruction-freedom does not prevent livelock[9, p.47]. As a result DSTM employs a contention manager to ensure progress inpractice[7][p. 93]. An extended version of DSTM called DSTM2 is presentedin [10]. Here the authors focuses on creating a simple and flexible API for theSTM library.


1.2.3 Language Support for Lightweight Transactions

In [11] the authors describe how they integrated STM into the Java program-ming language by modifying both the compiler[11, p. 4] and virtual machine[11,p. 9]. The authors design, implement and performance test an obstructionfree STM system. The STM system uses a non-blocking implementation, andthus guarantees the absence of deadlocks and priority inversion. Addition-ally, non-conflicting executions are executed concurrently. Per object, it usesan ownership record to track the object’s version number as well as whichtransaction currently owns the object[11, p. 6]. Transaction descriptors areemployed in order to keep track of the read and write operations performedby a transaction. Transactions are committed using CAS in order to ensureatomicity[11, p. 7]. The performance tests show how the STM system scalesalmost as well or better than locking, when the amount of cores available isincreased[11, p. 12]. The test cases were performed on data structures, e.g. aConcurrentHashmap, as opposed to an entire system.

1.2.4 Transactional Locking II

In [12] the TLII STM system designed at Sun Microsystems Laboratories byDice et al. is described. While many of the other STM systems described hereadopt an obstruction-free approach to implementing STM, TLII uses committime locking[12, p. 199]. A transaction explicitly records its read and writeoperations in a read and write set[12, p. 198]. Instead of writing directly tomemory, all writes are written to the write set. When a transaction is about tocommit it acquires the lock on each object in the write set and writes the valuescontained in the write set to the actual memory locations before releasingthe locks[12, p. 200]. This corresponds to a two phase locking scheme[13, p.455]. A global version clock is used to verify that transactions are executedin isolation[12, p. 201]. As a transaction starts it reads the current value ofthe global version clock, storing it locally so it can be used for validation. Asa transaction is about to commit, it validates its read set by comparing thelocally stored read stamp with each object’s associated write stamp[12, p. 200].If any write stamps are higher than the locally stored read stamp, a conflicthas occurred and the transaction must abort and re-execute.

1.2.5 A (Brief) Retrospective on Transactional Memory

Inside Microsoft, a group of architects and researchers led an incubationproject. Joe Du↵y, now director of the Compiler and Language Platform groupat Microsoft, gives a retrospective view on their work in [14]. Their goal wasto provide a language integrated STM system with support in the Just-In-Time Compilation (JIT) compiler, garbage collector, compiler and debugger.Their overall strategy was to use a version number for optimistic reads, and alock for writes. Initially they chose weak atomicity and update in-place, but


realized that this approach su↵ered from privatization issues, breaking theisolation. They settled on a write on-commit approach and chose unboundedtransactions to provide a broader appeal. Furthermore they relied on compileroptimization through static analysis to remove unnecessary barriers as well asfinding violations of the isolation introduced by the programmer. The authorsidentified STM as a systemic and platform wide technology shift, just likegenerics. Having a platform wide change, requires careful integration withexisting language features, in order to preserve the orthogonality. Severalcritical operations, that would cause trouble if permitted inside a transactionsince their actions are non-reversible, were identified. These include allocationof finalize objects, IO calls, GUI operations, P/Invokes to Win32, library callsand the use of locks. Ultimately, this led to the realization that not all problemsare transactional. Very little .NET code, but computations performed solelyin memory, could actually run inside transactions. This combined with theprivatization issue and several minor but continuous arising problems, causedJoe Du↵y to state that the research area of STM was, as of January 2010, notmature enough, and thus STM.NET never made it outside of the incubationproject.

1.3 Scope

STM has been an active area of research for almost 20 years[15]. While theresearch has come far from the initial proposal of statically sized memorytransactions, the area still has unsolved problems, including issues with sidee↵ects, IO, and exceptions occurring inside transactions[16]. While moreresearch into solving these known problems as well as the creation of new STMalgorithms with good performance is of interest to the research community,it is not the focus of this master thesis. Instead the focus is evaluating theintegration of STM in the C# programming language. Specifically C# 5.0,the most recent version at the time of writing[17].

The Roslyn compiler project contains compilers for both Visual Basic andC#[4]. As this master thesis focuses on C# we will restrict any investigationinto the Roslyn compiler to focus on the C# compiler as well as sharedfunctionality of interest. We realize that the Roslyn compiler contains featuresfor the unreleased C# 6.0, but as these features are uncompleted and not final,they will not be accounted for in the integration.

As of February 2015 Microsoft released the source code for an independentversion of its .NET runtime environment Common Language Runtime, calledthe Core CLR on Github[5]. While this undoubtedly presents many researchopportunities for the area of STM as well as other computer science areas,the CLR is considered out of scope for this thesis. The CLR Core is mainlywritten in C++[5], a language with which we have only limited experience.


Furthermore, the Core CLR consists of roughly 2.6 million lines of code[18]making it a complex and time demanding task to gain an understanding of itsstructure.

1.4 Problem Statement

The goal of this master thesis is to investigate the usability of languageintegrated support for STM in C#, compared to a library based solutionand existing locking features. To formalize our goal, we have constructed thefollowing hypothesis:

Hypothesis Language integrated STM provides a valid alternative to lockingin terms of usability, and provides additional benefits compared to librarybased STM, when solving concurrency problems in C#.

In order to evaluate the hypothesis, a language integrated STM system forC#, called AC#, and STM library for the .NET platform, will be designedand implemented. Using AC# and the STM library a number of representa-tive concurrent problems will be implemented. These implementations willbe compared to equivalent lock based implementations using the evaluationmethod defined in Section 1.5.

1.4.1 Problem Statement Questions

In order to structure our investigation we have identified a number of problemstatement questions. The questions are based on findings from the theoryinvestigated in our previous study[3], investigation of related work, exploratoryinvestigations into STM implementations, and the Roslyn compiler.

1. What features should an STM system for C# contain?

2. What problems exist in integrating STM in C#?

3. What di↵erent implementation strategies exist for STM?

4. How is the Roslyn compiler structured?

5. How can the Roslyn compiler be utilized to integrate STM into the C#language?

6. How does the characteristics di↵er when using locking, library-basedSTM and language-based STM in the context of C#.


1.5 Evaluation Method

The evaluation is conducted by analyzing characteristics of the di↵erent con-currency approaches in a qualitative manner, based on a number of concurrentimplementations. The characteristics utilized are an extended version of thecharacteristics employed in our prior study[3, p. 15-21]. Each of the character-istics highlight key di↵erences in the usability of the concurrency approaches.Combined, they form a comprehensive, although not exhaustive, view of theusability of the concurrency approaches. By evaluating both library-basedSTM and language based STM their di↵erences will be highlighted which canserve to justify language integration of STM.

1.5.1 Selected Problems

Four concurrency problems have been selected for evaluation:

1. The Dining Philosophers (Appendix B.1)

2. The Santa Claus Problem (Appendix B.2)

3. A Concurrent Queue (Appendix B.3)

4. A Concurrent Hashmap (Appendix B.4)

The Dining Philosophers problem represents a well known concurrency problemwhich highlights some of the pitfalls associated with synchronization of threads.The Santa Claus problem encompasses a high degree of modeling and requirescomplex synchronization, e.g. allowing only a predefined number of threadsto enter a critical region at a time and waiting on one of multiple conditions.Employing this problem helps investigate what advantages STM providescompared to locking in such scenarios. Both the concurrent queue and hashmaprepresent real world problems. Concurrent queues are widely used in concurrentapplications[19] and concurrent hashmaps can for example be used in a compilerto maintain a symbol table[20]. Both data structures also benefit from finegrained synchronization and is available in a number of languages, includingC#. Together these problems provide a varied perspective by exerting di↵erentaspects of each approach e.g. waiting on one of multiple conditions and finegrained synchronization.

The source code for each of the implementations can be found in Appendix C,together with a description of the chosen strategy. Common for all the solutionsare, that they must solve the problem by using the specified concurrencyapproach and utilize the strengths of the approach.


1.5.2 Evaluation of Characteristics

For each of the selected problems an implementation will be created usinglocking, library based STM and AC#. Based on these implementations eachconcurrency approach will be evaluated according to an extended version of thecharacteristics defined in our previous work[3, p. 15-21]. These characteristicsare a combination of general characteristics for concurrency models suchas pessimistic or optimistic concurrency, as well as characteristics such assimplicity and readability which have been used to evaluate the usability ofprogramming languages[21, p. 7]. The additional characteristics are DataTypes and Syntax Design, which are relevant as the characteristics are now usedto evaluate concrete implementations in a language as opposed to concurrencymodels in our previous work[3].

Each of these characteristics range from one extreme to another, e.g. highor low readability, however a concurrency approach may not reside at oneof these extremes. Therefore each concurrency approach will be given aplacement on the spectrum of each characteristic, based on the findings ofthe evaluation. In order to visualize this placement a scale similar to the onepresented in Figure 1.1 is employed. Here X and Y represent the two extremesof the spectrum while the indicators represent the placement of each of theconcurrency approaches on the spectrum. As an example X and Y could below and high writability, Figure 1.1 then shows that each of the concurrencyapproaches resides more towards the high writability end of the spectrum.As the evaluation of the characteristics is subjective, the placement of eachconcurrency approach on a spectrum allows for a more clear comparison ofthe findings in the evaluation.

YX

Library STM

AC#

Locking

Figure 1.1: Example of characteristic evaluation scale

2 Background KnowledgeThis chapter contains background knowledge required to understand the re-maining chapters. In Section 2.1 the locking constructs in C# are described,establishing the term “locking” in respect to our hypothesis. Furthermore,the key concepts of STM will be explained in Section 2.2, enabling the readerto understand the details of the design and implementation, discussed inChapter 5 and Chapter 6 respectively.

2.1 Locking in C#

Locking in C# can in the simplest cases be done with the lock state-ment. For more specialized cases the Monitor class can be used. Fur-thermore, a number of other special case locking constructs can be found inthe System.Threading namespace e.g. [22] Mutex , SemaphoreSlim ,SpinLock and ReaderWriterLockSlim. This is not an exhaustive list,but encompasses the constructs used to solve the concurrency problems definedin Section 1.5 as well as some of the more specialized constructs.

2.1.1 Lock Statement

The lock statement[17, p. 102] provides a way to acquire and release a lock ona resource. In the scope following the lock statement, a lock is automaticallyacquired at the beginning and released at the end. The lock provides mutual-exclusion, resulting in threads trying to acquire an already acquired lock,blocking until the lock is released. The lock statement ensures that theprogrammer does not forget to release the lock. Listing 2.1 exemplifies theusage of the lock statement.

Listing 2.1: Lock Statement

1 protected object thisLock = new object();2 public int Get() {3 lock(thisLock){4 if (!_queue.IsEmpty()){5 return _queue.Dequeue();6 }else{7 return default(int);8 }9 }

10 }

8

CHAPTER 2. BACKGROUND KNOWLEDGE 9

2.1.2 Monitor

The lock statement described in Section 2.1.1 is an assisting way of usingthe Monitor class[23]. The Monitor cannot be instantiated as an object,but can be used in any context through its static methods. The methodsEnter and Exit are used to acquire and release a lock on a resource. TheMonitor class also provides functionality such as allowing the programmerto specify a timeout on the acquisition of a lock. This is done by using theTryEnter method. The Monitor also facilitates communication betweenthreads with the Wait, Pulse, and PulseAll methods. A thread can callthe Wait method to release the lock its holding, thus making it possible forother threads to change the state of the resource. When the other thread isdone, it can use Pulse to notify the next waiting thread of changes, and itwill then try to reacquire the lock. PulseAll notifies all the waiting threads.

2.1.3 Mutex

The Mutex class[24] provides mutual exclusion to a shared resource by allowingonly a single thread to acquire the Mutex at a time. The Mutex ensuresthread identity, guaranteeing that only the thread which acquired the Mutexcan release it. A Mutex can be acquired by the method WaitOne, whichrequests ownership of the Mutex and blocks the calling thread until the Mutexis acquired or a supplied timeout is met. When a thread is done using theMutex, it can use ReleaseMutex to release it.

2.1.4 SemaphoreSlim

The SemaphoreSlim class[25] is similar to a Mutex , but allows multiplethreads to enter a shared resource at a time. The amount of threads allowedto enter the semaphore at a given time is set as a constructor argument.Contrary to Mutex, the SemaphoreSlim does not assure thread identityon the WaitOne or Release methods. Additionally, there is no guaranteedorder in which the waiting threads will enter the SemaphoreSlim.

2.1.5 SpinLock

The SpinLock struct[26] is similar to the Monitor class, and has the methodsEnter, TryEnter, and Exit which are called on an instance of the struct. Itis implemented as a struct, easing the pressure on the Garbage Collection (GC)but requiring the programmer to pass it by ref. The primary purpose is to allowa thread to spin wait, instead of blocking causing a context switch to occur.This is useful in scenarios where locks are fine grained and in large numbers,or when the holding time of the locks is consistently short. SpinLock isnot reentrant, consequently if the same thread tries to take the lock twice, anexception will be thrown.


2.1.6 ReaderWriterLockSlim

The class ReaderWriterLockSlim[27] is a lock with multiple states allow-ing threads to di↵erentiate between reading or writing to a shared resource.This enables concurrent reads while keeping writes exclusive, and thereforesafe. The methods EnterReadLock, EnterUpgradeableReadLock, andEnterWriteLock enter reading, upgrade and write mode respectively. Theread mode is only for reading the shared resource, enabling concurrency withother readers, but mutual exclusion to writers. The upgradeable readlockenables a common pattern where a value is read, and only updated under cer-tain conditions. If a thread acquires a ReaderWriterLockSlim in upgrademode and determines that an update to the protected resource is required, thelock can be upgraded to a write lock without releasing the update lock, thusdisallowing other threads from intervening when changing the lock to writemode. The write lock provides mutual exclusion, ensuring exclusive access tothe shared resource. All of these methods are also available in a version whichallows a timeout to be specified.

2.2 STM Key Concepts

This part contains a modified version of [3, p. 43-48]. The reader can skip thissection if she is familiar with STM.

2.2.1 Software Transactional Memory

STM provides programmers with a software based transactional model througha library or compiler interface[28]. It seeks to solve the issues introduced byrunning multiple processes concurrently in a shared memory space, e.g. raceconditions as discussed in [3, p. 22-26]. To handle these challenges, STM o↵ersthe programmer ways to define transaction scopes over critical regions. Thetransactions are executed concurrently and if successful, changes are committed.If a transaction is not successful it will re-executed. Just by defining criticalregions, STM ensures atomicity and isolation, and as a result the low leveldetails of synchronization are abstracted away from the programmer[6, p. 48].Therefore, STM provides a more declarative approach to handle shared-memoryconcurrency than by using locks. A number of issues related locking discussedin [3, p. 26-30], such as deadlocks, do not exist in STM.

2.2.2 Example of using STM

Languages supporting STM must encompass a language construct for specifyingthat a section of code should be executed as a transaction and managed by theSTM system. This basic language construct is often referred to as the atomicblock[6, p. 49][11, p. 3]. The atomic block allows programmers to specify atransaction scope wherein code should be executed atomically and isolated, as


Listing 2.2: Threadsafe queue

1 public int Get() {2 atomic {3 if (!_queue.IsEmpty()){4 return _queue.Dequeue();5 }else{6 return default(int);7 }8 }9 }

exemplified in Listing 2.2. Exactly how a transaction scope is defined variesbetween STM implementations. As an example, an STM integrated in thelanguage could look like Listing 2.2 on line 2, while a library based systemsuch as JDASTM[29] uses calls to the methods startTransaction andcommitTransaction.

2.2.3 Conflicts

By declaring an atomic block, the programmer delegates the responsibility ofsynchronizing concurrent code to the STM system. Avoiding race conditionsand deadlocks, while still allowing for optimistic execution introduces conflictsbetween transactions. In the context of STM a conflict is two transactionsperform conflicting operations on the same data, resulting in only one of thembeing able to continue[9, p. 20]. A conflict arises if one transaction reads thedata while the other writes to it. Di↵erent techniques of conflict resolution arediscussed in [3, p. 45-46 & 52-55]. Despite the di↵erent implementation details,the semantics does not change from the point of view of the programmer.However, it is important to know that transactions may conflict, since a highlevel of contention can negatively a↵ect performance[3, p. 52].

2.2.4 Retry

By enabling the programmer to interact further with the STM system beyonddeclaring atomic blocks, busy-waiting can be avoided. A common task in con-current programming is executing code whenever some event occurs. Considera concurrent queue shared between multiple threads in a producer consumersetup. It is desirable to only have a consumer dequeue an item whenever oneis available. Accomplishing this without the need for busy waiting frees thecomputational resources for other tasks.

In [6] Harris et al. introduce the retry statement for assisting in conditionalsynchronization within the context of STM. The retry statement is explicitlyplaced within an atomic block. If a transaction encounters a retry statement


during its execution it indicates that the transaction is not yet ready to runand the transaction should be aborted and retried at some later point[9, p.73]. The transaction is not retried immediately but instead blocks, waiting tobe awoken when one of the variables read in the transaction is updated byanother transaction[6, p. 51]. By blocking the thread instead of repeatedlychecking the condition, busy waiting is avoided.

A transaction using the retry statement is shown in Listing 2.3. If the queueis empty the transaction executes the retry statement of line 4, blocking thetransaction until it is retired at a later time.

Listing 2.3: Queue with retry

1 public int Get() {2 atomic {3 if (_queue.IsEmpty()) {4 retry;5 }6 return _queue.Dequeue();7 }8 }

2.2.5 orElse

In addition to the retry statement Harris et al. propose the orElse block.The orElse block handles the case of waiting on one of many conditions to betrue by combining a number of transaction alternatives. These alternatives areevaluated in left-to-right order and only one of the alternatives is committed[6,p. 52]. The orElse block works in conjunction with the retry statement todetermine which alternative to execute. An example of a transaction employingthe orElse block is shown in Listing 2.4. If an alternative executes withoutencountering a retry statement it gets to commit and the other alternativesare never executed[9, p. 74]. However, if an alternative encounters a retrystatement its memory operations are undone and the next alternative in thechain is executed[9, p. 74]. If the last alternative encounters a retry statement,the transaction as a whole is blocked awaiting a retry at a later time[9, p. 74].

Listing 2.4: Queue with orElse

1 public int Get() {2 atomic {3 if(_queue.IsEmpty())4 retry;5 return _queue.Dequeue();6 } orElse {7 if(_queue2.IsEmpty())8 retry;9 return _queue2.Dequeue();

10 } orElse {


11 if(_queue3.IsEmpty())12 retry;13 return _queue3.Dequeue();14 }15 }

3 RoslynThis chapter describes the Microsoft Roslyn project, hereafter referred to asRoslyn. Little literature on Roslyn exists. The main source is [30], which isa whitepaper from Microsoft presenting an overview of Roslyn. However thewhitepaper’s main focus is on the API side of Roslyn, described in Section 3.1,and not the internal details of compilation, which are required in order tointegrate STM into the Roslyn C# compiler. Beyond the whitepaper, blog andforum posts have been used, but most of these also only describe the API side.The rest of the knowledge described in this chapter is obtained by inspectingand debugging the source code.

The purpose of this chapter is to obtain the knowledge required to modifythe C# compiler with STM support, in order to build AC#. As a result thischapter does not cover the Visual Basic (VB) aspects of Roslyn. The extensionof the Roslyn compiler is described in Chapter 7.

In Section 3.1 an introduction to the Roslyn project is given. In Section 3.2 thearchitecture of Roslyn is described. Following this, Section 3.3 describes theinternal details of the compiler phases, information which is valuable in orderto select how and where to extend the compiler with STM support. Finally inSection 3.4, a detailed description of the syntax trees of the Roslyn compiler isgiven.

3.1 Introduction

Project Roslyn is Microsoft’s initiative of completely rewriting the C# andVB compilers, using their respective managed language. Roslyn was releasedas open source at the Microsoft Build Conference 2014[31].

Beyond changing the languages the compilers are written in, Roslyn providesa new approach to compiler interaction and usage. Traditionally a compiler istreated as a black box which receives source code as input and produces objectfiles or assemblies as output[30, p. 3]. During compilation the compiler buildsa deep knowledge of the code, which in traditional compilers is unavailable tothe programmer and discarded once the compilation is done. This is whereRoslyn di↵ers, as it exposes the code analysis of the compiler by providing anAPI, which allows the programmer to obtain information about the di↵erentcompilation phases[30, p. 3].

The compiler APIs available are illustrated in Figure 3.1 where each APIcorresponds to a phase in the compiler pipeline. In the first phase the sourcecode is turned into tokens and parsed according to the language’s grammar.

14

CHAPTER 3. ROSLYN 15

This phase is exposed through an API as a syntax tree. In the second phasedeclarations, i.e. namespaces and types from code and imported metadata,are analyzed to form named symbols. This phase is exposed as a hierarchicalsymbol table. In the third phase identifiers in the code are matched to symbols.This phase is exposed as a model which contains the result of the semanticanalysis. This model is referred to as a semantic model and exposes methodsthat answer semantics questions related to the syntax tree for which it iscreated[30]. Through the semantic model programmers can obtain informationsuch as:

• The type of an expression

• The symbol corresponding to a declaration

• The target of a method invocation

In the last phase, information gathered throughout compilation is used to emitan assembly. This phase is exposed as an API that can be used to produceCommon Intermediate Language (CIL) bytecode[30, p. 3-4].

Figure 3.1: Compiler pipeline in contrast to compiler APIs[30, p. 4].

Knowledge obtained through the APIs is valuable in order to create tools thatanalyze and transform C# or VB code. Furthermore Roslyn allows interactiveuse of the languages using a Read-Eval-Print Loop (REPL)[32], and embeddingof C# and VB in a Domain Specific Language (DSL)[30, p. 3].

3.2 Roslyn Architecture

The Roslyn solution available on github1, forked2 on the 9th February 2015,consists of 118 projects which include projects for Visual Studio development,interactive usage of the languages and more as illustrated on Figure 3.2. The

1https://github.com/dotnet/roslyn2https://github.com/Felorati/roslyn

https://github.com/dotnet/roslyn

https://github.com/Felorati/roslyn


Figure 3.2: Overview of projects in Roslyn solution.

Compilers folder contains the source code for the C# and VB compiler,each located in a separate folder. They share common code and functionalitylocated within the Core folder, including code for controlling the overallcompilation flow. Both compilers use the same patterns for compilation[33,09:36-10:36].

Figure 3.3: Overview of CSharp folder.

The projects contained in the CSharp folder are shown on Figure 3.3.The csc project is the C# command line compiler, which is the startingpoint of a C# compilation. The CSharpCodeAnalysis.Portable andCSharpCodeAnalysis.Desktop projects contain the C# code analysis,which is the actual code required for compilation. The rest of the projectsin the CSharp folder mainly involve tests for the C# compiler. The Corefolder has a structure similar to that of the CSharp folder, encompassing aCodeAnalysis.Portable and CodeAnalysis.Desktop project whichcontain the common core analysis code.

For more information about the architecture, an overview of the RoslynCompilers call chain can be found in Appendix A.


3.3 Compiler Phases

The C# compiler builds upon concepts from traditional compiler theory,such as lexing, parsing, declaration processing, semantic analysis and codegeneration[21][34]. Throughout the phases of compilation, traditional conceptssuch as syntax trees, symbol tables and the visitor pattern[35, p. 366] are alsoused. This section elaborates on the compiler phases in the compiler pipelineshown in Figure 3.1.

Figure 3.4: Overview of the CSharp.CSharpCodeAnalysis.Portableproject.

3.3.1 Initial Phase

The initial phase of compilation entails initial work, such as parsing thecommand line arguments and setting up for compilation, described in furtherdetail in Appendix A. This phase is executed sequentially[36].

3.3.2 First Phase

The first phase involves parsing the source code, which is done in a traditionalcompiler fashion by lexing source code into tokens and parsing them into asyntax tree, which represents the syntactic structure of the source code. Thelexer is implemented using a switch which identifies the type of token to lex,given the first character of the token string. The parser is implemented as atop-down parser using the common recursive descent approach. The parsingphase will check for syntax errors in source code, but does not have enoughinformation to check for semantic errors, such as scope or type errors. Thephase is concurrent, as several files may be parsed simultaneously[36]. The


code for this phase is mainly located within the Parser and Symbols folder.The syntax tree and its contents are described in more detail in Section 3.4.

3.3.3 Second Phase

The second phase involves creating a Compilation type object, specifically aCSharpCompilation object. A Compilation object contains informationnecessary for further compilation e.g. all assembly references, compiler optionsand source code. In the creation of a CSharpCompilation object, a declara-tion table is created which keeps track of type and namespace declarations insource code[36]. This is done sequentially[36]. Additionally the Compilationobject contains a symbol table, which holds all symbols declared in sourcecode or imported assemblies. Each namespace, type, method, property, field,event, parameter and local variable is represented as a symbol and stored inthe symbol table[30, p. 14]. Each type of symbol has its own symbol class, e.g.MethodSymbol, which derives from the base Symbol class. The symbol tablecan be accessed by the GlobalNamespace property, as a tree of symbols,rooted by the global namespace symbol. Furthermore a range of methods andproperties to obtain symbols also exists. The code for this phase is mainlylocated within the Declarations and Symbols folders.

3.3.4 Third Phase

In order to enable semantic analysis, the third phase entails fully binding allsymbols, which determines what each symbol actually refers to, e.g. whatnamespace and overloaded method, a particular method refers to. Any problemswith symbol binding, like inheritance loops, will be reported. Binding is doneconcurrently, however binding members of a type will force base types to bebound also. For symbols not related, they can be bound in any order [36].

Additionally, the binding phase also creates a bound tree, which is the Roslyncompilers internal tree used for flow analysis and emitting. In [37] Anthony D.Green states that they do not want to expose the bound tree through the APIas:

“It has been a long standing design decision not to expose thebound tree. The shape of the bound nodes is actually pretty fragilecompared to the shape of the syntax nodes and can be changedaround a lot depending on what scenarios need to be addressed. Wemight store something somewhere one day for performance andremove it the next for compatibility or vice versa”

– Anthony D. Green[37]


Data flow and control flow analysis uses the bound tree to for instance checkif statements are reachable, as the C# specification states that an unreachablestatement should produce a warning[38]. The code for this phase is mainlylocated in the Binder, BoundTree, and FlowAnalysis folders.

3.3.5 Final Phase

Finally, in the fourth and final phase all information built up so far is emittedas an assembly. Method bodies are compiled and emitted as CIL concur-rently. However methods within the same type are compiled sequentially ina fixed order, typically lexical. The final emitting to the assembly is donesequentially[36]. The code for this phase is mainly located in the CodeGenand Emitter folders.

3.4 Syntax Trees

Syntax trees are the primary structure used throughout compilation. Syntaxtrees in the Roslyn compiler have three key attributes[30, p. 6]:

1. A syntax tree is a full fidelity representation of source code, which meansthat everything in the source code is represented as a node in the syntaxtree. If programs are invalid, the syntax tree represents these errors insource code by tokens named skipped or missing in the tree.

2. A syntax tree produced from parsing must be able to be translated backto the original source code. This is referred to as being roundtripable.

3. Syntax trees are immutable and thread-safe. This enables multiple usersto use the same syntax tree in di↵erent threads without concurrencyissues. As syntax trees are immutable, factory methods exist in order tohelp create and modify trees. Upon a modification, the factory methodsdoes not copy the entire tree along with the modification, instead theunderlying nodes are reused. As a result, trees can be modified fast andwith a low memory overhead.

3.4.1 Red And Green Trees

The Roslyn team wanted a primary data structure for compilation with thefollowing characteristics[39]:

• Immutable.

• Form of a tree.

• Cheap access to parent nodes from child nodes.


• The ability to map from a node in the tree to a character o↵set in thesource code.

• The ability to reuse most nodes in the original tree when modifying trees.

However fitting all those characteristics into a single data structure is problematic[39]:

• One problem is simply constructing a tree node, because both the childand parent are immutable and must have a reference to each other, so itis not possible to create one before the other.

• Another problem is reusing nodes for other parents when modifying thetree, as nodes are immutable and it is therefore not possible to changethe parent of a node.

• A third problem is inserting a new character into the source code, asthe position in source code of all nodes changes after that point. Thisis makes it problematic to adhere to the characteristic of reusing mostnodes when modifying trees, because a modification to source code canchange the character o↵set of many nodes.

Instead the Roslyn compiler uses two types of trees, green trees and red trees,in order to fulfill all their required characteristics.

The green tree is immutable, has the ability upon modification to reusemost una↵ected nodes, has no parent references, is built bottom-up, anddoes not know the absolute positions of nodes in the source code, only theirwidths[39].

For the expression “5*5+5*5”, a typical parse tree is shown in Figure 3.5, anda potential green tree is shown in Figure 3.6. As the green tree nodes do nothave parent references and positions in source code, sub-trees and nodes canbe reused, which results in a more compact tree. Factory methods are usedto create new nodes in the tree in order to determine if existing nodes can bereused or new ones must be created. If nodes for a given expression alreadyexist they are reused, otherwise new nodes are created.

However reuse of existing nodes is not guaranteed, as that requires all nodesto be cached, which according to Vladimir Sadov from Microsoft in [36] makesthe reuse caches unnecessarily big. The caches are instead a fixed size, wherenew nodes will replace older ones when the maximum size is reached. Sadovstates that it works pretty well because recently accessed nodes are likely tobe accessed in the near future. Another trade-o↵ is that they do not reusenon-terminals with more than 3 children, as it gets more expensive and lesslikely to match, the more children a non-terminal has[36].


5*5 5*5+

5*5 + 5*5

5*5 5*5

Figure 3.5: Typical parse tree of expression.

5*5 5*5+

5*5

+

*

5

Figure 3.6: Green tree of expression, reusing identical sub-trees. Inspired by[36].


The red tree is an immutable facade built around the green tree. It hasparent references and knows the absolute positions of nodes in the source code.However these features prevent nodes from being reused, which means thatmaking modifications to a red tree is expensive. Therefore another approachthan building a new red tree upon every modification[39] is used.

The red tree is built lazily using a top-down approach, starting from the root ofthe tree and descending into children. Once the parent of a node is available, allinformation required to construct a red node is available. The internal data forthe node can be obtained from the corresponding green node. Furthermore theabsolute position of the node in source code can be computed, as the positionof the parent is known, along with the source code width of all children thatcome before the given node[36].

So when modifications are made to the source code, an entire new red tree isnot computed. Instead the green tree is modified, which is a relatively cheapoperation, because most nodes can be reused. In terms of the red tree, a newred node is created as root with 0 as position, null as parent and the rootgreen node as the corresponding green node. The red tree will then only builditself if a user descends into its children and it might only descend into a smallfraction of all the nodes in the tree[36].

3.4.2 Contents Of Syntax Trees

The elements contained within syntax trees are syntax nodes, tokens and trivia.In this section these constructs and some of their properties are described.Additionally the supporting constructs Spans, Kinds and Errors are described.

3.4.2.1 Syntax Nodes

Syntax nodes represent non-terminals of the language grammar, such as dec-larations, statements and expressions. Each syntax node type is representedas a separate class that derives from the base SyntaxNode class. As syntaxnodes are non-terminals, they always have children, either in the form of othersyntax nodes or syntax tokens.

In relation to navigating syntax trees, all syntax nodes has[30, p. 7]:

• A parent property to obtain the parent node

• For each child, a child property to obtain the child

• A ChildNodes method to return a list of all child nodes

• Descendant methods i.e. DescendantNodes, DescendantTokensand DescendantTrivia , to obtain a list of all descendant nodes,tokens or trivia for a given node.


Additionally, optional children are allowed, which are represented as null ifthey are not present. For example an IfStatementSyntax syntax node hasan optional ElseClauseSyntax syntax node[30, p. 7].

3.4.2.2 Syntax Tokens

Syntax tokens represent terminals of the language grammar, such as keywords,literals and identifiers. As opposed to syntax nodes, which do not have anychildren.

Di↵erent types of syntax tokens do not have a separate class, instead all syntaxtokens are represented by a single SyntaxToken type. This means that thereis a single structure for all tokens. To get the value of a token, three propertiesexist: Text, ValueText and Value. The first returns the raw source text asa String, including extra characters such as escape characters. The secondreturns only the value of the token as a String. The last returns the valueas the actual value type e.g. if the token is an integer literal then the propertyreturns the actual integer. To allow di↵erent return types, the return type ofthe last property is Object[30, p. 7-8].

Additionally, for performance reasons the SyntaxToken type is defined as astruct[30, p. 7].

3.4.2.3 Syntax Trivia

Syntax trivia represent parts of source code that can appear between anytwo tokens, such as whitespace and comments. Syntax trivia is not includedas a child node in the tree, but instead associated with a given token. Asyntax token holds all the following trivia on the same line, up to the nexttoken. Syntax tokens hold trivia in two collections: LeadingTrivia andTrailingTrivia. The first token holds all leading initial trivia, and theend-of-file token holds the last trailing trivia in source code[30, p. 8].

As trivia are not nodes in the tree, they do not have a Parent property.Instead the associated token for some trivia, can be accessed with the Tokenproperty. Additionally, like syntax tokens, trivia are also structs and have onlya single SyntaxTrivia type to describe them all.

3.4.2.4 Spans

Every node, token, and trivia knows its position in source code. This isaccomplished by the use of a TextSpan struct type. A TextSpan objectholds the start position of a node, token, or trivia in source code and a countof characters, both represented as 32-bit integers[30, p. 8].


Every node, token and trivia has two properties to obtain spans: Span andFullSpan. The Span property includes only the span of the node, token ortrivia and not any trivia, where the FullSpan property includes the normalspan and any leading or trailing trivia.

3.4.2.5 Kinds

Every node, token and trivia has an integer RawKind property, used to identifythe syntax element type. Each language, C# or VB, contains a SyntaxKindenumeration that contains all the nodes, tokens and trivia in the language. TheRawKind property corresponds to an item in the SyntaxKind enumerationfor the specific language. The CSharpSyntaxKind method gets and auto-matically cast the bscodeRawKind to an item in the CSharpSyntaxKindenumeration[40][30, p. 9].

Kinds are especially important for tokens and trivia, as they have only a singletype, SyntaxToken and SyntaxTrivia. Thus, the only way to identifythe particular token or trivia at hand, is by identifying its associated kind.

3.4.2.6 Errors

If programs are invalid as a result of errors in source code, a syntax tree is stillproduced. These errors are represented as special tokens in the syntax tree,which are added using one of the following techniques[30, p. 9].

1. Insert a missing token in the syntax tree when the parser scans for aparticular token but does not find it. The missing token represents theexpected token, but it has an empty span and has a true IsMissingproperty.

2. Skip tokens until the parser finds a token from where it can continuethe parse. The skipped tokens are added as a trivia node with theSkippedTokens kind.

3.4.3 Syntax Tree Generation

The nodes, associated factory methods and visitor pattern for the syntaxtrees are generated based on the contents of the Syntax.xml file located inthe CSharpCodeAnalysis (Portable) project. The contents of the filedescribes information such as the fields and base class for each node in thetree. Whenever the compiler is built, a tool which source code resides in theTools\CSharpSyntaxGenerator project is run. It generates the classesfor each node defined in Syntax.xml along with relevant factory methods,properties for getting the value of each field associated with a node, and factorymethods for generating an updated version of the node. The tool also generates


the visitor pattern implementation along with the required accept methodson each node. Both the red and green tree, described in Section 3.4.1, aregenerated during this process.

4 Requirements for AC#This chapter describes the requirements for the STM system powering AC#.The requirements are based on known STM constructs and the characteristics,described in Section 1.5, which the final system must be evaluated upon.The requirements will be used to design and integrate the STM features intoC#, described in Chapter 5. Furthermore, the requirements will be used toimplement the STM system, described in Chapter 6.

Tracking granularity is described in Section 4.1 followed by a description of therelationship between transactions and variables i.e. what and how variablesshould be tracked, is addressed in Section 4.2. Section 4.3 describes the choicebetween strong and weak atomicity. The di↵erent types of side-e↵ects andhow they are handled is discussed in Section 4.4. Conditional synchronizationis described in Section 4.5 and nesting in Section 4.6. Finally Section 4.7describes opacity before a summary of the requirement is given in Section 4.8.

4.1 Tracking Granularity

A variable tracked by the STM system can be tracked with di↵erent granularity:by tracking assignments to the variable, or tracking changes to the objectreferenced by the variable. These di↵erent approaches a↵ect the semantics ofthe STM system and will be discussed in the subsequent sections.

4.1.1 Tracking of Variables

Tracking changes to the variable directly limits the e↵ect of the STM systemto only variables, and not internal changes inside of the referenced object.This is the approach used in the STM system in Clojure[41]. It o↵ers a simplemental model for the programmer, as only changes visible in the transactionscope will be provided with the transactional guarantees of atomicity andisolation. Listing 4.1 shows an example, where the field car is assigned to anew modified car object on line 11. This assignment is tracked by the STMsystem, as opposed to the example in Listing 4.2, where the internals of theobject are not tracked, when a method with a side-e↵ect is called on line 12.The discussion of side-e↵ects is expanded upon in Section 4.4.

This approach can be used in combination with both user defined code and codein contained in binaries, but changes to their internals will not be tracked, andtherefore the reusability is limited. Should the programmer want transactionalsupport for the side-e↵ects, the internals must be written to use the STMsystem.

26

CHAPTER 4. REQUIREMENTS FOR AC# 27

Listing 4.1: Tracking Assignment to Variables

1 sealed class Car {2 private readonly int _kmDriven;3 public Car(int km) { _kmDriven = km; }4 public Car Drive (int km) {5 return new Car(_kmDriven + km);6 }7 }8 ...9 atomic {

10 _car = _car.drive(25);11 }

4.1.2 Tracking of Objects

Tracking of objects allows the STM system to track the internal changes to thefields of an object. This allows the STM system to reverse the modification ofan object e.g. when an item is added to a collection inside a transaction andthe transaction is aborted, the collection can be restored to the state presentbefore the item was added.

Tracking changes to the internals of an object can be done at the level of only asingle object or up to the entire object structure, e.g. the objects referenced bythe object etc. In [7] Herlihy et al. present a library based STM called DSTM.DSTM uses an approach where the programmer explicitly has to implement aninterface, which allows the STM system to create a shallow copy of an object.The STM system returns a copy of a variable’s value to which the transactioncan apply its changes. However as the copy is shallow, the internals canreference objects shared with the original. Side-e↵ects on such an object willalso a↵ect the original, causing a potentially unintentional breach of isolation.If the programmer want a deeper tracking, she must design the internals usingthe STM system. The example in Listing 4.2 demonstrates a side-e↵ect onthe object contained in the car variable. Because the kmDriven fieldis assigned a new value, the changes are detected by the STM system. Thechange to engine is however not detected as the Car object is not changedwhen the Engine is degraded.

In [11], Harris et al. present an STM system which tracks changes throughoutthe entire object structure. Changes to objects are bu↵ered in a log and writtento the original if the transaction commits. This deep traceability is enabledby having a part of the STM system in the runtime system, as the entirestructure is known at that level, even if an object is from a compiled library.This approach ensures isolation, but requires modifications to the runtimesystem. In the example of Listing 4.2, both kmDriven and engine will betracked by STM system presented in [11].


Listing 4.2: Tracking Changes to Object

1 public class Car {2 private int _kmDriven;3 private Engine _engine;4 public void Drive (int km) {5 _kmDriven += km;6 _engine.Degrade(km);7 }8 }9 ...

10 atomic {11 _car.Drive(25);12 }

4.1.3 Choice of Tracking Granularity

Deep tracking of objects requires changes to the runtime system which is outof scope for this thesis as described in Section 1.3. This leaves the option ofeither tracking changes to individual objects, but not the objects internals, oronly tracking assignments to variables. Providing support for changes to onlyindividual objects may seem inconsistent from the programmer’s point of view,due to changes to referenced objects not being tracked. Therefore trackingof assignments to variables is selected, since it provides a consistent, simplemental model for the programmer, displaying exactly what is tracked by theSTM system. The consequence of this choice is that the side-e↵ects cannot betracked automatically, thus potentially burdening the programmer.

4.2 Transactions & Variables

As described in Section 2.2.2 an STM system must o↵er some way of defininga transaction scope. As AC# is a language integrated STM system, C# mustbe extended with syntax for specifying a transaction scope.

An STM system abstracts away many details of how synchronization is achieved.Simply applying transactions over a number of C# variables provides a highlevel of abstraction but also hides the impact of synchronization. Explicitlymarking what variables are to be synchronized can assist the programmer ingauging the performance of a transaction as well as improving the understandingof the execution of transactions, both areas which usability studies[42][43] havefound to be problematic for some programmers. C# must be extended withsyntax for marking variables for synchronization inside transactions. A variablemarked for use in transactions is referred to as a transactional variable. Atransactional variable must function similarly to a volatile variable. Thatis, the language must treat a transactional variable as any other variable of the


same type when it is utilized. Thus a transactional variable can be passed asan argument into a method with a parameter that is not tracked. Just like avolatile variable, a transactional variable must be treated di↵erently thannormal variables by the compiler. In terms of usability, these di↵erences musthowever largely be unnoticeable to the programmer.

Applicability is an important aspect when evaluating the usability of AC#, incomparison to equivalent lock-based and library-based implementations. AC#must not be overly restrictive as this will limit its applicability. Due to this, theSTM system must allow reads from transactional variables to occur both insideand outside transactions. For writes to transactional variables a choice existsbetween allowing and disallowing writes from outside transactions. Disallowingwrites from outside transactions will ensure that non-transactional accesscannot interfere with transactional access, but it will hamper the usabilityof the STM system. Allowing writes from outside transactions increases thecomplexity of the implementation, as any conflicts such writes create mustbe detected and resolved by the STM system. Since allowing writes fromoutside transactions provides the best usability, AC# must provide support forsuch writes in addition to writes from inside transactions. This requirement isclosely related to the choice between strong and weak atomicity discussed inSection 4.3.

4.3 Strong or Weak Atomicity

The atomicity guarantee provided by STM systems varies, depending on thesemantics provided. In [44] Blundell et al. define two levels of atomicity:

Definition 4.1. [...] strong atomicity to be a semantics in which transactionsexecute atomically with respect to both other transactions and non-transactionalcode.

Definition 4.2. [...] weak atomicity to be a semantics in which transactionsare atomic only with respect to other transactions.

Strong atomicity provides non-interference and isolation between transactionaland non-transactional code, whereas weak atomicity does not. An example ofthe di↵erences between strong and weak atomicity is presented in Listing 4.3.Using the Car class defined in Listing 4.3, having the KmDriven setter calledfrom one thread, while another thread is calling the Drive method, strongand weak atomicity yields di↵erent results. Under strong atomicity, all changesmade inside the atomic block on line 12 are isolated from non-transactionalcode. Additionally, changes made from the setter are isolated from inside theatomic block. The result is that if the setter is called in the middle of theDrive method, a conflict will occur which.


If only weak atomicity is guaranteed, given the same scenario, the change madethrough the setter would be visible inside the atomic block. Thus accessingthe same variables from transaction and non-transactional code can lead torace conditions.

Listing 4.3: Level of Atomicity

1 public class Car {2 private int _kmDriven;3 public int KmDriven {4 get {5 return _kmDriven;6 }7 set {8 _kmDriven = value;9 }

10 }11 public void Drive (int km) {12 atomic {13 _kmDriven += km;14 }15 }16 }

4.3.1 Issues with Atomicity Levels

In [9, p. 30-35] Harris et al. summarizes a collection of issues related to thedi↵erent levels of atomicity. The collection is non-exhaustive, but based on awide selection of research. The consequence of race conditions can be either:

• Non-repeatable read - if a transaction cannot repeat reading the value ofa variable due to changes from non-transactional code in between thereadings.

• Intermediate lost update - if a write occurs in the middle of a read-modify-write series done by a transaction, the non-transactional writewill be lost, as it comes after the transaction has read the value.

• Intermediate dirty read - if eager updating[3, p. 53] is used, a non-transactional read can see an intermediate value written by a transaction.This transaction might be aborted, leaving the non-transactional codewith a dirty read.

The second case is exactly the case described in Listing 4.3, where weakatomicity led to the risk of race conditions between transactional and non-transactional code.


Another issue with weak atomicity is known as the privatization problem. Ifonly one thread can access a variable, the need for tracking it through the STMsystem ceases, and so does the associated overhead. It is therefore desirable, toprivatize a previously shared variable when doing intensive work that does notneed to be shared across threads. A technique used for privatizing a variable,x, is to use another variable as a shared marker priv, which indicates whetheror not the x is private. This is demonstrated in Listing 4.4. Intuitively onewould believe, that if Thread1 wants to privatize x, it can set priv to truein a transaction after which Thread1 has private access to x. This is howeverfalse, since Thread2 could read priv and assign to x, after which Thread1executes, setting the values of priv and x, causing the transaction executedby Thread2 to abort and rollback. During the rollback the value of x isrestored to the value it had when Thread2 wrote to it, causing Thread1’swrite to x, on line 5, outside of the transaction to be overwritten, and lost.This example assumes weak atomicity, commit-time conflict detection, and inplace updating[9, p. 34].

Listing 4.4: Privatization Problem

1// Thread 12 atomic {3 priv = true;4 }5 x = 100;6 // Thread 27 atomic {8 if (!priv) {9 x = 200;

10 }11 }

4.3.2 Choice of Atomicity Level

All the issues listed above are related to weak atomicity, and are not presentunder strong atomicity. Despite of the advantage of strong atomicity, itsshortcomings must be considered before choosing. The overhead of guarantee-ing atomicity between transactional and non-transactional code can occur aconsiderable cost[45]. In [45] Spear et al. propose four contracts, for whichprivatization may be guaranteed. Strong atomicity is ranked as the least re-strictive, but comes with a considerable cost. Although the performance is notoptimal, Hindman and Grossman show in [46] that strong atomicity with goodperformance is achievable by source-to-source compiling with optimizationsthrough static analysis.

As described in Section 1.4, the goal of this project is to validate whetherSTM is a valid alternative to locks and provides additional benefits comparedto library-based STM in terms of usability. Therefore strong atomicity in


combination with marked transactional variables is chosen for AC# as itprovides the best usability.

4.4 Side-e↵ects

Side-e↵ects in methods are a common idiom in C#, and come in di↵erentshapes and form. Here side-e↵ects are categorized as in-memory side e↵ects,exceptions or irreversible actions. This section discusses the requirements forhandling the di↵erent types of side-e↵ects in AC#.

4.4.1 In-memory Side-e↵ects

Side-e↵ects in memory is done by modifying state through references to vari-ables outside of the method scope. An example is Listing 4.2 where the Drivemethod updates the field kmDriven and invokes a method on engine ,potentially causing another side-e↵ect. As discussed in Section 4.1, AC# onlytrack assignments to variables, and not changes to the internals of objects.As a consequence, a side-e↵ect such as the one in Listing 4.2 will persistthrough an aborted transaction. To remedy this, classes must be implementedto track their internals or be immutable. Listing 4.1 shows an immutableimplementation of the Car class. This design avoids side-e↵ects, and changesto the object will return a new object, which will be tracked if assigned to atransactional variable.

The immutable approach suits STM well, as it is free of side-e↵ects. Addi-tionally it is a less error prone and secure design approach, than mutableobjects[47, p. 73]. Microsoft has an o�cial immutable collection package1, andis therefore giving first class support for immutability. Furthermore Microsoftrecommends the use of immutability for “[...] small classes or structs that justencapsulate a set of values (data) and have little or no behaviors”2. Guidelinesfor designing immutable objects can be found in Bloch’s E↵ective Java[47, p.73-80].

4.4.2 Exceptions

Di↵erent approaches exists for handling exceptions raised inside a transac-tion. AC# applies the programmers intuition of how exceptions work innon-transactional code, to transactional code. Therefore, transactions willnot be used as a recovery mechanism as proposed by Tim Harris et al. in[16]. Instead the exception will be propagated if, and only if, the transactionis able to commit at the point where the exception is raised. Otherwise thetransactions will be aborted and re-executed. This way, the programmer will

1https://www.nuget.org/packages/Microsoft.Bcl.Immutable2https://msdn.microsoft.com/en-us/library/bb384054.aspx

https://www.nuget.org/packages/Microsoft.Bcl.Immutable

https://msdn.microsoft.com/en-us/library/bb384054.aspx


only receive exceptions from code that actually takes e↵ect, and will be ableto recover by catching any exceptions, similar to non-transactional code.

4.4.3 Irreversible Actions

E↵ects such as IO performed on disk or network, native calls, or GUI operationsare not reversible. This makes them unsuitable for use in transactions, sincetheir e↵ect cannot be undone, should the transaction be aborted. In [14],Du↵y proposes using a well known strategy from transaction theory[48], havingthe programmer supply on-commit and on-rollback actions to perform orcompensate for the irreversible action. In [16], Harris et al. propose that IOlibraries should implement an interface, allowing any IO e↵ects to be bu↵ereduntil the transactions commit at which point the IO library is given a callback.These solutions either burden the programmer using STM, or the librarydesigner that must implement a special interface.

While the proposed solutions show potential, solving the issue of irreversibleactions in transactions is out of scope for this thesis as described in Section 1.3.Due to this, no guarantees are given on the e↵ect of using irreversible actionsin transactions, and it is thus discouraged.

4.5 Conditional Synchronization

To be a valid alternative to locking C#, an STM system must be applicableto the same use cases as locking. This requires support for conditional syn-chronization so that STM can be employed in well known scenarios such asshared bu↵ers and other producer consumer setups[13, p. 128]. Section 2.2discusses the retry and orElse constructs proposed in [6] for conditionalsynchronization and composition of alternatives. Supporting such constructsin C# will increase the applicability of the STM system.

Our previous work in [3] includes an implementation of the k-means clusteringalgorithm[49, p. 451] in the functional programming language Clojure. Clojurecontains a language integrated STM implementation which does not supportconstructs such as retry and orElse . As a result the implementationrequires the use of condition variables and busy waiting in scenarios where theretry construct could have been employed[14]. Supplying retry and orElseconstructs in C# will allow for simpler conditional synchronization withoutthe need for busy waiting, thereby increasing the simplicity and writability insuch scenarios.

A disadvantage of providing the retry and orElse constructs is reduced sim-plicity. However, as the retry and orElse constructs are optional, the e↵ectsof this disadvantage are reduced. Therefore the conditional synchronizationconstructs are included in AC#.


4.6 Nesting

The traditional lock-based approach to concurrency has issues with compos-ability due to the threat of deadlocks when composing lock based code[2, p.58]. STM attempts to mitigate these issues by removing the risk of dead-locks, by allowing transactions to nest. Nesting can occur both lexically anddynamically[50, p. 1][9, p. 42][28, p. 2081].

An STM system for C# must support nesting of transactions as this will allowthe system to mitigate one of the major caveats associated with lock basedconcurrency. A more in depth description of the composability problems ofthe lock-based concurrency model and nesting of transactions can be found inour prior work [3].

Di↵erent semantics exist for nesting of transactions. These are: 1. Flat, 2. Openand 3. Closed[50, p. 1][9, p. 42]. Flat nesting treats any nested transactions aspart of the already executing transaction, meaning that an abort of the nestedtransaction also aborts the enclosing transaction. Closed nested semanticsallows nested transactions to abort independently of the enclosing transaction.Under closed nested semantics, commits by nested transactions only propagateany changes to the enclosing transaction, as opposed to the entire system. Opennesting allows nested transactions to commit even if the enclosing transactionaborts and propagates changes made by nested transactions to the entiresystem whenever a nested transaction commits.

Flat nesting is the simplest to implement, but closed and especially open nestingallows for higher degrees of concurrency[9, p. 43]. Considering the simplicity,readability and level of abstraction provided by the di↵erent strategies, as wellas the degree of concurrency o↵ered, closed nesting is selected for AC# . Inorder to improve the orthogonality AC# is required to support both lexicaland dynamic nesting.

4.7 Opacity

Opacity is a correctness criteria requiring transactions to only read consistentdata throughout their execution[51, p. 1][9, p. 29]. This means that trans-actions must not read data which will cause them to abort at a later time.Consequently opacity requires that the value read is consistent when the readoccurs, but allows the variable to be changed at some later point by anothertransaction. Transactions must be aborted when reads cannot be guaranteedto be consistent.

By providing opacity, programmers do not have to reason about problemsthat occur as a result of inconsistent reads[9, p. 28], thereby simplifying theprogramming model. As an example, consider Listing 4.5


Listing 4.5: Opacity example

1 using System.Threading;2

3 public class Opacity4 {5 private atomic static int X = 10;6 private atomic static int Y = 10;7

8 public static void Main(string[] args)9 {

10 var t1 = new Thread(() =>11 {12 atomic13 {14 X = 20;15 Y = 20;16 }17 });18

19 var t2 = new Thread(() =>20 {21 atomic22 {23 var tmpx = X;24 var tmpy = Y;25 while (tmpx != tmpy)26 {27 }28 }29 });30

31 t1.Start();32 t2.Start();33 }34 }

where the two transactional variables X and Y are defined on lines 5 and 6as well as the two threads t1 and t2 are defined on lines 10 and 19. t1simply sets the value of X and Y as a transaction. t2 enters a transaction inwhich it reads the values of X and Y entering a loop if the values are not equal.Consider the interleaving shown in Figure 4.1. The transaction executed by t2reads the value 10 associated with the variable X after which t1’s transactionupdates the value of both X and Y to 20. t2 reads the value 20 associatedwith Y. In an STM system providing opacity, this would not be allowed sincethe transaction would read inconsistent data. If the STM system does notprovide opacity t2 will enter an infinite loop as tmpx and tmpy are not equal.

As opacity bolster the simplicity of using STM, it is required for AC#. Thiswill positively impact the readability and writability.


t1 t2

X = 20Time

tmpx = X

Y = 20

tmpy = Y

while(tmpx != tmpy)

Figure 4.1: Opacity interleaving example

4.8 Summary of Requirements

In the following section, the requirements will be summarized in order to givea clear overview of properties AC# must have.

The granularity of tracking AC# will provide, is on the variable level. Thatis, the assignments to variables will be tracked, however side-e↵ects to thereferenced object will not. Tracking the internals of an object will require thefields of the object to be marked as transactional. A transaction scope canbe defined by a syntax extension provided in AC#. Transactions in AC# areexecuted under the guarantee of strong atomicity, providing isolation betweentransactional and non-transactional code. As the STM system does not trackside-e↵ects, use of immutable objects is promoted. Exceptions occurring insidea transaction will be propagated out of the transaction if the transaction isable to commit. Thus transactions will only be used for synchronization, notrecovery of state. All irreversible actions, as mentioned in Section 4.4.3 arediscouraged. AC# must facilitate conditional synchronization, by supplyingthe retry and orElse constructs. Nesting is allowed in AC# under closednesting semantics, which strikes a balance between minimizing aborts andensuring simple semantics for nested transactions. Lastly, opacity will berequired for AC#, as this correctness criteria will bolster the simplicity of theSTM system.

5 Design and IntegrationThis chapter describes the considerations and design decisions of AC#, theintegration with existing language features, as well as AC#’s syntax & se-mantics. The design adheres to the requirements defined in Chapter 4, whileintegrating with existing language features. The decisions made in this chapterwill influence the implementation of the STM library described in Chapter 6,and the extension of the Roslyn C# compiler described in Chapter 7.

The atomic block and transactional variables are introduced in Section 5.1.The di↵erent kinds of parameters and their integration with transactions isdescribed in Section 5.2. This is followed by an example of the new syntax,demonstrated in Section 5.3. The retry and orelse keywords for supportingconditional synchronization are introduced in Section 5.4. Section 5.5 showshow AC# supports nesting of transactions. Finally, a summary of the designrequirements described in this chapter is given in Section 5.6.

Throughout this chapter, the proposed syntax extensions is described usingExtended Backus-Naur Form. The notation uses ? for signaling 0 or 1occurrences and + for signaling 0 or more occurrences. Parentheses are usedfor grouping: ( item1, item2 ). Items formatted as terminal are terminals,while items formatted as non� term are non-terminals. The syntax extensionspresented are based on the syntax used in C# Precisely[52]. This is chosen dueto its compact and readable format. As the examples serve to illustrate howthe new constructs integrate with the existing language features, some elementsof the syntax will not be explained in depth. A reference to where a detaileddescription can be found will be provided for each example. The complete C#grammar used for parsing can be found at the C# specification[17].

5.1 Transactional Blocks & Variables

As described in Section 4.2 AC# must supply language integrated support fordefining transaction scopes as well as declaration of transactional variables.For this purpose AC# extends C# with the atomic keyword. The atomickeyword will serve as both a statement denoting a transaction scope as well asa modifier applicable to the declaration of variables.

5.1.1 Transactional Blocks

The declaration of a transaction takes the format atomic { S }, where S isthe general class of C# statements. Any assignments to transactional variablesinside an atomic block will be perceived as executed in one atomic step by

37

CHAPTER 5. DESIGN AND INTEGRATION 38

other transactions and non-transactional code. Return statements inside anatomic block will cause a return from the contained method, just like returnsin if and try blocks.

5.1.2 Transactional Fields

The declaration of a transactional field takes the format:

field-modifiers type name = initializer ( , name = initializer )+ ;field-modifiers type name ( , name )+ ;

where field-modifiers includes the new atomic keyword in addition to theexisting modifiers: static, new, and access-modifiers[52, p. 36]. The field-modifiers readonly and const are not included, since their unchangeablenature makes tracking them in an STM system pointless. volatile is alsoleft out, as the STM system ensures safety in concurrent contexts.

The type can be any existing or user-defined type including dynamic. The cal-culation of initializer follows the same rules as standard C# field initializers[52,p. 40].

Listing 5.1 presents an example of atomic field declarations.

Listing 5.1: Local Transactional Variable

1 private atomic string s = "abc";2 public atomic static int x, y, z;

5.1.3 Transactional Properties

C# facilitates a language feature which eases the encapsulation of fieldstypically used in Object Oriented Programming (OOP). The feature is knownas properties, and can appear in an automatic or a manual form[52, p. 52].Only the automatic form of properties can be declared as transactional. Theformat of a transactional property is:

atomic method-modifiers type name { access-modifier? get;access-modifier? set; }

method-modifiers include the: static, new, virtual, override,sealed, and abstract keywords. This is the only change to properties inC#, and the normal rules of property usage still applies, as defined in [52, p.52].

The manual property lets the programmer specify a backing field manually,allowing him to specialize the functionality of the getter and setter. As a


consequence, specifying an atomic backing field enables the programmer toimplement a transactional property with a specialized getter and setter. Theautomatic properties generate a backing field automatically, but the getterand setter cannot have specialized functionality.

Atomic automatic properties simplifies the implementation whenever the prop-erty serves as a simple getter and setter. An example of such a property isshown in Listing 5.2. If the programmer wishes to supply additional logicfor the get and set operations the manual property can be made atomic byaccessing an atomic backing field as demonstrated in Listing 5.3. Both casesare frequently used in C#, thus supplying an atomic version makes thefeature usable in transactions in an orthogonal manner.

Listing 5.2: Automatic Transactional Properties

1 class Car {2 public atomic int KmDriven { get; set; }3 }

Listing 5.3: Manual Transactional Property

1 class Car {2 private atomic int _kmDriven;3 public int KmDriven {4 get {5 return _kmDriven;6 }7 set {8 _kmDriven = value;9 // Announce value changes

10 }11 }12 }

No restrictions are made on transactional properties marked virtual. Conse-quently the programmer can override a virtual atomic property supplyinga non-atomic implementation. Similarly the programmer is allowed to over-ride a non-atomic, virtual property with an atomic auto implementedproperty of the same type. This approach is flexible when supplying transac-tional implementations of existing non-transactional base classes and allowsnon transactional implementations based on a transactional base class, raisingthe orthogonality of AC#.

5.1.4 Transactional Local Variables

The declaration of transactional local variables follows the format:


atomic var name = expr;atomic type name ( = expr )? (, name ( = expr )? )+ ;

As with transactional fields, type can be any existing or user defined typeincluding the dynamic keyword. The deceleration of atomic const localvariables are not allowed in AC# as const variables cannot be the target ofassignments. Listing 5.4 depicts the declaration of two atomic local variablein AC#.

Listing 5.4: Local Transactional Variable

1 atomic var s = "abc";2 atomic int i = 12;

5.2 Transactional Parameters

In C# four di↵erent kinds of parameters exist: Value, Reference, Output, andParameter Array of which atomic Value, Reference and Output parametersare available in AC# . This gives the programmer the flexibility to trackparameters without requiring additional ceremonial code. The semantics,syntax, and integration of the three parameter types in AC# will be discussedin the following sections. Why atomic parameter arrays are not supported isdiscussed in Section 5.2.4. Transactional parameters adhere to the standardrules in regards to named parameters[17, p. 145]. Just as with the parametermodifiers ref and out, an overloaded method cannot only di↵er on the atomicmodifier, as it does not change its type. Thus, it follows the rules defined inthe specification[17, p. 153-157].

5.2.1 Transactional Value Parameters

According to [17, p. 97]: “A parameter declared without a ref or out modifieris a value parameter.” If the parameter is a value type, call-by-value semanticsis used, thus assignments to the argument have no e↵ect outside the methodscope. If the parameter is a reference type, it is also call-by-value semantics,however side-e↵ects that change the referenced object, will persist outside themethod scope. Due to the call-by-value semantics, assignments to an argumentof reference type will have no e↵ect outside the method scope[52, p. 76].

In AC#, a value parameter can be marked as atomic, thus becoming a trans-actional value parameter. Semantically a transactional value parameter isequivalent to a transactional local variable which is instantiated to the valueof the parameter[52, p. 76]. However, the transactional parameters improvethe usability by providing an orthogonal approach for tracking assignments tothe parameters in transactions.


For declaration of transactional parameters the format is as follows:

atomic param-modifier? type name (= value� initializer)

where param-modifier represents C#’s ref and out keywords, and typecan be any existing or user defined type including the dynamic keyword,as previously stated. The atomic keyword can be combined with optionalparameters. An optional parameter declared atomic follows the standardrules for optional parameters as defined in[52, p. 46-47].

Listing 5.5 presents an example of an atomic value parameter declaration.

Listing 5.5: Transactional Value Parameter

1 public void Method(atomic int x, atomic string s)2 {3 //Work with transactional parameters4 }

5.2.2 Transactional Reference Parameters

According to [17, p. 97]: “A parameter declared with a ref modifier is areference parameter.” As opposed to a value parameter, a reference parameteruses call-by-reference semantics, therefore it is the actual reference used asargument, not a copy. Consequently, assignments to the parameter will takee↵ect outside the method scope, regardless of the parameter being a value orreference type. Assignments to the parameter take e↵ect immediately, both inand outside of the method scope[52, p. 42]. Additionally, a reference parameteris required to be a variable which has been definitely assigned before it ispassed as a reference argument[17, p. 97]. Definitely assigned, meaning thatthe variable is sure to have been given a value at the point where it is used[17,p. 96], in this case passed as a reference parameter.

In AC#, a reference parameter can be marked as atomic, and become aTransactional Reference Parameter (TRP). A TRP di↵ers from a referenceparameter by being tracked in transactions. Assignments to a TRP do nottake e↵ect outside of the transaction immediately, but when the transactioncommits. This is a design choice made to enforce atomicity, as immediatechanges to a TRP would enable reading intermediate transaction states. Anoteworthy detail is that changes made to a TRP through side-e↵ects will beimmediate, since side-e↵ects are not tracked by the STM system, as discussedin Section 4.1.

Listing 5.6 presents an example of a Point class declaring two TRP’s asparameters for the ChangeMe method.


Listing 5.6: Transactional Reference Parameter

1 class Point {2 public atomic int X { get; set; }3 public atomic int Y { get; set; }4 public void CopyMe(atomic ref int x, atomic ref int y) {5 atomic {6 x = this.X;7 y = this.Y;8 }9 }

10 }

5.2.3 Transactional Output Parameters

According to [17, p. 97]: “A parameter declared with an out modifier is anoutput parameter.” An output parameter behaves as a reference parameter,except that it does not need to be instantiated before being passed as anargument. Additionally, an output parameter must be definitely assignedwhenever the method terminates[52, p. 42]. It is not required that a variableis definitely assigned before being passed as an output parameter, but it isallowed.

In AC#, an output parameter can be marked as atomic, and it becomes aTransactional Output Parameter (TOP). Similar to TRP, a TOP is trackedin transactions, and assignments in transactions take e↵ect outside of thetransaction when the transaction commits. The reason for this is the same asfor TRPs.

Listing 5.7 presents a modified example of the Point class declaring twoTOP’s as parameters for the ChangeMe method.

Listing 5.7: Transactional Output Parameter

1 class Point {2 public atomic int X { get; set; }3 public atomic int Y { get; set; }4 public void CopyMe(atomic out int x, atomic out int y) {5 atomic {6 x = this.X;7 y = this.Y;8 }9 }

10 }

5.2.4 Transactional Parameter Array

According to [17, p. 17]: “A parameter array permits a variable number ofarguments to be passed to a method”. A parameter array is declared using the


paramsmodifier. Inside the method, the arguments are represented as an arrayof length equal to the number of arguments supplied to the params parameter.Assignments to the parameter array are possible but this possibility is in ourexperience rarely used. Additionally, as only assignments to the parameterarray are tracked, adding or removing items from the array, would not betracked in transactions. Thus, the estimated value of adding this feature isclose to none. Therefore, a parameter array cannot be marked as atomic inAC#.

5.2.4.1 Transactional Reference & Output Arguments

Four di↵erent cases exist for passing variables to ref and out parameters inAC#. These are:

1. T ! atomic T

2. atomic T ! T

3. atomic T ! atomic T

4. T ! T

where T is an variable of some type T , atomic T is an atomic parameterof type T and T ! atomic T describes passing a variable of type T asan argument to an atomic parameter of type T . In the first case, a variablereference passed to an atomic reference parameter will result in the variablebeing tracked inside of the method, and assignments to the parameter takinge↵ect outside of the method. In the second case, assignments to the parameterare tracked as if they were assignments to the variable which was passed asref or out. In the third case, the variable passed by ref or out will continueto be tracked inside the method. That is assignments to the parameter aretracked as if they were assignments to the variable passed by bscoderef orout. The fourth case follows the standard rules defined in[17, p. 145].

5.3 Example of AC#

Listing 5.8, which is a modified version of the race condition example from ourprevious study[3, p. 23], presents an example of the syntax extensions AC#brings to C#. On line 6 the transactional field number is defined and assignedan initial value of 10. The main method defines and starts two threads. t1checks if the value of number is equal to 10 and assigns number times 3 to thefield if the condition is true. These operations are executed as a transactiondefined by the atomic keyword and associated block on line 11. t2 assigns20 to number inside a transaction defined on line 17. In this example AC#removes the race conditions associated with t2 changing the value of number


between the read on line 12 and the read and write on line 13. In the case thatsuch a change occurs, the STM system of AC# will abort and re-execute oneof the implicated transactions in order to resolve the conflict.

Listing 5.8: Transaction Syntax

1 using System;2 using System.Threading;3 public class RaceCondition4 {5 public static atomic int number = 10;6 public static void Main(string[] args)7 {8 Thread t1 = new Thread ( () => {9 atomic {

10 if (number == 10 )11 number = number * 3;12 }13 });14 Thread t2 = new Thread( () => {15 atomic {16 number = 20;17 }18 });19 t1.Start(); t2.Start();20 t1.Join(); t2.Join();21 int result;22 atomic {23 result = number;24 }25 Console.WriteLine("Result is: " + result);26 }27 }

As described in Section 4.2 and Section 4.3 AC# must provide strong atomicityas well as allowing reads and writes to occur from outside atomic blocks. As aresult, reads and writes occurring from outside transactions will be accountedfor when validating transactions. With that in mind, the previous examplecan be slightly simplified to the example in Listing 5.9. The atomic block online 17 of Listing 5.8 has been removed, since the atomic block contains onlya single write. The atomic block on line 24 of Listing 5.8 can also be removedas the read can safely be performed from outside a transaction.

Listing 5.9: Transaction Syntax Simplified

1 using System;2 using System.Threading;3 public class RaceConditionSimple4 {5 public static atomic int number = 10;6 public static void Main(string[] args)


7 {8 Thread t1 = new Thread ( () => {9 atomic {

10 if (number == 10)11 number = number * 3;12 }13 });14 Thread t2 = new Thread( () => {15 number = 20;16 });17 t1.Start(); t2.Start();18 t1.Join(); t2.Join();19 Console.WriteLine("Result is: " + number);20 }21 }

Allowing access to transactional variables from outside transactions dispensesthe need for defining a transaction whenever only a single read or write is tobe performed. The atomic block then serves the purpose of combining multipleoperations to be executed as a single atomic step.

5.4 Conditional Synchronization

As described in Section 4.5 AC# supports conditional synchronization via theretry and orElse constructs. AC# extends C# with a retry statementthat can only be used inside atomic blocks. A retry statement takes the format:retry . That is, the keyword is employed as a statement, much like C#’sbreak and continue statements[52, p. 102]. Listing 5.10 presents theDequeue method from a transactional queue. The queue is defined over asingly linked list from which items are dequeued from the front and enqueuedin the back. If Dequeue is called on an empty queue the thread is blockeduntil the queue is no longer empty. The Dequeue method consists of anatomic block, defined on line 11, performing the dequeue operation as a singleatomic step. On line 13 the transaction checks if the queue is empty in whichcase it executes the retry statement on line 14, blocking the transaction to beretried when the size variable changes. If the queue is not empty the headof the queue is removed and the next item in the linked list becomes the newhead. Finally the size of the queue is decreased and the value associated withthe previous head is returned.

Listing 5.10: Retry Syntax

1 using System;2 public class Queue<T>3 {4 private atomic Node _head = null;5 private atomic Node _tail = null;6 private atomic int _size = 0;


7

8 public T Dequeue()9 {

10 atomic{11

12 if (_size == 0)13 retry;14

15 var oldHead = _head;16

17 _head = _head.Next;18 if (_head == null)19 _tail = null;20

21 _size--;22

23 return oldHead.Value;24 }25 }26

27 ...28 }

Furthermore AC# supports an orelse keyword allowing zero to manyorElse blocks to be associated with an atomic block, much like catch clausesare associated with a try statement in many C like languages, includingC#[52, p. 96]. The format of the atomic block is therefore extended to:

atomic { S } ( orelse { S } )+

where S is the general class of C# statements. As an example of the orelseconstruct Listing 5.11 depicts a consumer which extracts an item from one oftwo bu↵ers via the ConsumeItem method. On line 10 the ConsumeItemmethod defines a transaction. If buffer1 is empty the transaction executesthe retry statement on line 12, if not, an item from buffer is returned.If buffer1 is empty and the retry statement is executed, control flows tothe orelse block defined on line 15. The orelse then executes its owntransaction returning an item from buffer2 in case it is not empty. Ifbuffer2 is empty the retry statement on line 17 is executed, resulting inthe entire atomic block blocking until one of the previously read transactionalvariables change, at which point the transaction is restarted.

Listing 5.11: OrElse Syntax

1 using System;2 public class Consumer<T>3 {4 private Buffer<T> _buffer1;5 private Buffer<T> _buffer2;6


7 public T ConsumeItem()8 {9 atomic {

10 if(_buffer1.Count == 0)11 retry;12

13 return _buffer1.Get();14 } orelse {15 if(_buffer2.Count == 0)16 retry;17

18 return _buffer2.Get();19 }20 }21 }

5.5 Nesting

As described in Section 4.6 AC# supports nesting of transactions under closednesting semantics. For this purpose AC# allows lexical nesting of atomicblocks as shown in Listing 5.12, where a small program reads two integersfrom the console. The string arguments are parsed to integers on line 12 to16. The program then defines two transactions, one starting on line 18 andthe other, which is nested inside the first transaction, starting at line 20. Thefirst transaction initiates a nested transaction which sets the transactionalvariables X and Y based on the input. Due to the semantics of closed nestingthese changes only become visible to the outer transaction when the nestedtransaction commits. As a result, the outer transaction uses the updated valueswhen it computes the new value of Z on line 26, but the remaining systemcannot yet see these updated values. When the outer transaction commits, theassignments to X, Y and Z become visible to the rest of the system as a singleatomic step.

Listing 5.12: Lexical Nesting

1 public class LexicalNesting2 {3 private atomic static int X = 0;4 private atomic static int Y = 0;5 private atomic static int Z = 0;6

7 public void Main(string[] args)8 {9 if (args.Length != 2)

10 return;11

12 int tmpx;13 int tmpy;14 if (!int.TryParse(args[0], out tmpx)


15 || !int.TryParse(args[1], out tmpy))16 return;17

18 atomic19 {20 atomic21 {22 X = tmpx;23 Y = tmpy;24 }25

26 Z = X * Y;27 }28 System.Console.WriteLine(Z);29 }30 }

To increase the usability of AC#, specifically its writability and orthogonality,dynamic nesting of transactions is allowed. As an example, consider Listing 5.13where a transaction is defined on line 1. The transaction transfers funds fromaccount1 to account2 by using the withdraw and deposit methods,which are themselves defined using transactions. Because of the semantics ofclosed nesting, the transfer is executed as single atomic step.

Listing 5.13: Dynamically nested transactions

1 atomic{2 var amount = 200;3 account1.withdraw(amount);4 account2.deposit(amount);5 }

5.6 Summary of Design

A transactional block can be declared by using the atomic statement, todenote a scope for a transaction. Within such a transaction, any assignmentsto transactional variables, parameters, fields, or properties, will be perceivedas executed in one atomic step, by other transactions and non-transactionalcode.

AC# allows the use of common C# features within a transactional context,including reference and output parameters. To increase the orthogonality ofAC# the atomic keyword can be used for making a variable, field, property,or parameter as trackable in transactions.

AC# also introduces the keywords retry and orelse, which can be usedfor conditional synchronization. retry will block the executing threaduntil a transactional variable previously read by the transactions is changed.


The orelse keyword associates an orElse block, with an atomic blockcontaining a retry statement. The orElse block will change the initialbehavior of the retry statement in the atomic block. Instead of blockingexecution when encountering a retry statement, the orElse block will beexecuted. If the last orElse block encounters a retry the atomic block asa whole blocks until one of the previously read values is changed by anothertransaction.

Additionally, AC# allows nesting of transactions, both lexically and dynami-cally. Nesting uses closed-nesting semantics, thus inner transactions commitsinto the outer transaction. The inner and outer transactions are not isolatedfrom each other, thus changes made by the inner transaction are visible to theouter transaction. They are however isolated from all other transactions andnon-transactional code.

6 STM ImplementationThis chapter describes the implementation of the STM system that executestransactions in AC#. Various implementation algorithms have been proposedfor STM of which a few were briefly described in Section 1.2 “Related Work”.This chapter covers the considerations which went into selecting an algorithmfor AC# as well as describing the implemented STM library. Specific attentionis payed to the requirements and design choices made in Chapter 4 andChapter 5. Additionally, as the goal of this thesis is to evaluate if languageintegrated STM is a valid alternative to locking in terms of usability, andprovides additional benefits compared to library based STM, building on anexisting algorithm is the logical choice, rather than developing a new.

This chapter describes a number of criteria in Section 6.1 that influence thechoice of STM algorithm. The final selection of STM algorithm is presentedin Section 6.2. The library interface and how the programmer interacts withit is demonstrated in Section 6.3. The internal details of the STM libraryimplementation is described in Section 6.4. Finally, the testing approach usedto test the STM library is described in Section 6.5

6.1 Implementation Criteria

This section describes a number of criteria which have an influence on thechoice of STM algorithm.

6.1.1 Strong Atomicity

As described in Section 4.3 AC# must support strong atomicity which ensuresisolation between transactional and non-transactional code.

Lazy update inherently supports non-transactional reads as the value associatedwith transactional variables is always the most recently committed value[28, p.2084][9, p. 21]. Thus a non-transactional read can never read a non-committedvalue and atomicity is ensured because a single non-transactional read alwaysreads the most recently committed value of a transactional variable. Ensuringatomicity for non-transactional reads is also possible using eager updating, butmay incur some overhead in selecting the correct value to read[7].

Although non-transactional writes to transactional variables cannot conflictwith one another, they must be tracked by the STM system in order to ensurethat any transactions with which they conflict are aborted and retried. In [46]Hindman et al. proposes the concept of mini-transactions. By encapsulating

50

CHAPTER 6. STM IMPLEMENTATION 51

non-transactional access to transactional variables in a transaction, mini-transactions allow the STM system to track their actions. This enablesreading from and writing to transactional variables from non-transactional code,while ensuring strong atomicity. Their approach goes even further, supplyingoptimized non-transactional access which skips much of the unnecessary loggingrequired for normal transactions. As AC# will benefit from using the describedapproach, the algorithm selected should be capable of supporting it.

6.1.2 Conditional Synchronization

The retry construct described in Section 4.5 requires knowledge of whichtransactional variables have been read, up to the point where a retry isrequested[6]. The STM system needs to maintain a set of transactional variableswhich has been read, for all active transactions, so that a transaction canbe blocked until one of these variables change. Such a set is often referredto as a read-set[12][9][53]. STM systems employing lazy conflict detectioncommonly use such a set to record reads for validation when the transaction isabout to commit[12][54]. Choosing an algorithm which already maintains aread-set will limit the overhead of the retry statement as well as simplifythe implementation as the algorithm must not be modified to support theaccumulation of a read-set. Thus selecting an algorithm which inherentlysupports a read-set is a priority.

6.1.3 Library Implementation

As described in Section 1.4, both a language integrated STM system for C#,in the form of AC#, and an STM library for the .NET platform must be build.Developing a single STM library facilitates reuse as it can be used in boththe evaluation and internally in the AC# compiler. The strategy employedmust therefore not entail details which require changes to the runtime system,as seen by a number of STM implementations[54][11]. Furthermore the STMlibrary’s API must be designed with usability in mind, as this is the focus ofthe evaluation. However the acSTM library must simultaneously facilitate usefrom compiler generated code, allowing the extension to the Roslyn compilerdescribed in Chapter 7 to generate code which utilizes the library to executetransactions.

6.1.4 Progress Guarantee

STM algorithms can coarsely be divided into blocking and non-blockingalgorithms[9, p. 47]. Blocking algorithms employ some form of locking inorder to ensure atomicity, while non-blocking do not[53, p. 59]. Non-blockingalgorithms guarantee that the failure of one thread does not keep other threadsfrom making progress[9, p. 47][55, p. 142][53, p. 59]. This makes it impossibleto employ locking, as the failure of some thread T , which is holding a lock,


will keep other threads from acquiring the lock, and thereby from progress-ing. Non-blocking algorithms can further be distinguished by the progressguarantees they provide:

• Wait-freedom is the strongest of these, and guarantees that any threadmakes progress on its own work if it keeps executing[55, p. 124][53, p.59].

• Lock-freedom is less strict as it only guarantees that if a thread T keepsexecuting then some thread (not necessarily T ) makes progress[9, p.47][53, p. 60].

• The least strict progress guarantee is obstruction-freedom[9, p. 47][8][53,p. 61]. As mentioned in Section 1.2, obstruction-freedom guaranteesthat any thread which runs for long enough without encountering asynchronization conflict makes progress[8, p. 1].

Any wait-free algorithm is also lock-free and any lock-free algorithm is alsoobstruction-free, but not vice versa[53, p. 60]. While all of these progressguarantees preclude deadlocks, only wait-freedom and lock-freedom precludelivelocks[9, p. 47]. Obstruction free algorithms have been shown to providee�cient implementations in practice[53, p. 61], especially when paired withtechniques such as exponential backo↵[53, p. 147] and contention managers[9,p. 51].

Whether STM implementations need to be wait-free, lock-free, obstruction-freeor even provide any of these guarantees at all is an ongoing discussion in theresearch community. Arguments have been made for even obstruction-freedombeing too strong a guarantee and that STM can be made faster by employingcontrolled locking[56]. Due to the complexities associated with non-blockingalgorithms[57][53, p. 61] and the evidence that blocking STM systems canperform well in practice[12], the initial focus is on blocking STM algorithms.The option of moving to a non-blocking algorithm after producing an initialimplementation, as demonstrated by Fernandes et al. in [58] will howeverbe kept open. Due to the limited applicability of C#’s volatile keyword[17,p. 302] ensuring that the most recently written value is always read withoutresorting to locking can be problematic for the non-supported types. Choosinga locking implementation mitigates this issue.

6.2 Selection of Algorithm

By investigating a number of di↵erent STM implementations, the field on whichthe final selection is based, was narrowed down to the following: McRT[59]developed by Saha et al., TLII[12] developed by Dice et al. and JVSTM[60]


developed by Cachopo. McRT and TLII are frequently referenced in the re-search community and JVSTM is a newer implementation. All of these systemsemploy locking but in di↵erent ways. The algorithms vary greatly in otherareas and will provide a broad perspective on possible STM implementations.

6.2.1 McRT

McRT employs encounter time locking and supports two algorithms forsynchronization[59, p. 189]. The first is based on reader-writer locks thatallows multiple readers to hold the lock simultaneously, while a writer musthave exclusive access[59, p. 189]. The second is based on versioned locks whereeach lock has an associated version number[9, p. 108]. The focus is only onthe second approach as this has been shown to have the best performance[59,p. 190].

McRT uses eager updates as this avoids having to check a write-set for anon-committed version of an item, when a read from that item occurs[9, p.109]. Locking allows the system to guarantee that no transaction reads a valuewhich has not yet been committed[9, p. 108]. Eager updates require the systemto maintain an undo list which keeps track of the old values of transactionalvariables to which a given transaction has written. The undo list is used torevert any items, to which a given transaction has written, to their previousstate in case of an abort[59, p. 189]. Eager update makes the commit phasefaster as no writes have to be redone[59, p. 190]. The versioned lock algorithmof McRT requires maintaining a read-set which must be validated as part ofthe commit process. The read-set contains a record of every item read as wellas the version number of the item at the time of reading. During the commitphase the read-set of a transaction is validated, ensuring that no read itemhas a higher version number than the version number present when the itemwas read, e↵ectively ensuring that no writes have been made to these itemswhile the transaction exectued[59, p. 190].

As McRT uses eager updating, conflicts will arise before transactions areallowed to commit. Therefore McRT detects conflicts only against activetransactions[59, p. 189]. Aborting transactions due to a conflict with an activetransaction can lead to cases where e.g. a transaction t1 is aborted due to aconflict with a transaction t2, but t2 is aborted before it can commit dueto a conflict with some transaction t3 [28, p. 2084]. t1 could then havebeen allowed to continue as the transaction with which it conflicted nevercommitted.

Conflicts between transactions appear as contention on the locks in the system.If multiple transactions attempt to acquire a lock on the same item a conflictis about to occur. In such cases McRT prefers letting transactions wait forthe lock over aborting one of the involved transactions, in order to increasethroughput[59, p. 189].


6.2.2 TLII

As briefly described in Section 1.2, TLII only holds locks during the commitprocess[12, p. 199]. This is known as commit time locking. TLII uses lazyupdates where writes are recorded in a write-set and written to the associatedmemory when the transaction commits. Additionally, any reads performedby a transaction are recorded in a read-set used to validate the transactionbefore it commits the content of its write-set[12, p. 198]. The accumulation ofa read-set for the retry will therefore incur no additional overhead.

TLII uses a global version clock to ensure atomicity[12, p. 201]. Transactionsrecord a time-stamp when they start. The time-stamp is used to validate thecontents of the read-set before the transaction commits. Each item trackedby the STM system has an associated time-stamp which is updated when atransaction commits a new value. If any item in the read-set has a time-stamphigher than the transactions time-stamp then the item has been modified whilethe transaction was executing. As a result the transaction must abort andrestart.

In order to commit, a transaction must go through the following steps[12, p.200]:

1. Acquire the lock on each item in the write-set

2. Increment and fetch the version clock

3. Validate the read-set as described above

4. Commit values in the write-set along with the new time-stamps

5. Release the lock on each item in the write-set

The transaction has exclusive access to the items in the write set duringthe commit phase and detects conflicts lazily against previously committedtransactions.

The version clock can be a source of contention as all transactions mustincrement the version clock as part of the commit operation[9, p. 120]. Anumber of ways of reducing contention has been proposed [61][62][63].

The base TLII algorithm can lead to false negatives when validating the read-set. If a transaction t2 executes and commits in between t1 reading theglobal clock and reading its first value, t2 will have incremented the versionclock and the time-stamp of t1 will be invalid before it starts executing itstransaction, even though the transaction would be valid in this scenario. In[64] Riegel et al. proposes attempting to update a transactions time-stamp inorder to eliminates such false negatives.


6.2.3 JVSTM

JVSTM uses Multiversion Concurrency Control (MVCC)[58, p. 1], as knownfrom databases[65, p. 791]. MVCC requires the STM system to keep a recordof the old values for any item that is tracked[65, p. 791]. Reads that would beinvalid in a system maintaining only the most recent value for each item, canbe redirected to read an older value, allowing the associated transaction tocontinue[65, p. 791]. As a result JVSTM guarantees that read only transactionswill always be able to commit[60, p. 97].

To keep track of the values associated with an item, JVSTM uses what theoriginal designer calls a versioned box[60, p. 63]. A versioned box is anabstraction for an item to be tracked, such as a variable. Each versioned boxkeeps an ordered sequence of values, called the history of the versioned box.This sequence represents the values that have been committed to the item.Each transaction and version in the history of a versioned box is assigned aversion number indicating when the transaction started and when the valuewas created respectively.

JVSTM uses lazy updates, bu↵ering writes in a transactional local write-set[60, p. 64]. As with TLII, writes are written to the tracked item when thetransaction commits. Reads from a versioned box first check if the transactionswrite-set contains a non-committed value, returning it if that is the case[60,p. 64]. If no such item exists, the history of the versioned box is searched fora suitable value. The value associated with the highest number which is lessthan or equal to the transactions version number is returned[60, p. 64].

The initial version of JVSTM uses a single global lock to ensure atomicitybetween committing transactions[60, p. 70], forcing the commit phase to beexecuted sequentially even though two transactions may not be committingto the same versioned boxes. A lock-free variation of JVSTM, correcting thisissue, is proposed in [58].

Keeping the old values for each tracked item takes up additional space and thespace consumption increases as new versions are added. Over time some oldvalues might no longer be needed and can therefore be removed. To handle thisJVSTM implements a data structure tracking for which number transactionsare active and what values where commit for a given transaction number.When a transaction commits it uses the data structure determine whetherit has caused any values to be unreachable, cleaning them up if that is thecase[60, p. 70][60, p. 88]. This is similar to a garbage collector[34, p. 472].


6.2.4 Final Selection

Based on the criteria presented in Section 6.1, the requirements defined inChapter 4 and the intended design presented in Chapter 5 it has been decidedto base the STM implementation on the TLII algorithm.

TLII uses lazy updating which simplifies the implementation of strong atomicityas a non-transactional read can access a value directly. Additionally thealgorithm makes use of a read-set which simplifies the implementation ofthe retry construct. An extension to TLII presented in [9, p. 107] allowssupport for opacity, which fulfills the requirement presented in Section 4.7.Furthermore the TLII algorithm is well documented from multiple sources,including a library based implementation[12][53, p. 438][9, p. 106].

McRT was discarded as its eager updating strategy based on locking mayrequire non-transactional reads to wait. If an active transaction t1 haswritten a value to some variable x and a non-transactional read to x occurs,the non-transactional read would have to wait for t1 to finish in order toensure that the value read is not rolled back at a later point. Additionallyconflict detection against active transactions can lead to unnecessary aborts asdescribed in Section 6.2.1.

JVSTM’s use of the well known MVCC approach to transactions facilitates aguarantee that read-only transactions will always be able to commit. Whilethis is an attractive property, MVCC also requires the implementation andoverhead of garbage collection old values that can no longer be read, therebycomplicating the implementation. Additionally, the lock based version ofJVSTM uses commit phases which execute sequentially due to global locking.As a result the scalability of the JVSTM algorithm is reduced[60, p. 86].

Performance of the STM system is not the primary concern of this thesis, asdescribed in Section 1.3. However, performance is not neglected completelywhen choosing an algorithm, as it is an important factor in the context ofconcurrency. Therefore an algorithm which fulfills the requirements with onlyminimal performance sacrifices is preferable.

6.3 Library Interface

The STM library is implemented as a C# library, as described in Section 6.1.3.The library interface has been influenced by the design of other STM librariessuch as Multiverse1 and Shielded2 as well as the introduction to the implemen-tation of STM libraries found in [53].

1https://github.com/pveentjer/Multiverse2https://github.com/jbakic/Shielded

https://github.com/pveentjer/Multiverse

https://github.com/jbakic/Shielded


6.3.1 Atomic Blocks

Transactions are started by using one of the overloads of the static Atomicmethod on the STMSystem class. The static method Atomic is overloadedwith support for transactions with and without return values as well as theassociation of zero to many orElse blocks. Listing 6.1 presents an exampleof transaction declarations. On lines 5-8 a simple transaction declarationcorresponding to an atomic block is shown. Lines 10-14 depict the declarationof a transaction with a return value. Finally lines 16-23 depict the declarationof a transaction with an associated orelse block.

Listing 6.1: Library Transaction Declarations

1 public class TransactionExamples {2

3 public static void AtomicExample4 {5 STMSystem.Atomic(() =>6 {7 //Transaction body8 });9

10 var result = STMSystem.Atomic<int>(() =>11 {12 //Transaction body13 return 1;14 });15

16 STMSystem.Atomic(() =>17 {18 //Transaction body19 },20 () =>21 {22 //orelse body23 });24 }25 }

6.3.2 Transactional Variables

Transactional variables are created by declaring instances of the generic TMVarclass. Instances of the TMVar class wrap an object to which transactionalaccess is provided. The constructor of the TMVar class takes a type parameterspecifying the type which the TMVar instance wraps.

The TMVar class exposes methods and a Value property for getting andsetting the wrapped object. Any access through the exposed methods andthe Value property are tracked by the STM system, both transactional andnon-transactional. Furthermore the TMVar class uses C#’s implicit conversion


feature to allow a TMVar to implicitly be converted to the wrapped valuewhenever the TMVar occurs as an r-value. Special types, deriving from TMVar,are supplied for the common types int, long, float, double, uint,and ulong. These types supply transactional support for common operationssuch as ++ and -- which are executed on the wrapped object. Listing 6.2shows the declaration of a number of transactional variables. Lines 5-7 use thegeneric TMVar class to wrap objects of various types. Line 8 declares a TMInt, which is the specialized type for transactional integers. Lines 11-17 declares atransaction that on line 13 checks if the boolean value wrapped by tmBool istrue and the string value wrapped by tmString is equal to the string "abc".tmBool and tmString are implicitly converted to the wrapped value whilepreserving transactional access, allowing the transactional variables to be usedas if it was an instances of the wrapped type. Line 15 uses the ++ operatorsupported by the TMInt type.

Listing 6.2: Library Transactional Variable


3 public static void TMVarExample()4 {5 TMVar<string> tmString = new TMVar<string>("abc");6 TMVar<bool> tmBool = new TMVar<bool>();7 TMVar<Person> tmPerson = new TMVar<Person>(new Person("Bo

Hansen", 57));8 TMInt tmInt = new TMInt(12);9

10

11 STMSystem.Atomic(() =>12 {13 if (tmBool && tmString == "abc")14 {15 tmInt++;16 }17 });18 }19 }

6.3.3 Retry

The library supports the retry construct via the static Retry method onthe STMSystem class. The retry method can be called from both in andoutside transactions but will only have an e↵ect when called inside transactions.Listing 6.3 presents an example of how the Retry method can be used. Lines5-11 show a transaction dequeuing an item from a shared bu↵er. On line 8 theRetry method is used to initiate a retry in case the queue is empty. The callon line 15 has no e↵ect as it is not enclosed in a transaction.


Listing 6.3: Library Retry


3 public static void RetryExample(Queue<Person> buffer)4 {5 Person result = STMSystem.Atomic(() =>6 {7 if (buffer.Count == 0)8 STMSystem.Retry(); //Initiates retry9

10 return buffer.Dequeue();11 });12

13 //Work with the result14

15 STMSystem.Retry(); //Has no effect16 }17 }

6.3.4 Nesting

The STM library supports both lexical and dynamic nesting of transactions.Lexical nesting is achieved by specifying a transaction within the body ofanother transaction, while dynamic nesting is achieved by calling a method,which is defined using transactions, within the body of another transaction.Nesting is done under the semantics of closed nesting as described in Section 4.6.An example of nesting using the STM library is shown in Listing 6.4. Line6-15 show a lexically nested transaction, while line 20-24 show two dynamicallynested transactions as the Enqueue and Dequeue methods are themselvesdefined using a transaction.

Listing 6.4: Library Nesting


3 public static void NestingExample()4 {5 TMVar<string> s = new TMVar<string>(string.Empty);6 var result = STMSystem.Atomic(() =>7 {8 s.Value = "abc";9 STMSystem.Atomic(() =>

10 {11 s.Value = s + "def";12 });13

14 return s.Value;15 });16


17 Queue<Person> buffer1 = new Queue<Person>();18 Queue<Person> buffer2 = new Queue<Person>();19

20 STMSystem.Atomic(() =>21 {22 var item = buffer1.Dequeue();23 buffer2.Enqueue(item);24 });25 }26 }

6.4 Internal Details

This section highlights the defining aspects of the STM libraries internalimplementation.

6.4.1 Writing a Value

As described in Section 6.2.2 TLII uses a write-set to bu↵er any writes to bewritten to the actual locations when the transaction commits. Therefore anywrites to transactional variables must be redirected to the write-set insteadof the actual location. Listing 6.5 shows how this is accomplished in theSTM library. Line 3 gets the currently executing transaction from threadlocalstorage. That is, each thread gets its own transaction instance when accessingthe LocalTransaction property. Based on the status of the transaction,di↵erent actions are taken. Each thread accesses a transaction that hasthe status Committed whenever no transaction is currently executing onthat specific thread. Therefore the case on line 6 covers writing a valueto a transactional variable from outside a transaction scope. How this isaccomplished is explained in Section 6.4.4. The case on line 9 covers writinga value as part of a transaction. Here the value to be written is put into thewrite-set of the transaction performing the write. When the given transactioncommits, the value will be written back to the transactional variable.

6.4.2 Reading a Value

How the value of a transactional variable is read depends on the context inwhich the read occurs. Reading the value of a transactional variable from non-transactional code amounts to reading the value contained within the variable,as this value is guaranteed to be the most recently committed value. On theother hand, a read from transactional code has to read an uncommitted valuewhich was previously written by the same transaction, if such a value exists.Only if no such value is present should the transaction read the value currentlycontained in the transactional variable. Listing 6.6 shows how reading thevalue of a transactional variable is implemented in the STM library. The case


Listing 6.5: Writing to a Transactional Variable

1 private void SetValueInternal(T value)2 {3 var me = Transaction.LocalTransaction;4 switch (me.Status)5 {6 case Transaction.TransactionStatus.Committed:7 SetValueNonTransactional(value);8 break;9 case Transaction.TransactionStatus.Active:

10 me.WriteSet.Put(this, value);11 break;12 }13 }

on line 6 corresponds to a non-transactional read and are forwarded directly tothe base class in order to retrieve the value of the transactional variable. Thecase on line 8 corresponds to a transactional read. If the transaction’s write-setdoes not contain a value for the current TMVar object, lines 10-11 reads thevalue directly. Otherwise line 13 gets the value, which was previously writtenby the current transaction, from the current transaction’s write-set. Finally, online 15, the transactional variable is added to the active transaction’s read-set,so that the read can be validated before the transaction commits.

Listing 6.6: Reading a Transactional Variable

1 private T GetValueInternal()2 {3 var me = Transaction.LocalTransaction;4 switch (me.Status)5 {6 case Transaction.TransactionStatus.Committed:7 return base.GetValue();8 case Transaction.TransactionStatus.Active:9 T value;

10 if (!me.WriteSet.Contains(this))11 value = base.GetValue();12 else13 value = (T)me.WriteSet.Get(this);14

15 me.ReadSet.Add(this);16 return value;17 }18 }

6.4.3 Committing a Transaction

Committing a transaction follows the steps described in Section 6.2.2. List-ing 6.7 shows the implementations of these steps. The Commitmethod declared


on line 1 drives the commit process and returns true if the commit succeeds.If the transaction is not nested within another transaction, the body of the ifstatement on line 3 is entered. The call to the Validate method on line 6acquires the lock on each object in the transaction’s write-set, validates theread-set and increments the version-clock acquiring a new writestamp. TheValidate method returns true if the read-set could be validated correctly andreturns the new writestamp using an out parameter. All locks are acquiredusing a timeout in order to prevent deadlocks and all acquired locks are releasedif validation of the read-set fails. If validation succeeds, the HandleCommitmethod commits all bu↵ered writes to their respective transactional variables,after which all acquired locks are released.

If the transaction is nested within another transaction the else case on line 11is executed. Nested transactions do not need to acquire the lock on items intheir write-set or increment the version clock as they will not be responsiblefor committing any written values under closed nested semantics. Therefore,a validation of the read-set, as seen on line 13, is su�cient. If the validationsucceeds the transaction merges its read and write sets with those of the outertransaction, in order for the outer transaction to commit any written valueswhen it reaches its commit stage. On line 17 the outer transaction is restoredas the currently active transaction so that it can be resumed. Finally on line20 the status of the transaction is changed to committed, before returning online 21.

6.4.4 Providing Strong Atomicity

As described in Chapter 4 the STM system must support strong atomicity. InSection 6.4.1 and Section 6.4.2 it is described how the STM system is ableto detect whether or not read and write operations occur inside or outside atransaction and take di↵erent actions based on this information. In the caseof a non-transactional read the system reads the most recently committedvalue as described in Section 6.4.2, ensuring that no uncommitted values canbe read. A non-transactional write is more complicated as the STM systemmust ensure atomicty between both non-transactional access and transactionalaccess. To that extent the approach described by Hindman et al. in [46] ofexecuting non-transactional access as optimized mini transaction is adopted.As no value is read during a non-transactional write, a non-transactional writein the context of the TLII algorithm consist only of (1) Acquiring the lockon the transactional variable to which the write occurs, (2) Incrementing andgetting the value of the version clock, (3) Committing the new value andtimestamp before, (4) Releasing the acquired lock. Therefore any access to theread-set can be optimized away. The STM library implements this approach inthe SetValueNonTransactional method of the TMVar<T> class shownin Listing 6.8.


Listing 6.7: Committing a Transaction

1 public bool Commit()2 {3 if (!IsNested)4 {5 int writeStamp;6 if (!Validate(out writeStamp))7 return false;8

9 HandleCommit(writeStamp);10 }11 else12 {13 if (!ValidateReadset())14 return false;15

16 MergeWithParent();17 Transaction.LocalTransaction = Parent;18 }19

20 Status = TransactionStatus.Committed;21 return true;22 }

Listing 6.8: Non-transactional Write

1 private void SetValueNonTransactional(T value)2 {3 Lock();4 Commit(value, VersionClock.IncrementClock());5 Unlock();6 }

6.4.5 Providing Opacity

As described in Section 4.7 providing opacity requires ensuring that transactionsdo not read invalid data throughout their execution. Therefore a transactionmust validate a read when it occurs, ensuring that the transaction is notallowed to continue when the read is inconsistent. For this purpose the STMlibrary uses the approach shown by [9, p. 117] and extends the reading ofvalues to the implementation depicted in Listing 6.9. As seen on lines 12-16,reading a value directly from the transactional variable has been extendedwith validation that ensures the transaction aborts if the variable has beenchanged since the transaction recorded its timestamp. The timestamp of thetransactional variable prior to reading it is stored locally on line 12. Afterthe variable has been read the locally cached timestamp is compared to thecurrent timestamp of the transactional variable and the timestamp of the


Listing 6.9: Providing Opacity

1 private T GetValueInternal()2 {3 var me = Transaction.LocalTransaction;4 switch (me.Status)5 {6 case Transaction.TransactionStatus.Committed:7 return base.GetValue();8 case Transaction.TransactionStatus.Active:9 T value;

10 if (!me.WriteSet.Contains(this))11 {12 var preStamp = TimeStamp;13 value = base.GetValue();14

15 if (preStamp != TimeStamp || me.ReadStamp < preStamp)16 throw new STMAbortException("Aborted due to

inconsistent read");17 }18 else19 value = (T)me.WriteSet.Get(this);20

21 me.ReadSet.Add(this);22 return value;23 }24 }

current transaction. If the timestamp was changed during the read or thecached timestamp is higher than the timestamp of the current transaction, thecurrent transaction is aborted by throwing a STMAbortException on line16.

6.5 Testing

In order to ensure that the STM library executes transactions according tothe requirements defined in Chapter 4, a number of tests have been created.

An improvisational approach for testing is to create some ad hoc throw-awaycode[66, Chap. 9] and manually interacting with the program, to ensure thatcode works correctly. This approach has the advantage that tests can be createdquickly and errors found quickly. However the approach has the disadvantagesthat it is unstructured and the testing can not be rerun automatically, so if alater change damages a feature which has previously been tested as workingcorrectly, this may not be discovered.

Another more structured approach is to use unit testing[67], where tests arestructured into unit tests, which only focus on the correctness of a small part of


the program. Furthermore, unit testing is often automated, so they can easilybe rerun to ensure the unit they test still works correctly. A drawback withthis approach is that it is more time consuming to setup than adhoc testing.

As unit testing will allow for automatic testing of new features not breakingexisting functionality, unit testing has been selected for testing the STM library.The defined tests cover areas such as the execution of transactions, nesting,retry, orelse, strong atomicity and exceptions in transactions. Additionally, anumber of tests based on a transactional queue and hashmap implementationhas been created. Where the first set of tests cover smaller parts of the STMlibrary, the second set of tests cover the STM library as a whole, ensuring thatthe library can be utilized for real world scenarios.

In addition a number of tests, which attempt to produce a result which canonly occur when a race condition is present, has been created. While such testscan ultimately not prove that no race conditions are present, they do providea degree of certainty that this is the case. Listing 6.10 shows the RaceTest1method conducting a test for possible race conditions. RaceTest1Internalis called 10,000 times producing a result for each iteration. Line 7 asserts thateach execution of RaceTest1Internal does not produce a result which canonly occur given that a race condition is present. RaceTest1Internalexecutes a read modify write operation on one thread, while another threadwrites to the same variable. The call to Thread.Yield on line 21 signalsthe underlying system that t1 can be descheduled in favor of other threads,allowing t2 to assign result a new value after t1 has read the value of result,but before it has computed and assigned the new value. Thread.Yielddoes not cause t1 to be desheduled in all cases, only when the underlyingscheduling determines that it is appropriate.


Listing 6.10: RaceTest1

1 [TestMethod]2 public void RaceTest1()3 {4 for (int i = 0; i < 10000; i++)5 {6 var result = RaceTest1Internal();7 Assert.IsTrue(result != 120);8 }9 }

10

11 private int RaceTest1Internal()12 {13 var result = new TMVar<int>(10);14

15 var t1 = new Task(() =>16 {17 STMSystem.Atomic(() =>18 {19 if (result.Value == 10)20 {21 Thread.Yield();22 result.SetValue(result.Value * 10);23 }24

25 return result.GetValue();26 });27 });28

29 var t2 = new Task(() => STMSystem.Atomic(() =>30 {31 result.Value = 12;32 }));33

34 t1.Start();35 t2.Start();36 t1.Wait();37 t2.Wait();38

39 return result.Value;40 }

7 Roslyn ExtensionThis chapter describes how the Roslyn C# compiler is extended to support theconstructs of AC# described in Chapter 5. Section 7.1 describes the overallextension strategy. In Section 7.2 the changes made to the lexer and parser aredescribed. Following this, Section 7.3 presents examples of the transformationsmade on the syntax tree by the extension while Section 7.4 describes the testingapproach employed to test the Roslyn extension. Finally, Section 7.5 describesareas where the initial prototype implementation, described in this chapter,conflicts with the intended STM design described in Chapter 5.

7.1 Extension Strategy

The Roslyn C# compiler is extended by modifying the lexing and parsingphases with support for the language constructs described in Chapter 5. Theextended parsing phase outputs an extended syntax tree containing directrepresentations of the language features provided by AC#. The syntax treeis then analyzed to identify AC# constructs, followed by a transformationwhere the language extension of AC# is transformed into equivalent C# codewhich utilizes the STM library described in Chapter 6. This syntax tree is thenpassed to the remaining C# compiler phases, utilizing the compilers semanticanalysis and code generation implementations. The approach is visualized inFigure 7.1, which is a modified version of Figure 3.1. The transformation phaseutilizes both the extended syntax tree and symbol information gathered throughthe Roslyn API. By doing modifications in the early phases, the amount ofchanges required is minimized, as the rest of the phases can be reused withoutmodifications. Furthermore, modifications are done on the stable syntax tree,rather than the unstable bound tree, as described in Section 3.3.

Figure 7.1: Extension occurs at the syntax tree and symbol level. The parseris extended to output an extended syntax tree and transformation of this treeoccurs before the binding and IL emission phases.

67

CHAPTER 7. ROSLYN EXTENSION 68

The described approach was selected due to the following reasons:

1. As described in Section 3.3.2 the C# lexer uses a simple switch caseto identify tokens, which chooses the type of token to identify based onthe first character of the token, while the parser uses a recursive descenttechnique. Both the lexer and parser are implemented by hand. Thetechniques employed have a low degree of complexity, as the first methoduses a simple switch and the latter method corresponds to parsing onenon-terminal at a time. Thus modifying the lexer and parser is simplerthan if more complex techniques had been employed, such as Look AheadLR (LALR) parsing[68].

2. The Roslyn compiler generates the nodes composing its syntax treesalong with factory methods and a visitor pattern implementation basedon the content of an Extensible Markup Language (XML) file. Thereforeadding new nodes to represent the extended language constructs, suchas the atomic block and retry statement, simply amounts to addingdefinitions for these nodes to the XML, allowing the employed tool togenerate source code for the new nodes.

3. As described in Section 3.4 Roslyn’s syntax trees are designed to be fastto modify by reusing underlying green nodes instead of creating completecopies[30, p. 6]. This speaks for conducting transformation on the levelof the syntax tree despite its immutable implementation.

4. The Roslyn project has been designed to allow programmers to utilizethe information gathered during the compilation phases by analyzingthe syntax tree, the information in the symbol table, and the results ofsemantics analysis. These parts of the compiler are exposed as a publicAPI allowing access to both syntactic and semantic information. Utilizingthis API during the transformation phase allows the transformation todraw on the existing semantic analysis to answer questions such as,what method is the target of a method invocation, without the need forimplementing complex analysis.

5. By parsing an extended syntax tree and transforming it into a regu-lar C# syntax tree, the existing semantic analysis and code emissionimplementations can be utilized.

Despite of the many advantages of the selected approach, a number of disad-vantages also exists. The following disadvantages has been identified:

1. By modifying the syntax tree, the roundtripable property, described inSection 3.4, is lost, as the syntax tree no longer represents the original


source code. Consequently, any error or warning generated by thecompiler refers to the transformed source code, which the programmerdoes not see, as opposed to the original AC# source code. This requiresour own code analysis, to give meaningful errors attached to the originalsource code. Alternatively, the transformation could be performed at alater compilation phase, e.g. modifying the bound tree in the bindingphase or do changes before emitting code in the emit phase. This wouldpreserve the roundtripable property, but also limit the reuse of theexisting compiler.

2. As atomic local variables are translated into variables of a correspond-ing STM type, the types of these need to be known. C# allows theprogrammer to utilize type inference for local variables using the varkeyword. Consequently the type of a local variable is not required tobe defined in the source code. To remedy this, the extension must inferthe type before translation. Roslyn o↵ers the possibility to evaluate thetype of an expression, which is used to infer the type of atomic localvariables with unknown types. Consequently, the extension must performsome of the work that the Roslyn compiler does at later stages of thecompilation, reducing the reuse of the later compilation phases.

7.2 Lexing & Parsing Phases

This section describes the changes conducted in order to extend the lexing andparsing phases of the Roslyn C# compiler to support the constructs describedin Chapter 5.

7.2.1 Lexer Changes

As described in Chapter 5, AC# introduces three new keywords: atomic,orelse and retry. Consequently the lexer has been extended to identifythese keywords as tokens of the correct kind. The C# lexer initially lexeskeyword tokens as if they are identifiers. If an identifier token corresponds toa keyword, a keyword token with the correct kind is returned instead.

In order to identify the new keywords, their definition has been added tothe lookup methods of the SyntaxKindFacts class. This class definesthe GetKeywordKind which the lexer uses to identify the keyword kindof an identifier, if the identifier represents a keyword. Additionally, theSyntaxKindFacts class defines the GetText method for determining thestring representation of a keyword based on its kind.

To allow tokens to represent the new keywords, the AtomicKeyword,OrelseKeyword, and RetryKeyword entries has been added to theSyntaxKind enumeration. As described in Section 3.4.2.5, the SyntaxKind


enumeration contains an entry for each type of node, token, or trivia in C#.Whenever the lexer identifies an occurrence of one of the new keywords, atoken with the corresponding kind is returned. For example an occurrence ofthe atomic keyword results in a token with the kind AtomicKeyword beingreturned.

7.2.2 Syntax Tree Extension

The design described in Chapter 5 adds the atomic, orelse and retryconstructs which the existing syntax tree cannot express. Therefore the syntaxtree must be extended to support these language constructs. As described inSection 3.4.3 nodes composing the syntax tree, along with factory methodsand the visitor pattern implementation, are generated on the basis of anXML file. Adding additional nodes to the syntax tree therefore amountsto defining them in the XML notation, which has been done for the threepreviously mentioned constructs. Listing 7.1 shows the XML code definingthe AtomicStatementSyntax node which represents an atomic block inthe syntax tree. Line 1 defines the name and base class of the node whileline 2 defines its kind. The AtomicStatement kind has been added tothe SyntaxKind enumeration as described previously. Line 3 defines aproperty on the node which holds the token representing the AtomicKeywordwhich starts the definition of the atomic block. The property has a constraintspecifying the kind of the token, that can be associated with the property whichis defined on line 4 as well as a comment given on lines 5-9. Line 11 defines astatement property which holds the block of statements associated with thedefined atomic block. Line 18 defines a property containing a SyntaxList oforelse blocks associated with the atomic block. This relationship has beenmodeled after the relationship between a C# try statement and its catchclauses, as both have a zero to many association. Finally lines 25-27 definesa comment for the AtomicStatementSyntax , while line 28-30 defines acomment for the factory method.

Listing 7.1: AtomicStatement XML definition

1 <Node Name="AtomicStatementSyntax" Base="StatementSyntax">2 <Kind Name="AtomicStatement"/>3 <Field Name="AtomicKeyword" Type="SyntaxToken">4 <Kind Name="AtomicKeyword"/>5 <PropertyComment>6 <summary>7 Gets a SyntaxToken that represents the atomic keyword.8 </summary>9 </PropertyComment>

10 </Field>11 <Field Name="Statement" Type="StatementSyntax">12 <PropertyComment>13 <summary>


14 Gets a StatementSyntax that represents the statement tobe executed when the condition is true.

15 </summary>16 </PropertyComment>17 </Field>18 <Field Name="Orelses" Type="SyntaxList<OrelseSyntax>">19 <PropertyComment>20 <summary>21 Gets a SyntaxList containing the orelse blocks associated

with the atomic statement.22 </summary>23 </PropertyComment>24 </Field>25 <TypeComment>26 <summary>Represents an atomic block.</summary>27 </TypeComment>28 <FactoryComment>29 <summary>Creates an AtomicStatementSyntax node</summary>30 </FactoryComment>31 </Node>

Declaration of transactional local variables, fields and parameters does notrequire modifications to the syntax tree as the standard nodes for theseconstructs allow a collection of modifiers to be associated with the declaration.Any atomic modifiers are simply added to the collection along with any othermodifiers.

7.2.3 Parser Changes

As described in Section 3.3.2 the C# parser uses a recursive descent strategyimplemented by hand. As customary for recursive descent implementations,each non-terminal has a method responsible for parsing that particular non-terminal. For the new atomic, orelse and retry such a method has beenadded. Furthermore, the methods for parsing local variables, fields, parameters,and properties have been modified to allow for an atomic modifier, as well asgenerating error messages for any unsupported modifier combinations such asatomic const and readonly const, as defined in Section 5.1.2. Errorsare associated with the erroneous nodes as is customary for the Roslyn compiler.A later compilation phase generates error messages and cancels code emissionif any errors are present in the syntax tree.

Listing 7.2 shows the ParseAtomicBlock method responsible for parsingan atomic block. Line 3 parses an atomic keyword by returning a tokenrepresenting the keyword while line 4 parses the block of statements representingthe transaction body. On line 6 to 21 any orelse blocks associated withthe atomic statement are parsed , if any are present. Line 7 allocates aSyntaxListBuilder . The Allocate method reuses existing space ifpossible, in order to limit the overhead of allocation. As seen on line 12 the


parsing of the actual orelse block is delegated to its corresponding method.On line 20 the space allocated by the call on line 7 is freed. The try finallyconstruct ensures that the space is freed in case of an exception. Finally online 23 the syntax factory is used to create the AtomicStatementSyntaxthat is returned.

Listing 7.2: Method for parsing atomic block

1 private StatementSyntax ParseAtomicBlock()2 {3 var atomicKeyword = this.EatToken(SyntaxKind.AtomicKeyword);4 var block = this.ParseEmbeddedStatement(false);5

6 var orelses = SyntaxFactory.List<OrelseSyntax>();7 var orelseBuilder = _pool.Allocate<OrelseSyntax>();8 try9 {

10 while (this.CurrentToken.Kind == SyntaxKind.OrelseKeyword)11 {12 var clause = ParseOrelse();13 orelseBuilder.Add(clause);14 }15

16 orelses = orelseBuilder.ToList();17 }18 finally19 {20 _pool.Free(orelseBuilder);21 }22

23 return _syntaxFactory.AtomicStatement(atomicKeyword, block,orelses);

24 }

7.2.4 Symbol Changes

The symbols representing fields, local variables, and parameters have beenmodified with a new IsAtomic property of type bool, indicating if the symbolrepresent an atomic variation of these constructs. The logic which createsthe symbol table has further been modified to, for each of these constructs,determine whether the declaration is atomic and set the IsAtomic to theappropriate value.

For each usage of a field, local variable or parameter, the semantic modelallows for the retrieval of a symbol, representing the corresponding declaration.Based on the added IsAtomic property this symbol can be used to determinewhether the usage represents the usage of an atomic variable. For cases whereonly access to atomic variables are of interest, in relation to the STM system,the symbol extension allows for easy distinguishing between access to atomicvariables or non-atomic variables.


7.3 Syntax Tree Transformations

This section presents the syntax tree transformations performed during thecompilation process. In order to prevent errors due to ambiguity between typenames, the transformation process uses fully qualified names[17, p. 73] for anytypes in the STM library. In the examples presented in this section the simplenames have been used, in order to improve readability.

7.3.1 Atomic Block

In Section 5.1.1 the design for the atomic block is described. Listing 7.3depicts the syntax of the atomic block before transformation.

The transformation of an atomic block is done using the following four steps:

1. Construct a lambda expression with a body equal to that of the atomicblock.

2. Construct lambda expressions for any orelse blocks associated withthe atomic block, with bodies equal to that of their respective originaldefinition.

3. Construct a syntax node for the invocation of the STMSystem.Atomicmethod, supplied with the created lambda expressions as arguments.

4. Replace theatomicblock with the invocation of theSTMSystem.Atomicmethod.

For syntax shown in Listing 7.3 the transformation produces the output shownin Listing 7.4.

Listing 7.3:atomic Block Before Transformation

1 atomic2 {3 //Block4 }5 orelse6 {7 //Orelse block8 }

Listing 7.4:atomic Block After Transformation

1 STMSystem.Atomic(() => {2 //Block3 },4 () => {5 //Orelse block6 });


As a consequence of translating an atomic block into a lambda expression,return statements inside the lambda expression do not return out of the atomicblock as described in Section 5.1.1. In order to ensure the wanted semantics, ananalysis is performed to identify all atomic blocks containing return statements.In such a case, a return statement is added before the method invocation.Return statements inside nested transactions must return out of the enclosingmethod. To accommodate this, the analysis also identifies nested transactions.

7.3.2 Field Types

In Section 5.1.2 the design of transactional fields are described. Any fielddeclaredatomicmust have its type substituted to the corresponding STM typein order for the STM system to track any changes to the variable. If a specializedtype exist, such as TMInt for int, then that type is used. Otherwise the genericTMVar is used. As the STM types act as wrapper objects that allows the STMsystem to track how the wrapped values are accessed, all atomic fields must beinitialized to an instance of a STM type, as accessing the wrapped value willotherwise cause a NullReferenceException. If an initializer expressionis given as part of the field declaration, the constructor of the wrapping STMobject is given the expression as an argument, initializing the wrapped valueto the value computed by the initializer expression. If no initializer expressionis given, the wrapped value is initialized to the default value for the wrappedtype by instantiating the STM object using its parameterless constructor. Thetransformation of an atomic field declaration follows the steps described below:

1. Determine the type of the wrapping STM object, based on the type ofthe field declaration.

2. For each variable declared as part of the field declaration, construct anobject instantiation expression following the approach described above.This expression serves as the new initializer for the particular variable itwas created for.

3. Construct a new field declaration with the same access modifiers andvariable names as the original declaration, but substituting the type withthe STM object type, and initializer expressions with the created objectinstantiation expressions.

4. Replace the original field declaration with the constructed field declara-tion.

Listing 7.5 presents an example of two atomic field declarations beforetransformation while Listing 7.6 shows the result of applying the transformation.Line 3 of Listing 7.5 is transformed to line 3 of Listing 7.6. The type of the


field is changed to the specialized integer STM type TMInt and the initializerexpression is used to initialize the value of the created TMInt object. Line 4of Listing 7.5 is transformed to line 4 of Listing 7.6. The type is transformedto TMVar<string>. For field3, which original definition does not containan initializer expression, an initializer expression has been created followingthe previously described procedure.

Listing 7.5:atomic Field Before Transformation

1 public class AtomicFieldExample2 {3 private atomic int field1 = 1;4 public atomic string field2 = "Hello world!", field3;5 }

Listing 7.6:atomic Field After Transformation

1 public class AtomicFieldExample2 {3 private TMInt field1 = new TMInt (1);4 public TMVar<string> field2 = new TMVar<string>("Hello

world!"), field3 = new TMVar<string>();5 }

7.3.3 Properties

In Section 5.1.3 the design for transactional properties was described. In orderto provide the wanted semantics for the automatic form, the transformationinvolves two parts: an atomic backing field, and a manual property. Thetransformation takes the following approach for each transactional propertyidentified in the syntax tree:

1. Construct a get-body with the access-modifier of the original property’sget. The get-body contains a block that returns the value of the backingfield. The backing field is not yet constructed, but a method is used todetermine its future identifier to ensure a correct reference.

2. Construct a set-body with the access-modifiers of the original property’sset-body. The set-body contains a block where an assignment of valueis made to the backing field.

3. Construct a manual property with the access modifier of the originalproperty and the get-body and set-body constructed earlier.

4. Construct a private transactional field of the same type as the originalproperty, used as backing field.


5. Insert the new property after the original property and replace theoriginal property with the private transactional field.

Transactional properties are transformed before transactional fields. Thisenables the transformation of transactional properties to simply generatea backing field which is declared atomic and rely on the transformation oftransactional fields to transform the type of the field to the correct STM type.No transformation has to be done for the manual form, as the transactionalfield used for backing the property is processed as described in Section 7.3.2.The automatic form before transformation is exemplified in Listing 7.7, wherethe transformation result is shown in Listing 7.8.

Listing 7.7:atomic Property Before Transformation

1 class Car {2 public atomic int KmDriven { get; set; }3 }

Listing 7.8:atomic Property After Transformation

1 class Car {2 private atomic int _kmDriven;3 public int KmDriven {4 get {5 return _kmDriven;6 }7 set {8 _kmDriven = value;9 }

10 }11 }

7.3.4 Local Variables

In Section 5.1.4 the design for transactional local variables is described. Similarto fields modified with the atomic keyword, atomic local variables musthave its type substituted to a corresponding STM type. Thus, the approach issimilar to the one described in Section 7.3.2, with the exception that a localvariable can be declared without specifying the type by using the var keyword,relying on compile-time type inference to determine the type. Since the typehas not yet been determined at the point of the transformation, Roslyn’s APIis utilized to infer the type, and replace the var keyword with the type. Thisis done by identifying all local declaration statements with the type var andthe atomic modifier.

The transformation is exemplified with a before example on Listing 7.9, whereon line 5 a local variable is declared by using the var keyword. The r-value of


the statement is an expression, which type can be identified using the Roslyncompiler’s API. In Listing 7.10 on line 5 the result shows that the type wasinfered to a string, thus the var is replaced by a TMVar<string>.

Listing 7.9:atomic Local Variables Before Transformation

1 public class AtomicVarExample2 {3 public void Method()4 {5 atomic var variable1 = "Hello World!";6 atomic int variable2 = 42;7 }8 }

Listing 7.10:atomic Local Variables After Transformation

1 public class AtomicVarExample2 {3 public void Method()4 {5 TMVar<string> variable1 = new TMVar<string>("Hello World!");6 TMInt variable2 = new TMInt(42);7 }8 }

7.3.5 Accessing Transactional Variables

As described in Section 6.3.2 setting the value of a transactional variable is donethrough the Value property of the supplied STM types. As the type of anyfield, local variable or parameter declared atomic in AC# is changed to thecorresponding STM type, any assignment must go through the Value property.Additionally, the STM types supply support for implicit conversion to thewrapped value, when appearing as r-values. The transformation can howevernot rely on implicit conversion in all cases as, for example the comparison oftwo TMInt objects results in reference comparison instead of the expectedinteger comparison. As such, also the r-value appearances of any transactionalfield, local variable and parameter must be transformed to access the Valueproperty, which ensures that the wrapped object is accessed instead of theSTM object. Special handling is given to the ++ and -- operators as thenumeric STM types supply transactional implementations of these operators.The transformation ensures that these operators can be used on a transactionalvariable directly, instead of on its Value property.

The transformation of access to transactional variables is divided into twoparts. The first deals with the usage of a transactional variable occurringas a single identifier such as someTMVar. The second part handles memberaccess expressions such as object.tmfield1.tmfield2. While these two


implementations di↵er in the syntactic constructs they work on, their overallapproach both follow the steps described below:

1. Identify each usage of a transactional field, local variable, or parameterincluding both r- and l-value occurrences.

2. Construct a member access expression that accesses the Value propertyof the identified variable.

3. Replace the usage of the variable with the constructed member accessexpression.

Listing 7.11 presents an example of accessing both a transactional field, localvariable and parameter. Listing 7.12 shows the result of applying the trans-formation. The assignment on line 8 of Listing 7.11 is transformed to accessthe Value property of both its left and right side, as both of the variablesinvolved are transactional. The result of the transformation is shown on line8 of Listing 7.12. The member access expression on line 10 of Listing 7.11 islikewise transformed to access the Value property of both transactional fieldsinvolved. The resulting code is shown on line 10 of Listing 7.12.

Listing 7.11: Usage of atomic Variables Before Transformation

1 public class AtomicExample2 {3 public atomic AtomicExample aField;4

5 public AtomicExample ExampleMethod(atomic int i)6 {7 atomic int k = 0;8 k = i;9

10 return aField.aField;11 }12 }

Listing 7.12: Usage of atomic Variables After Transformation

1 public class AtomicExample2 {3 public TMVar<AtomicExample> aField = new TMVar<AtomicExample>();4

5 public AtomicExample ExampleMethod(TMInt i)6 {7 TMInt k = new TMInt(0);8 k.Value = i.Value;9

10 return aField.Value.aField.Value;11 }12 }


7.3.6 Parameters

As with transactional local variables and transactional fields, the type of aparameter declared atomic must be changed to the corresponding STM typein order for the STM system to track assignments to it. Listing 7.13 presents amethod taking two atomic parameters while Listing 7.14 presents the result ofapplying the parameter transformation. Each atomic parameter is transformedindividually and any ref or out modifiers are preserved.

Listing 7.13:atomic Parameters Before Transformation

1 public class AtomicParameterExample2 {3 public void TestMethod(atomic int x, bool b, atomic ref string

s)4 {5 //Body6 }7 }

Listing 7.14:atomic Parameters After Transformation

1 public class AtomicParameterExample2 {3 public void TestMethod(TMInt x, bool b, ref TMVar<string> s)4 {5 //Body6 }7 }

7.3.6.1 Transactional Output Parameters

Transactional output parameters described in Section 5.2.3 require additionalhandling as C# requires every execution path in the method body to assign avalue to the parameter[52, p. 42]. As described in Section 7.3.5, any assignmentto a variable is replaced with an assignment to its Value property. As aconsequence no assignments occur to the transactional parameter itself, whichresults in an error in the generated code. To rectify this error, an assignment,assigning a new STM object with the same type as the parameter, is generatedin the top of the method body for every atomic out parameter of thatparticular method. For each atomic out parameter the AC# transformationcontains the following steps:

1. Construct an object initialization expression that creates a new STMobject of the same type as the parameter.

2. Construct an assignment statement that assigns the newly constructedobject initialization expression to the atomic out parameter.


3. Insert the assignment statement as the first statement in the body of theenclosing method declaration.

Listing 7.15 shows a method with an atomic out parameter while Listing 7.16shows the result of applying the transformation. Based on the atomic outparameter declared on line 3 of Listing 7.15 the assignment on line 5 ofListing 7.16 is generated.

Listing 7.15:atomic out Parameter Before Transformation

1 public class AtomicOutExample2 {3 public static void TestMethodAtomic(atomic out int i, int j)4 {5 i = 12;6 j = 12;7 }8 }

Listing 7.16:atomic out Parameter After Transformation

1 public class AtomicOutExample2 {3 public static void TestMethodAtomic(out TMInt i, int j)4 {5 i = new TMInt();6 i.Value = 12;7 j = 12;8 }9 }

While the generated assignment solves the problem of assignments to atomicout parameters it introduces two new problems. Firstly, C# requires thatevery out parameter is assigned a value before a method exits, generatinga compile time error if that is not the case[17, p. 94]. This error messageis lost for transactional output parameters due to the generated assignment.Secondly, C# requires that an out parameter is assigned to before readingfrom it[17, p. 94], generating a compile time error if a read occurs before anout parameter is assigned a value. If a read occurs before any assignment inthe original source code, no error is given due to the generated assignment.In both cases a meaningful error can be generated by applying analysis to atransactional output parameters Value property by following the rules definedin[17, p. 95]. No such analysis has however been implemented in the initialprototype of AC#.


7.3.7 Arguments

As described in Section 5.2.1 AC# support value parameters. Replacing thetype of an atomic parameter with the corresponding STM type, as describedin Section 7.3.6, results in a type mismatch when calling a method with anatomic parameter of some type T , as the parameter is no longer of typeT , but of T ’s corresponding STM type. As a result transformation must beapplied to arguments passed to an atomic parameter, ensuring that theargument represents an object of the required STM type. The transformationapplied when an argument is passed to an atomic parameter goes throughthe following steps:

1. Determine the STM type of the formal parameter to which the argumentcorresponds.

2. Construct an object initialization expression, which creates an object ofthe previously determined STM type, where the argument expression isgiven as argument to the constructor of the object.

3. Replace the argument with the constructed object initialization expres-sion.

Listing 7.17 presents an example of calling a method with anatomicparameter,while Listing 7.18 shows the result of the transformation. Line 3 is transformedas described in Section 7.3.6. Line 10 in Listing 7.17 is transformed to line10 in Listing 7.18. The argument to the atomic parameter is replaced withan object initialization expression that creates a new TMInt object which isinitialized using the original argument.

Listing 7.17:atomic Argument Before Transformation

1 public class AtomicArgumentExample2 {3 public static void TestMethod(atomic int x, int y)4 {5 //Body6 }7

8 public static void Main()9 {

10 TestMethod(1, 2);11 }12 }


Listing 7.18:atomic Argument After Transformation

1 public class AtomicArgumentExample2 {3 public static void TestMethod(TMInt x, int y)4 {5 //Body6 }7

8 public static void Main()9 {

10 TestMethod(new TMInt(1), 2);11 }12 }

7.3.7.1 Ref/Out Arguments

Supporting atomic ref and out parameters, as described in Section 5.2.4.1,presents a problem due to the type mismatch as a result of transformingatomic variables. ref and out parameters require the argument to be anassignable variable of the same type as the parameter. However as the type ofthe parameter is transformed, a variable of type T cannot be passed directlyas ref or out to an atomic parameter of type T .

Four di↵erent cases exist for passing ref and out arguments. These are:

1. T ! atomic T

2. atomic T ! T

3. atomic T ! atomic T

4. T ! T

where T is some arbitrary type, atomic T is T ’s corresponding STM typeand atomic T ! T represents passing an argument of type T into aparameter of the STM type corresponding to T . The third and fourth casesare handled by the C# compiler so transformation must only be applied inthe first and second cases. For each argument node in the syntax tree thetransformation goes through the following steps:

1. Determine whether the argument represents one of the two previouslydescribed cases. If so, the remaining transformation steps are applied.

2. Construct a local variable with a type definition equal to that of theparameter, initialized using the original argument, and insert it justbefore the method call.


3. Replace the original argument with an identifier equal to the name ofthe previously generated intermediate local variable, passed using thesame modifier as the original argument

4. Construct an assignment syntax node that assigns the value of theintermediate local variable to the original argument and insert it justafter the method call

Listing 7.19 shows an example containing three method calls, where the first andthird call falls in one of the categories which require transformation. Applyingthe transformation to the example presented in Listing 7.19 produces a syntaxtree representing the code shown in Listing 7.20. Line 18 of Listing 7.19 istransformed to lines 18 to 20 of Listing 7.20. As the parameter is atomic, thelocal variable inserted on line 18 is of the parameters corresponding STM type.The argument to the method call has been replaced as seen on line 19. Theassignment on line 20 assigns the value computed by the called method to theoriginal argument. Line 19 of Listing 7.19 is not transformed as it representsthe case atomic T ! atomic T . Line 20 of Listing 7.19 is transformedmuch like 18, except the generated local field is of type int instead of anSTM type, as the parameter is not declared atomic.

C# requires the argument for a ref or out parameter to be a variable. Forthe case T ! atomic T and atomic T ! T , compile time analysishas been implemented to generate an error if the original argument does notcorrespond to a variable, as this is always the case in the generated code.

Listing 7.19:ref Arguments Before Transformation

1 public class AtomicRefExample2 {3 public static void TestMethodAtomic(atomic ref int i)4 {5 i = 12;6 }7

8 public static void TestMethod(ref int i)9 {

10 i = 12;11 }12

13 public static void Main()14 {15 int i = 10;16 atomic int iAtomic = 10;17

18 TestMethodAtomic(ref i);19 TestMethodAtomic(ref iAtomic);20 TestMethod(ref iAtomic);21 }22 }


Listing 7.20:ref Arguments After Transformation

1 public class AtomicRefExample2 {3 public static void TestMethodAtomic(ref TMInt i)4 {5 i.Value = 12;6 }7

8 public static void TestMethod(ref int i)9 {

10 i = 12;11 }12

13 public static void Main()14 {15 int i = 10;16 TMInt iAtomic = new TMInt(10);17

18 TMInt _gen1 = new TMInt(i);19 TestMethodAtomic(ref _gen1);20 i = _gen1.Value;21

22 TestMethodAtomic(ref iAtomic);23

24 int _gen3 = iAtomic.Value;25 TestMethod(ref _gen3);26 iAtomic.Value = _gen3;27 }28 }

7.3.8 Retry

In Section 5.4 the design of conditional synchronization through the use ofretry is described. As retry is a keyword used as a statement, much likethe break and continue statements in C#, the transformation need toidentify all retry keywords and replace them with a method invocation to thestatic method STMSystem.Retry defined in the STM library, described inSection 6.3.3. The library then carries out the e↵ect of the retry statement.To inform programmers of unintended behavior, analysis generating an error ifthe retry statement is used outside of an atomic or orelse block, has beenimplemented.

7.4 Testing

Testing is employed to ensure the Roslyn extension fulfills the design decisionsof AC# and the integration with existing language features described inChapter 5.


For the reasons described in Section 6.5, unit testing is chosen for testing theRoslyn extension. Unit testing is valuable in relation to the maintenance andfurther development of the Roslyn extension. Furthermore, Roslyn is stillunder heavy development, which means radical changes will happen. Theunit test suite is beneficial to test if the extension still works, whenever newchanges are made to Roslyn. The disadvantage of unit testing being moretime consuming than ad hoc testing is outweighed by the many advantageswhich is gained. Additionally, each unit test is built using a black-box testingstrategy[69, p. 87], where the program is viewed as a black box and focus isonly on the input and output. This is done as the focus of the unit tests ison the functionality of the compiler. That is, the tests focus on what it does,and not on how it does it. Additionally it fits well with how a compiler isnormally treated, as it is treated as a black-box where the input and output isonly available.

Each STM construct and each integration with an existing language feature iscovered by at least a single unit test. Furthermore, integration with disallowedexisting language features is tested, e.g. there is a unit test that ensures anerror is produced if the programmer tries to use the retry statement outsideof an atomic or orelse block. This results in a rather extensive test suite,which is valuable in relation to the correctness of the STM constructs andintegration with language features of AC#.

Listing 7.21 shows an example of a unit test for an empty atomic block.On line 3 the string finalStr to be compiled is generated using theMakeSkeletonWithMain method, which generates a test namespace, classand main method, with the supplied string arguments strInClass andstrInMain included. Afterwards the string is written to a test file, whichis used as argument in the method on line 8, which executes csc.exe (C#command line compiler) and returns a CmdRes object that contains thecommand line output. On line 9 it is checked that the command line outputdoes not contain any warning, error or invalid exitcode. For unit tests thatcheck error generations, e.g. an illegal existing language feature used withan STM construct, this step is not present, instead CmdRes can be checkedif it contains the expected error or warning. On line 11 the expected resultof the compilation string expecStr is generated and on line 17 the actualcompilation string compiledStr is fetched. In order to retrieve the actualcompilation string, csc.exe is extended with an additional argument calledstmiout , which writes the source code after STM transformation to thespecified path. This argument is used in the method on line 8. Finally online 18 the expected string and the compiled string is checked for equality, ifequal the test passes and otherwise it fails. Before checking for equality, theAssertEqualStrings method removes formatting from the strings.


Listing 7.21: Empty Atomic Block Unit Test

1 public void AtomicBlockEmpty()2 {3 string finalStr = MakeSkeletonWithMain(4 strInClass: "",5 strInMain: "atomic{\n\t\t\t"+6 "}");7 StringToTestFile(finalStr);8 CmdRes res = RunCscWithOutAndStmIOut();9 AssertCmdRes(res);

10

11 string expecStr = MakeSkeletonWithMain(12 strInClass: "",13 strInMain: STM.STMNameSpace + ".STMSystem.Atomic(" +14 "() =>" +15 "{" +16 "});");17 string compiledStr = TestFileToString(currentCompiledCsFile);18 AssertEqualStrings(expecStr, compiledStr);19 }

7.5 Design and Integration Revisited

This section describes areas in which the initial prototype implementationdescribed in this chapter, conflicts with the intended design and languageintegration described in Chapter 5. Ideally no conflicts would happen, asthe analysis in the design and language integration chapter would foreseethem. However, implementing the design gave an insight and knowledge intounforeseen areas.

7.5.1 Transactional Local Variables and Fields

As described in Section 7.3.2 and Section 5.1.4 local variables and fields oftype T declared atomic are transformed to declarations with a type equalto T ’s corresponding STM type. As part of this process the local variable orfield is initialized to an STM object of the correct type, using any initializerexpression if present. If not, the value of the variable is initialized to thedefault value of type T through the STM object’s parameter less constructor.

For fields this presents no issue as fields are always initialized to the defaultvalue of their type if no initializer expression is present[17, p. 93]. This ishowever not the case for local variables. The C# compiler generates an errorif a local variable is accessed before it has been assigned a value[17, p. 96].As atomic local variables are always initialized, this error will never occurfor such variables, which can lead to unintended behavior in cases where theprogrammer forgets to assign an atomic local variable. In such cases the


default value of the atomic local variables original type will be the valueaccessed, potentially leading to some confusion.

In order to provide similar error messages for atomic local variables andregular C# local variables, the Roslyn flow analysis which detects whether agiven accessed local variable has been definitely assigned, must be extended.It must be extended to track both initial assignment, as part of the variabledeclaration, and assignment to the STM objects Value property, as opposedto assignments directly to the variable. Such analysis has however not beenincluded in the initial prototype described in this chapter.

7.5.2 Transactional Optional Parameters

C# requires that the default value given when declaring an optional parameteris one of the following[17, p. 309]:

1. A constant expression

2. An expression of the form new S() where S is a value type

3. An expression of the form default(S) where S is a value type

As described in Section 7.3.6 the transformation of an atomic parametertransforms the type of the parameter to one of the STM types. Providinga default value to an atomic parameter therefore amounts to initializinga new STM object with the same type as the parameter, supplied with thedefined default value to its constructor. All STM types are however referencetypes and the creation of a new STM object can therefore, per the three rulesdescribed above, not be supplied as the replacement default value, as thiswould result in a compile time error. As a result default values for transactionalvalue parameters are not implemented in the initial prototype described in thischapter. Analysis has been implemented to produce an error if an atomicvalue parameter is given a default value. This is done in the parsing phase,by checking if a parameter is declared with the atomic modifier and has anoptional value.

7.5.3 Transactional Ref/Out Parameters & Arguments

As described in Section 5.2.2 and Section 5.2.3 the intended design was thatassignments to transactional ref/out parameters would take e↵ect when thetransaction in which the assignment takes place commits. This behavior holdstrue for the case:

• atomic T ! atomic T


That is, the case involving an atomic argument and parameter. For the cases:

• T ! atomic T

• atomic T ! T

the behavior is however di↵erent. Due to the type mismatch between theargument and parameter, an intermediate local variable is passed as the actualargument, and the original argument is assigned as the first thing after thecall finishes. This conflicts with traditional C#, where any assignment to aref/out parameter takes e↵ect immediately[52, p. 76]. If the parameter isdeclared atomic and the method call is wrapped in an atomic block, therewill be no di↵erence in timing as any assignments to atomic variables takee↵ect when the transaction commits. However, if the call is not wrapped inan atomic block the di↵erence in timing can have unintended consequences,especially when considering concurrent execution. Due to this issue a numberof ways for handling atomic ref/out parameters and arguments wereinvestigated:

1. Warn the programmer if one of the two problematic cases is used.

2. Disallow the two problematic cases described above.

3. Disallow atomic ref/out parameters.

In C#, passing a volatile field as ref/out results in the field being treatedas non-volatile within the method body[70]. If such an operation is detectedthe compiler creates a warning informing the programmer of the change insemantics. The same approach could be taken whenever one of the twoproblematic cases described earlier is detected.

Disallowing the ref/out cases where both an atomic and non-atomicvariable/parameter are involved removes the cases where the desired timingcannot be provided. However, it will, for example, no longer be possibleto pass an atomic integer into an out integer parameter such as thatof Int32.TryParse. Such restrictions limits the flexibility when usingatomic variables, and the programmer needs to circumvent this, by the useof intermediate variables which clutters the code, and reduces the readability.

Disallowing atomic ref/out parameters, prevents the programmer fromdeclaring atomic ref/out parameters, which removes the timing issuespresent in the cases where an intermediate atomic variable is required.However, the ability to pass an atomic variable by reference is lost and thecase of passing a variable of type atomic T into a parameter of type T is


still unsolved. Consequently, this would limit the orthogonality of the atomicconstruct.

For the initial prototype presented in this chapter the first option of warningthe programmer of the change in semantics has been selected. This ensuresthat the change in semantics is not hidden from the programmer therebyallowing her to adapt the program. This choice follows the approach taken byC# in the case of a volatile field passed by ref into a method as describedabove.

8 Evaluation ofCharacteristicsThis chapter evaluates AC#, its associated STM library, and locking in C#according to the evaluation method described in Section 1.5. The evaluationfacilitates a conclusion on our hypothesis “Language integrated STM providesa valid alternative to locking in terms of usability, and provides additionalbenefits compared to library-based STM, when solving concurrent problems inC#” defined in Section 1.4. The following sections describes how each of thethree concurrency approaches are evaluated according to the characteristics.Each of the three concurrency approaches is given a placement on the spectrumof a given characteristic, as defined in Section 1.5.

8.1 Implicit or Explicit Concurrency

All the selected concurrency approaches rely on starting threads in order tointroduce concurrency as well as manually specifying critical regions usingeither locks or the atomic block. This makes all the approaches lean towardthe explicit end of the spectrum. The two STM based approaches are howevermore implicit as STM abstracts away synchronization details. Locking in C#on the other hand requires explicitly stating how synchronization is achievedand is therefore placed close to the explicit concurrency extreme. AC# andthe STM library reside slightly more towards the implicit concurrency endof the spectrum. The placement of the three concurrency approaches on theimplicit - explicit concurrency spectrum is depicted in Figure 8.1.

ExplicitImplicit

Locking

STM Library

AC#

Figure 8.1: Concurrency approaches on the implicit - explicit concurrencyspectrum

90

CHAPTER 8. EVALUATION OF CHARACTERISTICS 91

8.2 Fault Restrictive or Expressive

Locking presents the programmer with a set of constructs aimed at solvingconcurrency problems, but does little to guarantee correct usage. The lockingconstructs in C# are explained in detail in Section 2.1. Locking enablescontrol of synchronization at a very low level of detail, which is very expressive.Therefore, locking in C# is placed at the expressive end of the spectrum.

The STM based approaches delegate the details of how synchronization isachieved to the underlying STM system, allowing STM based concurrency toavoid some of the errors associated with locking, such as deadlocks. The STMbased approaches however still rely on shared memory for communication andrequire programmers to define transaction scopes and introduce concurrencyby starting threads. The abstractions provided by STM limits the possibility ofexpressing synchronization at a low level of detail. The STM based approachesreside towards the expressive end of the spectrum but contain fault restrictiveelements pulling them more towards the fault restrictive extreme than locking.AC# however implements a number of compile time errors and warningskeeping the programmer from utilizing undesirable combinations such asretry statements outside atomic or orelse block, moving it slightlytowards the fault-restrictive end of the spectrum. The placement of the threeconcurrency approaches on the fault restrictive - expressive spectrum is depictedin Figure 8.2.

ExpressiveFault Restrictive

LockingSTM Library

AC#

Figure 8.2: Concurrency approaches on the fault restrictive - expressive spec-trum

8.3 Pessimistic or Optimistic

Locking applies synchronization by enforcing mutual exclusion. This approachis well suited in scenarios where errors are common which would result in ahigh number of aborts if an STM based approach had been used. Lockingis therefore an inherently pessimistic concurrency approach as it preventserrors from occurring instead of correcting them when they occur. STM onthe other hand allows multiple threads to proceed simultaneously, correcting


any errors that may occur by aborting and re-executing transactions. HenceSTM takes an optimistic approach to concurrency. However, the employedSTM system uses lazy updates as opposed to eager updates, which is a slightlymore pessimistic approach to STM keeping AC# and the STM library fromreaching the optimistic extreme.Therefore AC# and the STM library residesclose to at the optimistic extreme of the pessimistic - optimistic spectrumwhile locking resides close to at the pessimistic extreme. The placement ofthe three concurrency approaches on the pessimistic - optimistic spectrum isdepicted in Figure 8.3.

OptimisticPessimistic

Locking

AC#

STM Library

Figure 8.3: Concurrency approaches on the pessimistic - optimistic spectrum

8.4 Readability & Writability

The evaluation of readability and writability is based on a number of sharedcharacteristics: Data Types, Syntax Design, Simplicity, and Orthogonality.The characteristics Data Types and Syntax Design will not be evaluated on aspectrum of two extremes, as the choices made are trade-o↵s a↵ecting othercharacteristics, e.g. simplicity and readability. The evaluation of these willtherefore be taken into account when the other characteristics are evaluated.

In addition to the shared characteristics, writability is based on level ofabstraction and expressivity.

8.4.1 Data Types

All three concurrency approaches are either integrated into or build around C#,which means they have almost the same data types. A di↵erence lies in the STMapproaches where the types of transactional variables are treated di↵erently.In the STM library, transactional variables are defined using transactionaltypes, such as TMInt, which means that it is possible to create an array or listcontaining these types, e.g. TMInt[] defines an array of TMInt types. AC#instead preserves the original types of transactional variables, allowing theprogrammer to utilize an atomic variable of type T as if it was of type T even


though the compiler transforms the type of the variable to T ’s correspondingSTM type, as described in Section 7.3.2. As transactional variables are notspecial types in AC# it is not possible to define a list or array of transactionalvariables directly, instead a wrapper class with an atomic field must beemployed.

The need to define an array of transactional types was experienced during thedevelopment of the hashmap implementations which use an array to representthe buckets of the hashmap. Listing 8.1 and Listing 8.2 show how the hashmapbuckets are defined in the STM library and AC#, respectively. In the librarySTM example on line 3 the buckets are defined directly. The type of thebuckets field will be explained gradually. The inner Node type, is a simpleclass which represents a key value pair, where the value is a transactionalvariable, defined in the STM library code on line 8 and in AC# code on line 12.In case of collisions the Node class allows for chaining of nodes representingthe key/value pairs inserted into the hashmap through its Next property.Each bucket is represented by a transactional variable which points to the firstelement in the collision list, if any such element exists. The Next propertyof the Node class is transactional, allowing the system to detect changes tothe internals of the collision list. This, along with the transactional variablepointing to the first element in the list, allows for changes to the entire collisionlist to be tracked. Finally the backing array is also wrapped in a transactionalvariable allowing the STM system to track assignments to the entire backingarray in order track when the backing array is resized. In the AC# example online 3 it is not possible to define the array of transactional variables directly, soa wrapper type called Bucket defined on line 7 is used. This is a consequenceof the design of the AC#.

Ultimately prohibiting the programmer to define an array or list of transactionaltypes directly, lowers the readability and writability of AC#, as it requires theprogrammer to write, read and maintain an extra wrapper class. To remedythis, a library of data structures which provides tracking to internal changescould be included in the language.

The special types used by the STM library to represent transactional variablesrequires the programmer to use the Value property whenever she needs tomake an assignment. Implicit conversion ensures that an object of one ofthe STM types, such as TMInt, in most cases can be used as an object ofthe type it is wrapping. Listing 8.3 shows an equality comparison of twotransactional variables using the STM library. The comparison on line 5requires the programmer to access the Value property of the TMVar objectsas implicit conversion will not be used if the TMVar’s are compared directly.If the value property had not been used the TMVar objects would have beencompared, producing an incorrect result. A similar problem exists when calling


Listing 8.1: HashMap Buckets Array - STM Library

1 public class StmHashMap<K,V> : BaseHashMap<K,V>2 {3 private readonly TMVar<TMVar<Node>[]> _buckets =4 new TMVar<TMVar<Node>[]>();5

6 ... //Other code7

8 private class Node9 {

10 public K Key { get; private set; }11 public TMVar<V> Value { get; private set; }12 public TMVar<Node> Next { get; private set; }13 public Node(K key, V value)14 {15 Key = key;16 Value = new TMVar<V>(value);17 Next = new TMVar<Node>();18 }19 }20 }

Listing 8.2: HashMap Buckets Array - AC#

1 public class StmHashMap<K,V> : BaseHashMap<K,V>2 {3 private atomic Bucket[] _buckets;4

5 ... //Other code6

7 private class Bucket8 {9 public atomic Node Value { get; set; }

10 }11

12 private class Node13 {14 public K Key { get; private set; }15 public atomic V Value { get; set; }16 public atomic Node Next { get; set; }17 public Node(K key, V value)18 {19 Key = key;20 Value = value;21 }22 }23 }


Listing 8.3: Equality comparison of TMVar<bool>

1 public bool TestMethod()2 {3 TMVar<bool> v1 = new TMVar<bool>(false);4 TMVar<bool> v2 = new TMVar<bool>(false);5 return v1.Value == v2.Value;6 }

a method on a TMVar object, implicit conversion does not allow methods ofthe wrapped type to be called on the TMVar directly.

The two STM based approaches have problems combining with existing datastructures. For example, if a programmer utilizes a List<T> inside a transac-tion, any operations on the list are not tracked by the STM system as List<T>is not implemented using transactional variables. Therefore operations on thelist take e↵ect immediately. A transaction may be executed multiple timesdue to being aborted and retried, resulting in the operations, such as addingan element to the list, being executed and taking e↵ect multiple times. Con-sequently data structures utilized in transactions must be: a) Implementedusing transactional variables, allowing the STM system to track any changes,or b) Immutable and tracked through assignment to a transactional variable.

8.4.2 Syntax Design

The use of keywords reduce simplicity but increase readability, as the intentstands out clearly[21, p. 12-13]. Locking in C# employs only a single keyword,lock. The rest of the functionality is provided by library calls as describedin Section 2.1. The use of libraries for locking constructs makes it blendin with other library code, even though it has a special purpose related tosynchronization. This decreases the readability, but keeps the simplicity of thelanguage as it does not introduce keywords for all constructs.

The STM library also uses library calls as described above, thus it has the samedisadvantages and the readability is decreased compared to AC# which haskeywords for the special constructs. The four keywords introduced in AC# dohowever decrease the simplicity of the language. As the atomic keyword hasdi↵erent meanings depending on the context in AC#, its readability is reduced.The simplicity is however raised by employing a minimum of keywords.

The STM library requires the programmer to use static method calls, wraptransactional code in lambdas, wrap transactional variables in TMVars, use theValue property when getting or setting a value, supply orelse transactionsas arguments to the atomic method call, and write return in front of the static


method call if a method must return the value computed by a transaction. Allof these concerns are abstracted away in AC#.

In C# it is possible to name variables after reserved keywords, by prefixingthe name with @. Due to this, the impact of the additional keywords in AC#on the naming of variables is reduced and it is distinguishable when atomicis used as a name or a keyword.

8.4.3 Low or High Simplicity

Locking is based on the idea of mutual exclusion. C# supplies a number ofdi↵erent constructs for defining synchronization using locking as described inSection 2.1. Applying locking correctly in complex scenarios is considered tobe hard[2, p. 56], mainly due to the number of errors that can arise, such asdeadlocks.

Locking requires the programmer to explicitly state how synchronization isto be applied. Both library-based STM and AC# take a more declarativeapproach than locking when specifying synchronization, thus it is simpler asthe programmer does not have to worry about low level details. The usabilitystudies produced by Rossbach et al. in [42] and Pankratius et al. in [43] findthat the surveyed students found STM simpler than fine grained locking butmore complex than coarse grained locking due to issues understanding theexecution of transactions.

In case of conditional synchronization, both forms of STM can leverage theretry functionality, making a transaction block until the variables previouslyread by the transaction are changed. In C# a Monitor allows blocking untilanother thread notifies the blocked thread, but contrary to retry this is noton a declarative level. The retry statement was used in the two STM basedqueue implementations to, with only two lines of code, make the calling threadblock until the queue is non-empty in case Dequeue is called on an emptyqueue. The same feature in the lock-based implementation would require theuse of a semaphore or the Monitor class’s Wait and Pulse/PulseAllmethods, which both require the programmer to manage low level details.

Listing 8.4 and Listing 8.5 show the Add method for the locking and library-based STM hashmap implementations respectively. The locking hashmapimplementation divides the backing array into a number of stripes, where eachstripe is protected by its own lock. When accessing a bucket the hashmap mustdetermine what lock to acquire based on the index of the bucket. This strategyis further described in Appendix C.1.4. In Listing 8.4 on line 4, a calculation ismade to determine which lock needs to be used, additionally on line 18 anotherlock is needed for synchronizing access to the size variable as the previouslyacquired lock only covers part of the buckets, meaning that another thread


could be changing the size variable. This is avoided in Listing 8.5 wherethe method is wrapped in an atomic block, and the STM system detects andresolves potential conflicts.

Library-based STM does not have support for checking implicit dependencybetween static calls at compile time, e.g. a retry outside an atomic callwill not produce a compile time error. Additionally, the STM types providedmust be used to track variables in transactions. These types cause a typemismatch when used together with standard types, thus making the librarymore complex to use. This issue is partly resolved by providing implicit typeconversion on the STM types, but the implicit conversion is not possible in allcases. In the cases where it is not, the programmer must access the wrappedvalue by the Value property available on all STM types.

AC# introduces additional language constructs which reduces the simplicity ofthe language. It does however increase the simplicity of using the STM part,e.g. using retry outside of an atomic block will result in a compile timeerror. Additionally, atomic can also be used along with fields, local variables,and parameters, making them traceable in transactions without having touse specific STM types. This simplifies interaction between code that usesatomic variables and code that does not.

Listing 8.4: ConcurrentHashMap Add Method - Locking

1 public override void Add(K key, V value)2 {3 var hashCode = GetHashCode(key);4 lock (_locks[GetLockIndex(hashCode)])5 {6 var bucket = _buckets[GetBucketIndex(hashCode)];7 var node = FindNode(bucket, key);8

9 if (node != null)10 {11 //If node is not null, key exist in map. Update the

value12 node.Value = value;13 }14 else15 {16 //Else insert the node17 bucket.AddFirst(CreateNode(key, value));18 lock (_sizeLock)19 {20 _size++;21 }22 }23 }24

25 ResizeIfNeeded();26 }


Listing 8.5: ConcurrentHashMap Add Method - AC#

1 public override void Add(K key, V value)2 {3 atomic4 {5 var bucketIndex = GetBucketIndex(key);6 //TMVar wrapping the immutable chain list7 var bucketVar = _buckets[bucketIndex];8 var node = FindNode(key, bucketVar.Value);9

10 if (node != null)11 {12 //If node is not null key exist in map. Update the

value13 node.Value = value;14 }15 else16 {17 //Else insert the node18 bucketVar.Value = bucketVar.Value.Add(CreateNode(key,

value));19 _size++;20 ResizeIfNeeded();21 }22 }23 }

The placement of the three concurrency approaches on the low simplicity -high simplicity spectrum is depicted in Figure 8.4.

High simplicityLow simplicity

Locking STM Library AC#

Figure 8.4: Concurrency approaches on the low - high simplicity spectrum

8.4.4 Low or High Orthogonality

As described in Section 2.1, locking encompasses a number of basic constructsaiding in di↵erent concurrency scenarios. These constructs can be combined tohandle complex concurrency issues. Some of these combinations are howevererroneous, producing hard to debug problems such as deadlocks. Locking doesnot put any restraints on the language features with which it can be combinedand may therefore seem to be highly orthogonal. The risk of deadlocks when


combining lock-based implementations or employing multiple locks limit theorthogonality, keeping locking from reaching the high orthogonality extreme.Locking in C# provides no solutions to these issues. As a result locking in C#is placed between the middle and the high orthogonality extreme.

STM removes the issue of deadlocks and allows STM based code segments to becombined using transactional nesting. STM combines poorly with irreversibleactions, that is actions which cannot be rolled back in case a transaction abortssuch as IO. Neither the STM library nor AC# o↵ers a solution to this problemsignificantly reducing their orthogonality. Furthermore both the STM libraryand AC# have problems combining with existing data structures reducingtheir orthogonality further.

As described in Section 8.4.1 the programmer cannot rely on implicit conversionfrom an STM object to the value it wraps in all cases, reducing the orthogonalityof the STM library. The STM library however benefits from the advantagesprovided by STM, but is hampered by the not being able to combine withirreversible actions and existing data structures. Consequently library-basedSTM is placed just above the middle on the low - high orthogonality spectrum.AC# treats transactional variables as an object of their original type, removingthe type mismatch between the regular types and the corresponding STM types.As a result, AC# allows transactional variables to be compared directly andmethods can be invoked directly on the transactional variable, without havingto reason about implicit conversion or manually accessing the Value property,as described in Section 8.4.1. Therefore AC# is placed further towards thehigh orthogonality end of the spectrum than library-based STM. AC# howeverremain more towards the low orthogonality extreme than locking.

The placement of the three concurrency approaches on the low - high orthogo-nality spectrum is depicted in Figure 8.5.

High orthogonalityLow orthogonality

Locking

AC#

STM Library

Figure 8.5: concurrency approaches on the low - high orthogonality spectrum


8.4.5 Low or High Readability

As described in Section 8.4.2, locking in C#, except the lock statement, isprovided by library calls, and library STM is only provided by library calls.This negatively a↵ect their readability because these calls blends in withother library code. To avoid this confusion AC# uses special keywords whichpositively a↵ects its readability, however it also negatively a↵ects simplicity andthereby the readability because the programmer has to know these keyword.Also the atomic keyword has di↵erent meanings depending on the context.This negatively a↵ects the readability, as the programmer must be aware ofthese di↵erent meanings. Much of the boilerplate code necessary in the STMlibrary is abstracted away in AC#, e.g. use of the Value property, whichimproves its readability as there is less code to reason about.

In Section 8.4.1 the data types characteristic is described. The STM librarytreats transactional variables as special types, where AC# treats transactionalvariables as regular types with an associated atomic keyword modifier. As aresult it is not possible to define an array or list of transactional types directlyin AC#, which reduces its readability, because of the extra boilerplate codethat the required wrapper class introduces. Additionally the STM libraryallows for implicit conversion when reading a transactional value, therebyavoiding the specification of the Value property, which positively a↵ects thereadability. Implicit conversion can however produce incorrect results and isnot applicable in all scenarios, so ultimately the readability su↵ers.

In Section 8.4.3 the simplicity of locking is placed close to the low simplicityextreme, where the STM library is placed a bit higher than the middle of thespectrum and AC# even higher. In Section 8.4.4 the orthogonality of lockingresides between the middle and high end of the spectrum, while library-basedSTM is placed just above the middle of the spectrum with AC# just aboveit. These characteristics directly influence the readability of the concurrencyapproaches.

All the concurrency approaches have to explicitly mark critical regions of codein order to ensure a program is race condition free. This negatively a↵ectstheir readability, as by reading the program it is hard to know whether criticalregions are marked correctly everywhere, and it is especially hard in largeprograms.

The basic lock construct and idea behind locking is simple, if a resource islocked, only a single thread is allowed access to the resource. This positivelya↵ects the readability of locking and in small code segments it may seem highlyreadable. However locking su↵ers in particular from the concurrency issue ofdeadlocks. This negatively a↵ects the readability of locking drastically, as inorder to reason about deadlocks, the programmer has to reason about every


code segment where locking is applied and how these segments interact. Aslocking code can be fragmented it can become hard for the programmer toreason about. Additionally, if a program makes use of other libraries thatuses locking internally, the programmer also have to reason about the lockscontained within those libraries, as locks do not compose. The readability oflocking is therefore placed towards low readability on the spectrum.

The STM approaches remove the issue of deadlocks, which positively a↵ectsthe readability, as the programmer does not have to reason about deadlocks.Additionally, STM also removes the issue of composing synchronized codesegments, as STM allows nested transactions. Similar to locking, STM su↵ersfrom code fragmentation which can make it hard to get an overview of allthe STM synchronization in a program, which a↵ects the readability nega-tively. Furthermore understanding the concept of memory transactions can,as described in [42], at first be hard to grasp, as it promotes a new way ofsynchronizing programs. This problem is worsened since both the orelseand retry constructs have been included in the STM library and AC#.Additionally, AC# is able to produce compile time errors and warnings, e.g.when a retry keyword is defined outside an atomic scope, which improves itsreadability over the STM library, which does not have this capability. LibrarySTM have a type mismatch between STM types and regular types, which AC#does not. Therefore AC# has higher readability than the STM library.

Ultimately the STM library is placed close to the middle on the spectrumand AC# further towards the high readability extreme. The main drawbackfor both approaches, is that the programmer has to mark critical regions ofcode which can be very fragmented. Additionally understanding the conceptof memory transactions may at first be hard, especially as both the orelseand retry constructs have been introduced.

The placement of the three concurrency approaches on the low readability -high readability spectrum is shown in Figure 8.6.

High readabilityLow readability

Locking

AC#

STM Library

Figure 8.6: Concurrency approaches on the low - high readability spectrum


8.4.6 Low or High Level of Abstraction

Locking is tightly coupled with the hardware architecture through hardwareinstructions such as Test-and-set and Compare-and-swap[71, p. 1990]. Fur-thermore the low-level abstraction of threads isused in order to introduceconcurrency. Additionally the programmer has to state exactly how synchro-nization should be applied using locks. C# o↵ers the lock statement whichprovides a small abstraction over the release of a lock. The lock statement ishowever not applicable in all cases. If for example a timeout on the acquisitionof a lock is required or a more advanced form of lock, such as a semaphore, isrequired, the lock keyword is not applicable. That is the case in the lockingDining Philosophers implementation, which requires the second lock to betaken with a timeout. Furthermore the locking Santa Claus implementationuses semaphores to signal between Santa, the elfs and the reindeer. The lockkeyword does not provide such capabilities. Additionally correct usage ofthe Monitor class leads to the use of a try/finally block to ensure thatthe acquired lock is released in case of an exception[72], reducing the levelof abstraction. Ultimately locking in C# is placed close to the low level ofabstraction extreme of the spectrum with the small abstractions providedkeeping it from being at the extreme.

The STM approaches also rely on threads, but provide a higher level ofabstraction for synchronization. STM uses transactions, where code segmentsare marked and the details of how synchronization is achieved are abstractedaway by the STM system. Both AC# and the STM library facilities strongatomicity, as described in Section 4.3 and the STM system will thereforemanage both transactional and non-transactional access, thus keeping thelevel of abstraction high. Based on the above, the general STM approach liesbetween the middle and high end of the spectrum, where the main drawbacksis that the programmer still has to manage threads and mark regions of codethat should be synchronized.

Some di↵erences in the level of abstraction exist with regards to AC# andthe STM library. As described in Section 8.4.2 the STM library requires theprogrammer to use static method calls and lambdas for defining transactions.AC# substitutes this with language based support for the atomic block. List-ing 8.6 and Listing 8.7 present an implementation of the SleepUntilAwokenmethod from the library and AC# Santa Claus problem implementations,respectively. The method ensures that Santa sleeps until he is awoken byeither three elfs at his door, or all reindeers back from vacation, ready to flyhis sleigh. The return statement on line 3 of Listing 8.6 is not present inthe AC# implementation as it is abstracted away by AC#. Furthermore thestatic method call and creation of lambas to represent transaction bodies online 3 is abstracted away by the atomic block and orelse block on lines 3 and11 of Listing 8.7.


Listing 8.6:SleepUntilAwoken Method - STM Library

1 private WakeState SleepUntilAwoken()2 {3 return STMSystem.Atomic(() =>4 {5 if (_rBuffer.Count != SCStats.NR_REINDEER)6 {7 STMSystem.Retry();8 }9 return WakeState.ReindeerBack;

10 },11 () =>12 {13 if (_eBuffer.Count != SCStats.MAX_ELFS)14 {15 STMSystem.Retry();16 }17 return WakeState.ElfsIncompetent;18 });19 }

Listing 8.7:SleepUntilAwoken Method - STM Language

1 private WakeState SleepUntilAwoken()2 {3 atomic4 {5 if (_rBuffer.Count != SantaClausProblem.NR_REINDEER)6 {7 retry;8 }9 return WakeState.ReindeerBack;

10 }11 orelse12 {13 if (_eBuffer.Count != SantaClausProblem.MAX_ELFS)14 {15 retry;16 }17 return WakeState.ElfsIncompetent;18 }19 }

The STM library exposes transactional variables as special STM types, whereAC# instead allows variables to be marked as atomic, letting the programmerpreserve the original type. AC# is therefore considered to have a higher levelof abstraction than library STM. As described previously STM howevercombines poorly with irreversible actions leaving it up to the programmerto correctly handle actions such as IO in combination with transactions. Asneither the STM library nor AC# can abstract over these problem, their level


of abstraction is reduced. Therefore library-based STM is placed just abovethe middle of the spectrum while AC# resides further towards the high levelof abstraction extreme.

The placement of the three concurrency approaches on the low level of abstrac-tion - high level of abstraction spectrum is depicted in Figure 8.7.

High level of abstractionLow level of abstraction

Locking AC#STM Library

Figure 8.7: Concurrency approaches on the low - high level of abstractionspectrum

8.4.7 Low or High Expressivity

The locking constructs in C# present di↵erent ways to express exactly howsynchronization should be applied, thereby a↵ecting the expressivity positively.Since locking is prone to a number of concurrency related issues, the expres-sivity is severely reduced as the programmers focus is limited by the complexsynchronization form. Locking in C# gives the programmer control over manylow level details concerning how synchronization is applied but at the sametime requires the programmer to specify these details. This along with therisk of deadlocks and other issues limits how the programmer can expressfunctionality concisely and conveniently. Locking is therefore placed towardslow expressivity, being drawn a bit towards the middle of the spectrum becauseof the many choices in locking constructs C# o↵ers.

STM also builds upon threads which limits its expressivity. Memory trans-actions however represent a more declarative and expressive approach thanlocking, as the programmer only has to specify critical regions, allowing theSTM system to manage the details of how synchronization is applied. This af-fects the expressivity positively, as it accomplishes a great deal of computationwith little code. Furthermore STM eliminates the issues of deadlocks whichmakes it more convenient to express synchronization, as the programmer doesnot have to reason about it. However, neither in the STM library nor AC# aconvenient way to express irreversible actions exists, such as exceptions andIO, the programmer is left alone in ensuring these actions work correctly withtransactions, which negativity a↵ects the expressivity. Furthermore, having torely solely on transactional or immutable data structures limits the expressivity


of the STM based approaches. Based on the above, the general STM approachlies between the middle and high end of the spectrum.

Some di↵erences with regards to expressivity between library-based STM andAC# exist. The expressivity of the approaches is closely related to theirlevel of abstraction, described in Section 8.4.6. As described in Section 8.4.2,library STM requires the programmer to put extra e↵ort into expressing theintended functionality e.g. wrapping transactional code in lambdas, wrappingtransactional variables in TMVar types and using the Value property whengetting or setting a value on a transactional variable. That is abstracted away inAC# which makes it more expressive as it provides a more convenient and lesstedious way of specifying computations. However, the abstraction that AC#provides, disallows the programmer from defining an array of transactionaltypes directly and instead requires the programmer to use a wrapper class, asdescribed in Section 8.4.1. Based on the above, AC# is considered to have ahigher expressivity than library STM because of the higher level of abstractionthat it facilitates. The di↵erence is however limited as AC# does not allow forthe definition of an array or list of transactional types as described above.

The placement of the three concurrency approaches on the low expressivity -high expressivity spectrum is depicted in Figure 8.8.

High expressivityLow expressivity

Locking AC#STM Library

Figure 8.8: Concurrency approaches on the low - high expressivity spectrum

8.4.8 Low or High Writability

As discussed in Section 8.4.6 locking in C# has a low level of abstractiondue to its usage of low level constructs. The lock statement provides anabstraction over the acquisition and release of a lock, providing increasedwritability in cases where it is applicable. The writability of locking in C# ishowever reduced as a result of the low level of abstraction. The STM library’slevel of abstraction is placed just above the middle towards the high level ofabstraction end of the spectrum, while AC# has a higher level of abstractionthan the STM library. This positively impacts the writability of the STMlibrary and AC#.


The locking constructs described in Section 2.1 are prone to common lockingproblems, such as deadlocks, resulting in a low simplicity score which negativelyimpacts its writability. The expressivity of the STM library resides above themiddle of the spectrum, reduced by the need to wrap transactional code inlambdas, wrap transactional variables in TMVar types and use the Valueproperty when getting or setting a value on a transactional variable, hinderingthe programmer from concisely expressing the intended behavior, resultingin reduced writability. AC# removes this burden from the programmer, bydelegating the work to the compiler, resulting in a higher expressivity scoreand improved writability.

With respect to simplicity, locking in C# is, as described in Section 8.4.3,placed close to the low simplicity end of the spectrum mainly due the number oferrors that can arise. This negatively impacts the approach’s writability. Thetwo STM based approaches benefit from a declarative approach to definingsynchronization. The STM library is placed just above the middle of thespectrum, while AC# is placed towards the high simplicity end of the spectrum.AC# is given a higher simplicity score due to its cleaner syntax and the abilityto use existing types instead of the STM equivalents, resulting in a greaterpositive impact on writability.

As described in Section 8.4.4 locking in C# has orthogonal properties but it islimited due to the risk of errors when combining locking constructs. The STMbased approaches benefit from simplified composition based on transactionalnesting but simultaneously introduces issues with irreversible actions suchas IO as well as existing data structures. The STM library has problemscombining with existing types in all cases. Furthermore the implicit conversionfeature cannot be relied on in all cases, requiring the programmer to access theValue property, reducing the orthogonality of the STM library. Ultimatelythis results in the STM library being placed just above the middle on the low -high orthogonality spectrum. AC# addresses many of the issues present inthe STM library by allowing the programmer to use any existing types insteadof the STM types and by delegating much of the work required by the STMlibrary to the compiler. This results in AC# being placed just above the STMlibrary on the low - high orthogonality spectrum. Locking is however placedfurther towards the high orthogonality extreme due to its ability to correctlycombine with most language constructs, excluding locking itself.

Locking su↵ers from the risk of a number of serious errors, resulting in itbeing hard to use in complex scenarios, limiting its writability. The STMbased approaches solve many of the errors present with locking but introducesnew problems with irreversible actions and data structures. The STM libraryhas problems with types, accessing the wrapped value, and the syntax oftransaction definitions, which limits its writability. AC# addresses many ofthe writability problems present in the STM library by delegating the work


to the compiler. Both STM based approaches however require that datatypes, including collections, are constructed using transactional variables orimplemented as immutable in order to interact with the STM system, reducingtheir writability. Based on these observations and the evaluations of theprevious characteristics, the three approaches have been placed on the low -high writability spectrum as depicted in Figure 8.9.

High writabilityLow writability

TL

AC#

STM Library

Figure 8.9: Concurrency approaches on the low - high writability spectrum

9 ConclusionThis thesis tests the hypothesis: “Language integrated STM provides a validalternative to locking in terms of usability, and provides additional benefitscompared to library-based STM, when solving concurrent problems in C#”. Inorder to do so an extension to C#, called AC# was designed and implemented.AC# provides language integrated support for STM which allows programmersto utilize transactions alongside existing C# features. The STM systempowering AC# was implemented as a C# library. To implement AC#, theRoslyn C# compiler was extended to transform AC# source code into regularC# code, which utilizes the aforementioned STM library to execute any definedtransactions. For each of the concurrency approaches: AC#, the STM libraryand locking in C#, implementations of the Dining Philosophers problem, theSanta Claus problem, a concurrent queue, and a concurrent hashmap werecreated. These implementations served as the basis for a usability evaluationaccording to an extended version of the characteristics defined in our previousstudy[3].

In order to develop AC# a set of requirements were defined, detailing howthe underlying STM system should behave in relation to for example trackinggranularity, atomicity level and nesting. Based on the requirements AC#was designed. The design includes a description of new language constructsas well as a description of modifications to existing language features. Anumber of STM implementations were investigated. Based on this investigationas well as the requirements and design, an STM library, utilizing the TLIIalgorithm, was implemented. The STM library was tested using a numberof unit tests, ensuring that transactions are executed correctly. In order toperform the actual integration of STM into C#, the open source Roslyn C#compiler was extended. This required a deep knowledge of the Roslyn projectand its structure, which initiated an investigation of Roslyn. Due to thelimited availability of literature with regards to Roslyn, much of the knowledgeobtained in this area was acquired by reading and debugging Roslyn’s sourcecode. A number of unit tests were defined for the Roslyn extension, in order toensure that all STM constructs and the integration with the existing languagefeatures work correctly. Finally AC#, the STM library and locking in C# wereevaluated according to a number of usability related characteristics, facilitatinga conclusion upon the hypothesis.

This chapter addresses the problem statement questions and summarizes theevaluation of the characteristics in order to conclude on our hypothesis.

108

CHAPTER 9. CONCLUSION 109

9.1 Problem Statement Questions Revisited

The goal of this master thesis was to investigate the usability of languageintegrated support for STM in C#, compared to a library-based solution andexisting locking features. This section will address the problem statementquestions used for structuring our investigation.

Selecting features to an STM system for C# was done by analyzing di↵er-ent approaches to tracking granularity, transactions & variables, atomicity,side-e↵ects, conditional synchronization, nesting, and opacity. These featuresdetermined the requirements for the design. The requirements and the existinglanguage features of C# were taken into account in the design and integra-tion process of AC#, identifying how to make an integration encompassing:atomic blocks, transactional variables, parameters & arguments, conditionalsynchronization, and nesting. The implementation strategies McRT, TLII, andJVSTM were analyzed based on the criteria from the requirements. TLII wasselected based on its fit with strong atomicity and retry, as well as being welldocumented. To find out how the Roslyn compiler was structured, a review ofwhitepapers, blogs, forum posts, and the source code were conducted. Thisresulted in a description of Roslyns compiler phases, syntax tree, and API.This knowledge allowed us to extend Roslyn to encompass the features ofAC#, by modifying its lexer, parser, syntax tree and symbols. Additionally,the Roslyn compiler was extended with a transformation phase, transformingAC#’s constructs to ordinary C# code, allowing the remaining compilationphases to be reused.

9.2 Hypothesis Revisited

With a functioning AC# language, library-based STM, and locking in C#, weevaluated the concurrency approaches based on cases representing di↵erentaspects of concurrency. Based on this evaluation, we conclude on the hypothesisin Section 9.2.1.

STM has a more implicit degree of concurrency than locking, as synchroniza-tion details are abstracted away. This also makes library-based STM morefault restrictive than locking as synchronization details are handled correctly bythe underlying STM system, hindering potential errors known from locking e.g.deadlocks. AC# is even more fault restrictive, since static analysis identifies anumber of errors. Both locking and STM use shared memory for communi-cation, and require the programmer to specify the scope of synchronizationreducing their readability and writability.

The optimistic nature of STM relies on conflicts being uncommon, correctingthem if they occur. The correction results in aborting a transaction, andrunning it again. Thus all e↵ects occurring in transactions must be able to


be discarded in case of an abort. This is however not the case for irreversibleactions, decreasing the usability of STM, as the programmer is required toknow which operations cannot be used in transactions.

From the programmer’s point of view AC# does not change the type of trans-actional variables, which increases its readability. This is a benefit comparedto the library-based STM approach, which requires transactional variables tobe wrapped in TMVar objects. The programmer can for example comparetwo transactional variables directly in AC#, where in the STM library she isrequired to compare the Value properties of the TMVars instead. FurthermoreAC# and the STM library require data types, including collections, to bebuild using transactional variables which lowers the writability of the two STMbased approaches.

AC# allows the programmer to define transactional variables by supplyingthe atomic modifier. As a consequence an array of transactional variablescannot be defined in AC#. Instead the programmer must either wrap eachelement in an object with a transactional field, employ an immutable list, oruse a specialized STM list that tracks elements internally. These workaroundsdo however reduce both readability and writability, which negatively a↵ectsthe usability of AC#.

AC# integrates STM at a level similar to that of locking in C#, providinga number of benefits compared to library-based STM. The STM relatedkeywords supplied by AC# increases its readability as the intent stands outclearly. However, simplicity is reduced, as special keywords add complexity tothe language. The syntax of AC# is improved compared to library-based STM,as boilerplate syntax is abstracted away. This is a benefit to AC# comparedto library-based STM.

The declarative synchronization approach in STM provides high simplicity, ahigh level of abstraction and high expressivity. Locking requires the programmerto explicitly control synchronization details, which is hard to get right. STMon the other hand handles synchronization as long as the critical regions arespecified, which improves usability. Additionally AC# provides static analysis,capable of identifying errors in the code, a feature not present in the STMlibrary.

The locking constructs provided by C# can be combined to handle di↵erentscenarios which improves the orthogonality. However the risk of deadlockswhen combining locks reduces the orthogonality. STM removes the issue ofdeadlocks, and allows transactions to be nested, which is highly orthogonal.A major caveat is, that STM cannot be combined with irreversible actions,keeping the orthogonality, level of abstraction and expressivity of the STMbased approaches from reaching their potential, impacting both readability


and writability. Library-based STM supplies implicit conversion from an STMobject to its wrapped value. However certain cases, e.g. comparisons, requireexplicitly accessing the Value property of the STM object. AC# does nothave the type mismatch between regular types and the corresponding STMtypes. Due to this, no extra actions must be taken when retrieving the valueof a transactional variable. This provides a benefit to AC# over library-basedSTM, as it improves writability.

9.2.1 Final Conclusion

Based on the above findings we conclude that language integrated STM providesa valid alternative to locking when solving concurrency problems. This is basedon it being evaluated as having more implicit concurrency, being more faultrestrictive, being simpler, having a level of abstraction, expressivity, readability,and writability. Locking does however have higher orthogonality due to STM’sproblems combining with irreversible actions and existing data structures,reducing the applicability of AC#.

Furthermore, AC# provides additional benefits over language integrated STM.This is based on being evaluated as equal in implicit concurrency and op-timism, while having improved readability and writability based on a lessintrusive syntax design, higher simplicity, orthogonality, level of abstraction,and expressivity.

10 ReflectionThis chapter presents a reflection on the decisions made throughout the report.Section 10.1 presents a reflection on the findings of the preliminary analysis.Section 10.2 describes reflections related to transactional local variables andparameters as well as ref/out arguments and parameters. Reflections relatedto the STM library, described in Chapter 7, are presented in Section 10.3.Finally, the reflections related to the Roslyn extension and C# 6.0 are coveredin Section 10.4 and Section 10.5 respectively.

10.1 Preliminary Investigation

A preliminary investigation was conducted, in order to reduce the risk associatedwith the Roslyn compiler and the implementation of an STM system.

A number of papers describing STM system implementations were investigated.This gave us a deeper understanding of the known approaches for implementingSTM systems. To get some hands on experience, a prototype STM systembased on the two implementations described in [53, p. 424] was developed.Developing such a prototype system furthered our understanding of the di↵erentimplementation strategies.

As the Roslyn compiler was released 10 months prior to the start of the project,the amount of literature on the subject is limited, consisting of mostly: a whitepaper[30], documentation associated with the Roslyn Github repository[73],sample implementations and walkthroughs[74], as well as blog posts. Mostof these sources describe the compiler API as opposed to the structure ofthe compiler. To further our understanding of the Roslyn compiler we haveinvestigated these sources. In addition, an exploratory modification of thecompiler was done to investigate its structure. The lexer and parser of thecompiler were modified to handle the syntax of an atomic block. Theexploratory implementation furthered our knowledge of the compilers structureand API design, and served as an onset into the deeper exploration of thecompiler documented in Chapter 3.

Getting preliminary hands on experience with the required technologies at anearly stage of the project, helped ensure the feasibility for the implementationsand provided the project group with confidence in the feasibility of the project.Bringing more information to the table at an early stage assisted in decidingon the implementation strategies for the STM system and for the extension ofthe Roslyn compiler.

112

CHAPTER 10. REFLECTION 113

10.2 Design

These sections describes the project group’s reflections over areas related tothe design of AC# described in Chapter 5.

10.2.1 Transactional Local Variables & Parameters

In order to improve the orthogonality and flexibility of AC# the design allowsfor the declaration of transactional local variables and parameters. Neitherof these features have however been used in the implementations on whichthe evaluation is build, indicating that the features may not be required.The reason may be that the problems selected in this thesis, described inSection 1.5.1, does not fit well with the use of transactional local variables andparameters, where other types of problems and applications may fit better. Itmay also be that the way we have implemented the selected problems did notpresent a reason to use transactional local variables and parameters, whereother ways may have.

The initial idea behind allowing transactional local variables was that localvariables can be captured in a lambda expression which is executed by multiplethreads, resulting in the need for synchronization. The option to do sowas however not employed in the evaluation implementations. Allowing theprogrammer to declare atomic local variables instead of relying solely onatomic fields makes AC# more orthogonal but also less simple.

A parameter is similar to a local variable which is initialized using its corre-sponding argument[52, p. 76]. Therefore the idea behind allowing transactionalparameters is similar to that of transactional local variables. If transactional pa-rameters were not allowed, the programmer would have to declare an atomiclocal variable and initialize it to the value of the parameter, if she wished toutilize the parameter as a variable in a transaction. As with transactionallocal variables, transactional parameters improve the orthogonality of AC#but reduce the simplicity.

C# does not allow for the declaration of volatile local variables andparameters, but only volatile fields. Similarly allowing only atomicfields may be su�cient for AC#. Determining whether these features aretruly required, through further studies e.g. usability studies, could assist infurthering AC# as well as aid others seeking to supply language integratedSTM.

10.2.2 Transactional Ref/Out Parameters & Arguments

The initial idea behind including transactional ref/out parameters was topreserve the functionality provided by C# with regards to transactional argu-ments and parameters, thereby providing an integration with existing language


Listing 10.1: STM library interfaced based on applying functions

1 public class Account2 {3 private double _balance;4

5 public void AtomicTransfer(Account other, double amount)6 {7 STMSystem.Atomic(() =>8 {9 var newBalance = STMSystem.TMRead(ref _balance)+amount;

10 STMSystem.TMWrite(ref _balance, newBalance);11 newBalance = STMSystem.TMRead(ref other._balance)-amount;12 STMSystem.TMWrite(ref other._balance, newBalance);13 });14 }15 }

features. As with transactional local variables and parameters, transactionalref/out parameters were however not used in the implementations for theevaluation, suggesting that further work is required to determine whether thefeature is required or if it simply complicates AC#, reducing its simplicity.

As described in Section 7.5.3, providing the intended semantics for transactionalref/out parameters & arguments proved problematic due to the types oftransactional variables being substituted with their corresponding STM types.As a result the simplicity of AC# is reduced due to the programmer having toreason about the changed semantics.

To remove the problem, a library with an interface based on central metadataand applying functions to variables in order to read/modify could be adopted.Such an interface will remove the need for transforming the types duringcompilation, allowing transactional variables to be passed directly into non-transactional parameters and vice versa. If a transactional variable is passedby ref/out into a method, transactional access will however still be lost, as thebody of the method does not treat the variable as transactional. As describedin Section 7.5.3 C# utilizes a similar approach for volatile variables passedby ref/out.

Listing 10.1 presents an example of how an STM interface based on applyingfunctions to variables could be designed. The STMSystem.TMRead andSTMSystem.TMWrite are applied to the balance variables in order toread and write their associated value.


10.3 STM Implementation

This section describes the project groups reflections related to the implementa-tion of the STM system described in Chapter 6.

10.3.1 STM Algorithm

The implemented STM system utilizes the TLII algorithm[12]. TLII usescommit time locking to ensure that any values written by a transaction becomesvisible to the remaining system as a single atomic step. The TLII algorithmis well documented [12][53, p. 438][9, p. 106], which allowed us to gain deepinsight into the workings of the algorithm, easing the implementation of moreadvanced features such as orelse, retry and nesting, which are not described inthe reference materials.

Due to the utilization of locking the TLII algorithm is unable to supply anyof the progress guarantees described in Section 6.1.4. Selecting an algorithmwhich could supply one of these progress guarantees would have positivelya↵ected the simplicity of the STM library and of AC#. In case of Wait-freedomthe programmer would know that all transactions would eventually commitallowing her to utilize this knowledge in the implementation. Alternativelythe inclusion of a contention manager, dictating conflict resolution, into theexisting implementation could be investigated. A contention manager mayallow the existing implementation to provide similar guarantees in the absenceof failures.

10.3.2 STM Library Interface

As described in Section 1.5, locking in C#, library based STM and AC#is evaluated according to a set of characteristics. As the STM library wasevaluated according to its usability, its interface was required to be designedto maximize usability, while still satisfying the defined requirements. AC#however does not impose such a requirement on its backing STM library. Theprogrammer does not see the transformed code, removing the need for an STMlibrary with high usability. In fact it may be advantageous for performanceto utilize a backing STM library with a high degree of flexibility, allowingthe compiler to optimize the generated code. As described in Section 10.2.2adopting a library interface based on central metadata and applying functionsto a variable in order to read and modify, will remove the issues related to thetypes of transactional variables and ref/out parameters.

10.4 Roslyn Extension

As described in Section 7.1 the Roslyn compiler was extended at the levelof the syntax tree/symbol information. The lexer, parser, syntax tree, and


symbol table were modified to support the constructs described in Chapter 5,and transformations were applied to the modified syntax tree, producing astandard C# syntax tree that represents the execution of transactions as callsto the STM library. The transformed syntax tree is passed to the remainingcompiler phases, utilizing the existing semantic analysis and code generationof the Roslyn compiler.

Alternatively the transformation could be applied at the level of what in Roslynis referred to as the bound tree. The bound tree is a semantic representation ofthe source code in the form of a tree structure. The bound tree is constructedbased on the syntax tree and is the basis for the code generation phase.During the construction of the bound tree, semantic analysis and source codetransformation is applied. As described in Section 7.1 applying transformationsto the syntax tree causes it to lose its roundtripable property, due to the syntaxtree no longer representing the original AC# source code. The transformationsrequired for executing transactions could be applied along with the standardC# transformations, as part of constructing the bound tree. This wouldcause the syntax tree to preserve its roundtripable property but will howeverrequire the implementation of the transformation to be mingled with theexisting C# transformation implementations and semantic analysis, makingthe implementations more complex. Alternatively the transformations couldbe applied after the bound tree has been constructed. However this approachrequires that the logic which constructs the bound tree, as well as the semanticanalysis, is modified to support the STM constructs described in Chapter 5.The bound tree and the associated transformation and analysis implementationsare not part of the public API provided by the Roslyn compiler due to it oftenundergoing significant changes[37]. Furthermore any documentation is limitedto blog/forum post as well as the comments embedded in the source code.Due to these factors, applying transformations at the level of the boundtree is significantly more complex than the approach utilized in Section 7.1.Furthermore, it may require the transformation implementation to be rewrittenif AC# is moved to a newer version of C# and the bound tree has undergonesignificant changes since the previously utilized implementation was created.

10.5 C# 6.0

As described in Section 1.3, the development of AC# is based on C# 5, whichat the time of writing is the most recent version of C#. C# 6.0 is, as of May2015, however under active development and a number of new features havebeen shown to the public. This section presents our reflections upon how thenew features impact the findings of this project. The new features supplied byC# 6.0 are listed in [75] and described in [76], [77] and [78]. Only new featuresrelevant to locking in C#, AC# or the STM library are described.


Listing 10.2: STM library with static using statement

1 private WakeState SleepUntilAwoken()2 {3 return Atomic(() =>4 {5 if (_rBuffer.Count != SCStats.NR_REINDEER) {6 Retry();7 }8 return WakeState.ReindeerBack;9 },

10 () =>11 {12 if (_eBuffer.Count != SCStats.MAX_ELFS) {13 Retry();14 }15 return WakeState.ElfsIncompetent;16 });17 }

10.5.1 Static Using Statements

Static using statements allows the programmer to import the static methodsof a particular class, giving her the ability to call these methods as if they weremethods of the class in which the call takes place[76][78]. This feature is ofinterest to programmers utilizing the STM library as it will allow them to callthe atomic and retry methods of the STMSystem class without having tospecify the class for each call, thereby improving the writability of the library.Listing 10.2 shows how the SleepUntilAwoken method of the STM librarybased Santa Claus problem implementation could look if the static methods ofthe STMSystem class were imported through a static using statement.

10.5.2 Auto-Property Initializers

Auto-Property intializers allow the programmer to supply an initializer expres-sion to the definition of an Auto-Property[76][77][78]. This causes the propertyto be initialized using the initializer expression. If the Auto-Property definesonly a getter, the backing field is automatically declared readonly[76][78].

The Roslyn extension will have to be modified to handle atomic Auto-properties with an initializer expression. The extension must transform aninitializer expression to an object creation expression which creates a new STMobject of the STM type corresponding to the type of the atomic property,and initialize it using the defined initializer expression. Instead of initializingthe property directly, the initializer should be applied to the backing fieldgenerated as part of the transformation of atomic auto-properties describedin Section 7.3.3.

11 Future WorkThis chapter presents possibilities for extending upon the work done in thisthesis. Section 11.1 suggests a performance test in order to examine howAC# performs in comparison to locking in C#. Section 11.2 suggests a deepSTM integration in order to achieve better performance. Finally, Section 11.3suggests to investigate possible ways of combining irreversible actions andSTM in the context AC#.

11.1 Performance Test

The focus of this thesis has been on evaluating the usability of languageintegrated STM compared to that of library-based STM and locking. Asdescribed in Section 1.3, the performance of STM has not yet been considered.However having language integrated STM is not su�cient, as concurrencyusually is introduced to achieve good performance. It is essential to know howAC# performs in comparison to locking in C#, since if it performs far worse,it is not a valid alternative, in its current state. A performance test is thereforea candidate for future work, and the outcome may result in changes to thedesign and underlying algorithm of AC#, in order to improve its performance.

To investigate performance, an extensive performance suite with a number ofproblems or algorithms with great diversity could provide valuable insights intohow the three concurrency approaches perform under varied circumstances. Inthis thesis, four concurrency problems have been developed, which advanta-geously could be included in the test suite, by measuring their performance.Furthermore a number of concurrent performance suites already exists, whichcould be investigated in order to help select additional problems. In [79] a per-formance suite, specifically for the actor model is proposed. The performancesuite contains 28 test cases that “range from popular microbenchmarks toclassical concurrency problems to applications that demonstrate various stylesof parallelism” which are “diversive, realistic and compute intensive”. Thedining philosophers and concurrent hashmap1 are also included in this perfor-mance suite. Another benchmark specifically designed for STM is proposedin [80], which uses “a set of workloads that correspond to realistic, complex,object-oriented applications which benefit from multi-threading” to comparethe performance between STM implementations. Comparing AC# to otherSTM implementations, could give valuable insights to evaluate the relationbetween STM design decisions and performance.

1Refered to as Concurrent Dictionary in the article

118

CHAPTER 11. FUTURE WORK 119

11.2 Integration into CLR

Chapter 7 describes how STM is integrated into C# by modifying the Roslyncompiler to transform the source code containing STM language constructs tostandard C# code which utilizes the STM library. As a result no modificationsare required to the CLR as it receives byte code in the standard CIL formatproduced by the Roslyn C# compiler’s code generator.

As an alternative, a deeper integration, similar to that of [11] and [14], couldbe utilized. One approach is to extend the CLR and its associated byte codeformat CIL with support for respectively, the execution and definition oftransactions. Extending CIL with support for the definition of transactionsallow the JIT compiler to apply further optimization to the execution oftransactions. Additionally, if the executing hardware supports it, the CLRcould delegate smaller transactions to the hardware providing a performanceboost.

If CIL is extended with support for the definition of transactions, the Roslyncompiler will have to be extended as well, as the compiler must generate codeutilizing the new CIL instructions. As a result the lexing, parsing, semanticanalysis and code generation of the Roslyn compiler must be extended tosupport the STM constructs. However, the lexing and parsing implemented inthis project can be reused for such an extension.

11.3 Irreversible Actions

As described in Section 1.3, solving the integration problems between transac-tions and irreversible actions, has not been an area of focus for this thesis. Asirreversible actions in the context of STM still contains a number of issues, thearea could benefit from an increased research e↵ort. AC# o↵ers no support ofor warning against irreversible actions in transactions, which hurts its usability,as it leaves it up to the programmer to manually handle it correctly.

The integration between transactions and irreversible actions is therefore acandidate for future work, both in terms of how to manage it in AC#, but alsoin terms of STM in general. A number of possible integration approaches exist,we briefly scratched the surface in our prior study[3, p. 51-52]. One approach,presented in [11, p. 4], disallows the use of native calls inside transactions, byraising a runtime exception. A similar approach is, to enable the developerto mark a function, so the STM system is aware of its side e↵ect. In Clojurethis is possible with the #io macro. If a function is marked, and used in atransaction, a runtime exception will be raised. In [16] Harris et al. proposesanother approach where IO libraries should implement an interface, allowingthe STM system to initiate a callback when the transaction is committed,allowing the e↵ect to be bu↵ered until then. In [14] Du↵y proposes using a

CHAPTER 11. FUTURE WORK 120

well known strategy from the transaction theory[48], having the programmersupply on-commit and on-rollback actions to perform or compensate for theirreversible action. All of these approaches however either impose additionale↵ort unto the programmer or completely disallow irreversible actions. A moreelegant solution will benefit the area of STM in general and may instigate theadaptation of STM as a language feature.

Appendix

121

A Roslyn Compiler CallChainIn Figure A.1 an overview of the compilation call chain is shown. A compi-lation starts with csc’s main method being invoked, which calls the staticRun method on the Csc class. This method creates a new compilerobject of type Csc . Csc is a subtype of CSharpCompiler , locatedin the CSharp.CSharpCodeAnalysis.Desktop project, which again isa subtype of CommonCompiler , located in Core.CodeAnalysis . Thismeans that the creation of the compiler object, calls the constructors of Csc,CSharpCompiler and CommonCompiler. Afterwards the Run method is in-voked on the compiler object, which further invokes its parent’s Run method,which again invokes its parent’s Run method. Finally the RunCore method inCommonCompiler is invoked, which contains the general code that controlsthe overall flow of the compiler pipeline, illustrated in Figure 3.1. The RunCoremethod will, for each phase, call the language specific implementation of thephase, located in either C# in CSharp.CSharpCodeAnalysis.Portableor VB in VisualBasic.BasicCodeAnalysis.Portable , through dy-namic dispatch.

122

APPENDIX A. ROSLYN COMPILER CALL CHAIN 123

Figure

A.1:Sequen

cediagram

show

ingan

overview

ofthecallchainof

aC#

compilation.

B Concurrency ProblemsB.1 Dining Philosophers

The Dining Philosophers problem is defined as follows:

“Five philosophers spend their lives thinking and eating. Thephilosophers share a common dining room where there is a circulartable surrounded by five chairs, each belonging to one philosopher.In the center of the table there is a large bowl of spaghetti, and thetable is laid with five forks (see Figure B.1). On feeling hungry, aphilosopher enters the dining room, sits in his own chair, and picksup the fork on the left of his place. Unfortunately, the spaghetti isso tangled that he needs to pick up and use the fork on his right aswell. When he has finished, he puts down both forks, and leaves theroom. The room should keep a count of the number of philosophersin it.”

– Edgar Dijkstra[81, p. 673]

Figure B.1: The Dining Philosophers

124

APPENDIX B. CONCURRENCY PROBLEMS 125

B.2 The Santa Claus Problem

The Santa Claus problem is defined as follows:

“Santa Claus sleeps in his shop up at the North Pole, and canonly be wakened by either all nine reindeer being back from theiryear long vacation on the beaches of some tropical island in theSouth Pacific, or by some elves who are having some di�cultiesmaking the toys. One elf ’s problem is never serious enough to wakeup Santa (otherwise, he may never

¯get any sleep), so, the elves

visit Santa in a group of three. When three elves are having theirproblems solved, any other elves wishing to visit Santa must waitfor those elves to return. If Santa wakes up to find three elveswaiting at his shop’s door, along with the last reindeer having comeback from the tropics, Santa has decided that the elves can waituntil after Christmas, because it is more important to get his sleighready as soon as possible. (It is assumed that the reindeer don’twant to leave the tropics, and therefore they stay there until thelast possible moment. They might not even come back, but sinceSanta is footing the bill for their year in paradise . . . This couldalso explain the quickness in their delivering of presents, since thereindeer can’t wait to get back to where it is warm.) The penaltyfor the last reindeer to arrive is that it must get Santa while theothers wait in a warming hut before being harnessed to the sleigh.”

– John A. Trono[82]

B.3 Concurrent Queue

A queue is a First In First Out (FIFO) data structure, operating like the queueat a cash register[20, p. 234]. A queue supports the operations Enqueue andDequeue. Enqueue inserts an item at the back of the queue while Dequeuegets and removes the item at the front of the queue, per the FIFO principle.A queue is commonly implemented using a linked list of nodes[20, p. 234]. Tobe able to enqueue and dequeue, the queue holds a reference to the first andlast node in the list, referred to as the head and tail of the queue.

B.4 Concurrent HashMap

A HashMap is a data structure which allows looking up values based on theirassociated key and provides the operations Add, Get, and Remove. It benefitsfrom the Get operation having an average time complexity of O(1) underreasonable assumptions[20, p. 256]. Implementation details of a HashMap

APPENDIX B. CONCURRENCY PROBLEMS 126

may vary, but traditionally they follow the concept described in[20, p. 256].Internally it uses an array for storing a list of key/value pairs. When insertinga value into the HashMap, the array index is commonly calculated as:

Hash(key) mod arraylength (B.1)

That is, a hash function applied to the key, modulo the size of the backingarray. If another key is already stored in the array index calculated, it is saidto be a collision. To handle this, the value is not stored directly in the arrayindex, but in a list referred to as a bucket. This bucket is implemented as alinked list as this ensures fast insertions[20, p. 257]. Index based lookup andremoval is not needed as the index of a particular key is not known. Thereforeit is always required to iterate through the bucket list when searching for aparticular key. The calculation of the bucket index must distribute the valuesevenly to lessen the collisions. When a certain percentage of the buckets arefilled, the internal array must be resized to keep the risk of collisions low. Thisinternal operation is expensive in terms of both time and space, as a newinternal array of an increased size must be allocated, and the key/value pairsmust be inserted into the new array.

C EvaluationImplementationsThis appendix presents the implementations of the selected problems, describedin Section 1.5, on which the evaluation presented in Chapter 8 is based. Alongwith each implementation follows a brief description.

C.1 Lock-Based

This sections presents the lock based implementations of the selected problems.

C.1.1 Dining Philosophers Problem

The lock based implementation of the dining philosophers problem requires theuse of the Monitor class to acquire the second lock with a time out on line35. As a result a try/finally block is used, on lines 33 to 48, to ensuredthat the acquired lock is released in case an error occurs while the lock is held.

Listing C.1: Lock Based Dining Philosophers Implementation

1 public class LockingDiningPhilosophers2 {3 public static void Start()4 {5 var fork1 = new object();6 var fork2 = new object();7 var fork3 = new object();8 var fork4 = new object();9 var fork5 = new object();

10

11 var t1 = StartPhilosopher(fork1, fork2);12 var t2 = StartPhilosopher(fork2, fork3);13 var t3 = StartPhilosopher(fork3, fork4);14 var t4 = StartPhilosopher(fork4, fork5);15 var t5 = StartPhilosopher(fork5, fork1);16

17 t1.Join();18 t2.Join();19 t3.Join();20 t4.Join();21 t5.Join();22 }23

24 private static Thread StartPhilosopher(object left, objectright)

25 {

127

APPENDIX C. EVALUATION IMPLEMENTATIONS 128

26 var t1 = new Thread(() =>27 {28 while (true)29 {30 lock (left)31 {32 var lockTaken = false;33 try34 {35 Monitor.TryEnter(right, 100, ref lockTaken);36 if (lockTaken)37 {38 Console.WriteLine("Thread: " +

Thread.CurrentThread.ManagedThreadId + "eating.");

39 Thread.Sleep(100);40 }41 }42 finally43 {44 if (lockTaken)45 {46 Monitor.Exit(right);47 }48 }49 }50 Thread.Sleep(100);51 }52 });53

54 t1.Start();55 return t1;56 }57 }

C.1.2 Santa Claus Problem

As seen on line 62, the lock Based Santa Claus problem implementation uses asemaphore to allow the elfs and reindeer to wake Santa, given their respectiveconditions are true. When Santa is awoken he must check whether he wasawoken by the elfs or by the reindeer, as shown on lines 66 to 72. As thereindeer must take priority, as defined in Appendix B.2, their condition ischecked in order to determine the action which Santa takes. Similarly, asshown on line 143, the elfs utilize a semaphore to ensure that only three elfsgo to santa at a time. As shown on line 145 to 152 as well as 204 to 211, boththe elfs and the reindeer enqueue themselves and check their condition whileholding a lock on their respective queues, ensuring that only a single threadobserves the condition as true and notifies Santa.

Listing C.2: Lock Based Santa Claus Implementation


1 public class LockingSantaClausProblem2 {3 public static void Start()4 {5 var santaHandle = new SemaphoreSlim(0, 2);6 var sleigh = new SemaphoreSlim(0, SCStats.NR_REINDEER);7 var warmingHut = new SemaphoreSlim(0, SCStats.NR_REINDEER);8 var reindeerDone = new SemaphoreSlim(0, SCStats.NR_REINDEER);9 var elfWaiting = new SemaphoreSlim(0, SCStats.MAX_ELFS);

10 var elfDone = new SemaphoreSlim(0, SCStats.MAX_ELFS);11 var maxElfs = new SemaphoreSlim(SCStats.MAX_ELFS,

SCStats.MAX_ELFS);12 var rBuffer = new Queue<LockingReindeer>();13 var eBuffer = new Queue<LockingElf>();14 var santa = new LockingSanta(rBuffer, eBuffer, santaHandle,

sleigh, warmingHut, reindeerDone, elfWaiting, elfDone);15 santa.Start();16

17 for (var i = 0; i < SCStats.NR_REINDEER; i++)18 {19 var reindeer = new LockingReindeer(i, rBuffer, santaHandle,

sleigh, warmingHut, reindeerDone);20 reindeer.Start();21 }22

23 for (var i = 0; i < SCStats.NR_ELFS; i++)24 {25 var elf = new LockingElf(i, eBuffer, santaHandle, maxElfs,

elfWaiting, elfDone);26 elf.Start();27 }28 }29 }30

31 public class LockingSanta : IStartable32 {33 private readonly Queue<LockingReindeer> _rBuffer;34 private readonly Queue<LockingElf> _eBuffer;35 private readonly SemaphoreSlim _santaHandle;36 private readonly SemaphoreSlim _sleigh;37 private readonly SemaphoreSlim _warmingHut;38 private readonly SemaphoreSlim _reindeerDone;39 private readonly SemaphoreSlim _elfsWaiting;40 private readonly SemaphoreSlim _elfsDone;41

42 public LockingSanta(Queue<LockingReindeer> rBuffer,Queue<LockingElf> eBuffer, SemaphoreSlim santaHandle,

43 SemaphoreSlim sleigh, SemaphoreSlim warmingHut,SemaphoreSlim reindeerDone, SemaphoreSlimelfsWaiting, SemaphoreSlim elfsDone)

44 {45 _rBuffer = rBuffer;46 _eBuffer = eBuffer;47 _santaHandle = santaHandle;


48 _sleigh = sleigh;49 _warmingHut = warmingHut;50 _reindeerDone = reindeerDone;51 _elfsWaiting = elfsWaiting;52 _elfsDone = elfsDone;53 }54

55 public Task Start()56 {57 return Task.Run(() =>58 {59 while (true)60 {61 //Santa is resting62 _santaHandle.Wait();63

64 var wakeState = WakeState.ElfsIncompetent;65

66 lock (_rBuffer)67 {68 if (_rBuffer.Count == SCStats.NR_REINDEER)69 {70 wakeState = WakeState.ReindeerBack;71 }72 }73

74 switch (wakeState)75 {76 case WakeState.ReindeerBack:77 Console.WriteLine("All reindeers are back!");78

79 //Release reindeers from warming hut80 _warmingHut.Release(SCStats.NR_REINDEER);81

82 //Setup the sleigh83 _sleigh.Release(SCStats.NR_REINDEER);84

85 //Deliver presents86 Console.WriteLine("Santa delivering presents");87 Thread.Sleep(100);88

89 //Release reindeer90 _rBuffer.Clear();91 _reindeerDone.Release(SCStats.NR_REINDEER);92 Console.WriteLine("Reindeer released");93 break;94 case WakeState.ElfsIncompetent:95 Console.WriteLine("3 elfs at the door!");96

97 _elfsWaiting.Release(SCStats.MAX_ELFS);98

99 //Answer questions100 Thread.Sleep(100);101


102 //Back to work incompetent elfs!103 _eBuffer.Clear();104 _elfsDone.Release(SCStats.MAX_ELFS);105

106 Console.WriteLine("Elfs helped");107 break;108 }109 }110 });111 }112 }113

114 public class LockingElf : IStartable115 {116 private readonly Random _randomGen = new

Random(Guid.NewGuid().GetHashCode());117 public int ID { get; private set; }118 private readonly Queue<LockingElf> _buffer;119 private readonly SemaphoreSlim _maxElfs;120 private readonly SemaphoreSlim _santaHandle;121 private readonly SemaphoreSlim _waitingToAsk;122 private readonly SemaphoreSlim _doneAsking;123

124 public LockingElf(int id, Queue<LockingElf> buffer,SemaphoreSlim santaHandle, SemaphoreSlim maxElfs,SemaphoreSlim waitingToAsk, SemaphoreSlim doneWaiting)

125 {126 _buffer = buffer;127 ID = id;128 _maxElfs = maxElfs;129 _santaHandle = santaHandle;130 _waitingToAsk = waitingToAsk;131 _doneAsking = doneWaiting;132 }133

134 public Task Start()135 {136 return Task.Run(() =>137 {138 while (true)139 {140 Thread.Sleep(100 * _randomGen.Next(21));141

142 //Only a fixed amount of elfs can go to santa at a time143 _maxElfs.Wait();144

145 lock (_buffer)146 {147 _buffer.Enqueue(this);148 if (_buffer.Count == SCStats.MAX_ELFS)149 {150 _santaHandle.Release();151 }152 }


153

154 Console.WriteLine("Elf {0} at the door", ID);155

156 //Wait for santa to be ready157 _waitingToAsk.Wait();158

159 //Asking questions160 _doneAsking.Wait();161

162 //Allow a new elf to visit santa163 _maxElfs.Release();164 }165 });166 }167

168 public void AskQuestion()169 {170 _waitingToAsk.Release();171 }172 }173

174 public class LockingReindeer : IStartable175 {176 private readonly Random _randomGen = new

Random(Guid.NewGuid().GetHashCode());177 public int ID { get; private set; }178

179 private readonly Queue<LockingReindeer> _reindeerBuffer;180 private readonly SemaphoreSlim _santaHandle;181 private readonly SemaphoreSlim _sleigh;182 private readonly SemaphoreSlim _doneDelivering;183 private readonly SemaphoreSlim _warmingHut;184

185 public LockingReindeer(int id, Queue<LockingReindeer> buffer,SemaphoreSlim santaHandle, SemaphoreSlim sleigh,SemaphoreSlim warmingHut, SemaphoreSlim doneDelivering)

186 {187 ID = id;188 _reindeerBuffer = buffer;189 _santaHandle = santaHandle;190 _sleigh = sleigh;191 _warmingHut = warmingHut;192 _doneDelivering = doneDelivering;193 }194

195 public Task Start()196 {197 return Task.Run(() =>198 {199 while (true)200 {201 //Tan on the beaches in the Pacific until Chistmas is

close202 Thread.Sleep(100 * _randomGen.Next(10));


203

204 lock (_reindeerBuffer)205 {206 _reindeerBuffer.Enqueue(this);207 if (_reindeerBuffer.Count == SCStats.NR_REINDEER)208 {209 _santaHandle.Release();210 }211 }212

213 //Console.WriteLine("Reindeer {0} is back",ID);214

215 //Block early arrivals216 _warmingHut.Wait();217

218 //Wait for santa to be ready219 _sleigh.Wait();220

221 //Delivering presents222

223 //Wait for delivery to be done224 _doneDelivering.Wait();225 //Head back to Pacific islands226 }227 });228 }229 }

C.1.3 Concurrent Queue

The lock based queue implementation is based on Michael L Scott’s lock-basedqueue algorithm described in [19]. The implementation uses two locks, as seenon line 17 and 26, to protect the tail and head of the queue respectively. Thefirst node in the queue is a dummy node which allows the enqueue and dequeueoperations to only operate on the tail and head respectively, thereby allowingenqueuing and dequeuing to occur concurrently. Only a single enqueue ordequeue operation can however execute at a time, due to the use of locking.The dummy node is created by the queue’s constructor on line 10.

Listing C.3: Lock Based Concurrent Queue Implementation

1 public class Queue<T> : IQueue<T>2 {3 protected readonly object HeadLock = new object();4 protected readonly object TailLock = new object();5 private Node _head;6 private Node _tail;7

8 public Queue()9 {

10 _head = new Node(default(T));


11 _tail = _head;12 }13

14 public void Enqueue(T item)15 {16 var node = new Node(item);17 lock (TailLock)18 {19 _tail.Next = node;20 _tail = node;21 }22 }23

24 public bool Dequeue(out T item)25 {26 lock (HeadLock)27 {28 var newHead = _head.Next;29 if (newHead == null)30 {31 item = default(T);32 return false;33 }34

35 _head = newHead;36 item = newHead.Value;37 return true;38 }39 }40

41 private class Node42 {43 public Node Next { get; set; }44 public T Value { get; private set; }45

46 public Node(T value)47 {48 Value = value;49 }50 }51 }

C.1.4 Concurrent Hashmap

The lock-based concurrent hashmap implementation is based on the conceptof lock striping. Lock striping allows L locks to protect B buckets whereB � L and B mod L = 0 so that each lock l protects each bucket b whereindexof(b) = indexof(l) mod L[53, p. 304]. When the number of buckets isdoubled as a result of resizing the hashmap the same lock protects a particularbucket before and after the resize completes. The buckets protected by aparticular lock are referred to as a stripe and multiple stripes can be accessedconcurrently.


Along with the backing array of buckets defined on line 49 an array of objectsrepresenting the locks is defined on line 48. Before a thread access a bucketit must acquire the lock on the stripe containing the bucket in question. Asan example, the Add method defined on line 195 calculates the index ofthe lock to acquire on line 198 before adding the item to the bucket. TheResizeIfNeeded method defined on line 287 acquires the lock on all stripeson line 291 ensuring that no other threads are accessing the backing arraywhile being resized. The condition for resizing is checked before all locks areacquired on line 289 and again after all locks have been acquired on line 294,ensuring that only a single thread resizes the backing array for every instancewhere a resize is required. The object defined on line 47 protect the sizevariable defined on line 50. This is required as multiple threads can add orremove items to di↵erent stripes simultaneously requiring their access to thesize variable to be synchronized.

Listing C.4: Lock Based Concurrent Hashmap Implementation

1 public abstract class BaseHashMap<K,V> : IMap<K,V>2 {3 protected const int DefaultNrBuckets = 16;4 protected const double LoadFactor = 0.75D;5

6 public abstract bool ContainsKey(K key);7 public abstract V Get(K key);8 public abstract void Add(K key, V value);9 public abstract bool AddIfAbsent(K key, V value);

10 public abstract bool Remove(K k);11 public abstract V this[K key] { get; set; }12 public virtual int Count { get; protected set; }13

14 protected int CalculateThreshold(int nrBuckets)15 {16 return (int)(nrBuckets * LoadFactor);17 }18

19 protected int GetHashCode(K key)20 {21 var hashCode = key.GetHashCode();22 return hashCode < 0 ? 0 - hashCode : hashCode;23 }24

25 protected int GetBucketIndex(int length, K key)26 {27 return GetBucketIndex(length, GetHashCode(key));28 }29

30 protected int GetBucketIndex(int length, int hashCode)31 {32 return hashCode % length;33 }34


35 public abstract IEnumerator<KeyValuePair<K, V>> GetEnumerator();36

37 IEnumerator IEnumerable.GetEnumerator()38 {39 return GetEnumerator();40 }41 }42

43 public class LockingHashMap<K,V> : BaseHashMap<K,V>44 {45 private const int DefaultNrLocks = DefaultNrBuckets;46

47 private readonly object _sizeLock = new object();48 private readonly object[] _locks;49 private Node[] _buckets;50 private int _size;51 private int _threshold;52

53 public LockingHashMap() : this(DefaultNrBuckets){}54

55 public LockingHashMap(int size) : this(size, DefaultNrLocks){}56

57 private LockingHashMap(int size, int nrLocks)58 {59 if (size % nrLocks != 0)60 {61 throw new Exception("The intital size % nrlocks must be

equal to zero");62 }63 _buckets = MakeBuckets(size);64 _locks = MakeLocks(nrLocks);65 _threshold = CalculateThreshold(size);66 }67

68

69 private Node[] MakeBuckets(int nrBuckets)70 {71 return new Node[nrBuckets]; ;72 }73

74 private object[] MakeLocks(int nrLocks)75 {76 var temp = new object[nrLocks];77 for (var i = 0; i < nrLocks; i++)78 {79 temp[i] = new object();80 }81

82 return temp;83 }84

85 #region Utility86

87 private int GetLockIndex(int hashCode)


88 {89 return hashCode % _locks.Length;90 }91

92 private Node CreateNode(K key, V value)93 {94 return new Node(key, value);95 }96

97 private int GetBucketIndex(int hashCode)98 {99 return GetBucketIndex(_buckets.Length, hashCode);

100 }101

102 private Node FindNode(Node node, K key)103 {104 while (node != null && !key.Equals(node.Key))105 node = node.Next;106 return node;107 }108

109 private void LockAll()110 {111 foreach (var lo in _locks)112 {113 Monitor.Enter(lo);114 }115 }116

117 private void UnlockAll()118 {119 foreach (var lo in _locks)120 {121 Monitor.Exit(lo);122 }123 }124

125 private void InsertInBucket(Node[] buckets, Node node, intindex)

126 {127 InsertInBucket(buckets, node, buckets[index], index);128 }129

130 private void InsertInBucket(Node node, int index)131 {132 InsertInBucket(node, _buckets[index], index);133 }134

135 private void InsertInBucket(Node node, Node curNode, int index)136 {137 InsertInBucket(_buckets, node, curNode, index);138 }139

140 private void InsertInBucket(Node[] buckets, Node node, NodecurNode, int index)


141 {142 if (curNode != null)143 {144 node.Next = curNode;145 }146 buckets[index] = node;147 }148

149 #endregion Utility150

151 public override int Count {152 get {153 lock (_sizeLock)154 {155 return _size;156 }157 }158

159 protected set {160 lock (_sizeLock)161 {162 _size = value;163 }164 }165 }166

167 public override bool ContainsKey(K key)168 {169 var hashCode = GetHashCode(key);170 lock (_locks[GetLockIndex(hashCode)])171 {172 var bucket = _buckets[GetBucketIndex(hashCode)];173 return FindNode(bucket, key) != null;174 }175 }176

177 public override V Get(K key)178 {179 var hashCode = GetHashCode(key);180 lock (_locks[GetLockIndex(hashCode)])181 {182 var bucket = _buckets[GetBucketIndex(hashCode)];183 var node = FindNode(bucket, key);184

185 if (node == null)186 {187 //If node is null, key is not in map188 throw new KeyNotFoundException("Key not found. Key:

"+key);189 }190

191 return node.Value;192 }193 }


194

195 public override void Add(K key, V value)196 {197 var hashCode = GetHashCode(key);198 lock (_locks[GetLockIndex(hashCode)])199 {200 var index = GetBucketIndex(hashCode);201 var bucket = _buckets[index];202 var node = FindNode(bucket, key);203

204 if (node != null)205 {206 //If node is not null, key exist in map. Update the value207 node.Value = value;208 }209 else210 {211 //Else insert the node212 InsertInBucket(CreateNode(key, value),bucket,index);213 lock (_sizeLock)214 {215 _size++;216 }217 }218 }219

220 ResizeIfNeeded();221 }222

223 public override bool AddIfAbsent(K key, V value)224 {225 var hashCode = GetHashCode(key);226 lock (_locks[GetLockIndex(hashCode)])227 {228 var index = GetBucketIndex(hashCode);229 var bucket = _buckets[index];230 var node = FindNode(bucket, key);231

232 if (node != null) return false;233 //If node is not in map insert new node234 InsertInBucket(CreateNode(key, value), bucket, index);235 lock (_sizeLock)236 {237 _size++;238 }239 ResizeIfNeeded();240 return true;241 }242 }243

244 public override bool Remove(K key)245 {246 var hashCode = GetHashCode(key);247 lock (_locks[GetLockIndex(hashCode)])


248 {249 var index = GetBucketIndex(hashCode);250 var bucket = _buckets[index];251 return RemoveNode(key, bucket, index);252 }253 }254

255 private bool RemoveNode(K key, Node node, int index)256 {257 if (node == null)258 {259 return false;260 }261

262 if (node.Key.Equals(key))263 {264 lock (_sizeLock)265 {266 _size--;267 }268 buckets[index] = node.Next;269 return true;270 }271

272

273 while (node.Next != null && !key.Equals(node.Next.Key))274 node = node.Next;275

276 //node.Next == null || node.Next.Key == key277 if (node.Next == null) return false;278

279 lock (_sizeLock)280 {281 _size--;282 }283 node.Next = node.Next.Next;284 return true;285 }286

287 private void ResizeIfNeeded()288 {289 if (ResizeCondtion())290 {291 LockAll();292 try293 {294 if (!ResizeCondtion())295 {296 return;297 }298 //Construct new backing array299 var newBucketSize = _buckets.Length * 2;300 var newBuckets = MakeBuckets(newBucketSize);301


302 //For each key in the map rehash303 for (var i = 0; i < _buckets.Length; i++)304 {305 var node = _buckets[i];306 while (node != null)307 {308 var bucketIndex = GetBucketIndex(newBucketSize,

node.Key);309 InsertInBucket(newBuckets,CreateNode(node.Key,node.Value),bucketIndex);310 node = node.Next;311 }312 }313

314 //Calculate new resize threshold and assign therehashed backing array

315 threshold = CalculateThreshold(newBucketSize);316 _buckets = newBuckets;317 }318 finally319 {320 UnlockAll();321 }322 }323 }324

325 private bool ResizeCondtion()326 {327 lock (_sizeLock)328 {329 return _size >= _threshold;330 }331 }332

333 public override V this[K key]334 {335 get { return Get(key); }336 set { Add(key,value); }337 }338

339 public override IEnumerator<KeyValuePair<K, V>> GetEnumerator()340 {341 LockAll();342 try343 {344 var list = new List<KeyValuePair<K,V>>(_size);345 for (var i = 0; i < _buckets.Length; i++)346 {347 var node = _buckets[i];348 while (node != null)349 {350 list.Add(new KeyValuePair<K, V>(node.Key, node.Value));351 node = node.Next;352 }353 }


354

355 return list.GetEnumerator();356 }357 finally358 {359 UnlockAll();360 }361 }362

363 private class Node364 {365 public K Key { get; private set; }366 public V Value { get; internal set; }367 public Node Next { get; internal set; }368

369 public Node(K key, V value)370 {371 Key = key;372 Value = value;373 }374 }375 }

C.2 STM Library

This section presents the library-based STM implementations used for theevaluation.


The library based STM uses five STM objects of type TMVar<bool> , de-fined on lines 8 to 12, to represent each of the forks on the table. TheStartPhilosopher method on line 27 is called for each philosopher in orderto start a thread with access to the correct forks. The transaction definedon line 33 attempts to acquired both forks in order to begin eating. Implicitconversion is employed on line 35 to read the state of the forks without havingto access the Value property. The call to STMSystem.Retry on line 37blocks the calling thread until the state of the forks change, if atleast one ofthe forks is unavailable. The transaction on line 48 simply makes both forksavailable, after the philosopher has finished eating on lines 44 to 46.

Listing C.5: STM Library Based Dining Philosophers Implementation

1 public class DiningPhilosophers2 {3 private const int MAX_EAT_COUNT = 1000;4

5 public static void Start()6 {


7 var eatCounter = new TMInt(0);8 var fork1 = new TMVar<bool>(true);9 var fork2 = new TMVar<bool>(true);

10 var fork3 = new TMVar<bool>(true);11 var fork4 = new TMVar<bool>(true);12 var fork5 = new TMVar<bool>(true);13

14 var t1 = StartPhilosopher(eatCounter, fork1, fork2);15 var t2 = StartPhilosopher(eatCounter, fork2, fork3);16 var t3 = StartPhilosopher(eatCounter, fork3, fork4);17 var t4 = StartPhilosopher(eatCounter, fork4, fork5);18 var t5 = StartPhilosopher(eatCounter, fork5, fork1);19


27 private static Thread StartPhilosopher(TMInt eatCounter,TMVar<bool> left, TMVar<bool> right)

28 {29 var t1 = new Thread(() =>30 {31 while (eatCounter < MAX_EAT_COUNT)32 {33 STMSystem.Atomic(() =>34 {35 if (!left || !right)36 {37 STMSystem.Retry();38 }39

40 left.Value = false;41 right.Value = false;42 });43

44 Console.WriteLine("Thread: " +Thread.CurrentThread.ManagedThreadId + " eating.");

45 Thread.Sleep(100);46 Console.WriteLine("Eat count: " + ++eatCounter);47

48 STMSystem.Atomic(() =>49 {50 left.Value = true;51 right.Value = true;52 });53

54 Thread.Sleep(100);55 }56 });57

58 t1.Start();


59

60 return t1;61 }62 }


As shown on line 124, the library-based Santa Claus problem implementationuses a transaction with an associated orelse clause to wake Santa given eitherthe reindeer or elf condition is true true. The calls to STMSystem.Retryon lines 128 and 137 causes Santa to block if the transaction is executedand non of the conditions are true. If one of the read variable change Santaautomatically tests the conditions again. As a result the elfs and reindeerdoes not have to take an explicit action in order to wake Santa. In fact theyneed not to even know that Santa exists. A call to STMSystem.Retry isused on line 171 to ensure that only three elfs can go to Santa at the sametime. Generally, transactions containing calls to STMSystem.Retry are usedensure that the elfs and reindeer do not progress to the next state before therequired conditions e.g. having delivered the presents, are true. The queueutilized is a transactional queue implementation available through the STMlibrary.

Listing C.6: STM Library Based Santa Claus Implementation

1 public class SantaClausProblem2 {3 public static void Start()4 {5 var rBuffer = new Queue<Reindeer>();6 var eBuffer = new Queue<Elf>();7 var santa = new Santa(rBuffer,eBuffer);8 santa.Start();9

10 for (int i = 0; i < SCStats.NR_REINDEER ; i++)11 {12 var reindeer = new Reindeer(i, rBuffer);13 reindeer.Start();14 }15

16 for (int i = 0; i < SCStats.NR_ELFS; i++)17 {18 var elf = new Elf(i, eBuffer);19 elf.Start();20 }21 }22 }23

24 public class Santa : IStartable25 {26 private readonly Queue<Reindeer> _rBuffer;


27 private readonly Queue<Elf> _eBuffer;28

29 public Santa(Queue<Reindeer> rBuffer, Queue<Elf> eBuffer)30 {31 _rBuffer = rBuffer;32 _eBuffer = eBuffer;33 }34

35 public Task Start()36 {37 return Task.Run(() =>38 {39 while (true)40 {41 var wakestate = SleepUntilAwoken();42

43 switch (wakestate)44 {45 case WakeState.ReindeerBack:46 HandleReindeer();47 break;48 case WakeState.ElfsIncompetent:49 HandleElfs();50 break;51 }52 }53 });54 }55

56 private void HandleElfs()57 {58 Console.WriteLine("3 elfs at the door!");59 STMSystem.Atomic(() =>60 {61 foreach (var elf in _eBuffer)62 {63 elf.AskQuestion();64 }65 });66


70 //Back to work incompetent elfs!71 STMSystem.Atomic(() =>72 {73 for (int i = 0; i < SCStats.MAX_ELFS; i++)74 {75 var elf = _eBuffer.Dequeue();76 elf.BackToWork();77 }78 });79

80 Console.WriteLine("Elfs helped");


81 }82

83 private void HandleReindeer()84 {85 Console.WriteLine("All reindeer are back!");86

87 //Call reindeer from the warming hut88 STMSystem.Atomic(() =>89 {90 foreach (var reindeer in _rBuffer)91 {92 reindeer.CallToSleigh();93 }94 });95

96 //Setup the sleigh97 STMSystem.Atomic(() =>98 {99 foreach (var reindeer in _rBuffer)

100 {101 reindeer.HelpDeliverPresents();102 }103 });104


109 //Release reindeer110 STMSystem.Atomic(() =>111 {112 while (_rBuffer.Count != 0)113 {114 var reindeer = _rBuffer.Dequeue();115 reindeer.ReleaseReindeer();116 }117 });118

119 Console.WriteLine("Reindeer released");120 }121

122 private WakeState SleepUntilAwoken()123 {124 return STMSystem.Atomic(() =>125 {126 if (_rBuffer.Count != SCStats.NR_REINDEER)127 {128 STMSystem.Retry();129 }130

131 return WakeState.ReindeerBack;132 },133 () =>134 {


135 if (_eBuffer.Count != SCStats.MAX_ELFS)136 {137 STMSystem.Retry();138 }139

140 return WakeState.ElfsIncompetent;141 });142 }143 }144

145 public class Elf : IStartable146 {147 private readonly Random _randomGen = new

Random(Guid.NewGuid().GetHashCode());148 public int ID { get; private set; }149 private readonly Queue<Elf> _buffer;150 private readonly TMVar<bool> _waitingToAsk = new

TMVar<bool>(false);151 private readonly TMVar<bool> _questionAsked = new

TMVar<bool>(false);152

153 public Elf(int id, Queue<Elf> buffer)154 {155 _buffer = buffer;156 ID = id;157 }158

159 public Task Start()160 {161 return Task.Run(() =>162 {163 while (true)164 {165 Thread.Sleep(100 * _randomGen.Next(21));166

167 STMSystem.Atomic(() =>168 {169 if (_buffer.Count == SCStats.MAX_ELFS)170 {171 STMSystem.Retry();172 }173

174 _buffer.Enqueue(this);175 _waitingToAsk.Value = true;176 });177

178 Console.WriteLine("Elf {0} at the door",ID);179 //Waiting on santa180 STMSystem.Atomic(() =>181 {182 if (_waitingToAsk)183 {184 STMSystem.Retry();185 }


186 });187

188 //Asking question189

190 //Done asking191 STMSystem.Atomic(() =>192 {193 if (!_questionAsked)194 {195 STMSystem.Retry();196 }197

198 _questionAsked.Value = false;199 });200

201 }202 });203 }204

205 public void AskQuestion()206 {207 _waitingToAsk.Value = false;208 }209

210 public void BackToWork()211 {212 _questionAsked.Value = true;213 }214 }215

216 public class Reindeer : IStartable217 {218 private readonly Random _randomGen = new


221 private readonly Queue<Reindeer> _reindeerBuffer;222 private readonly TMVar<bool> _workingForSanta = new

TMVar<bool>(false);223 private readonly TMVar<bool> _waitingAtSleigh = new

TMVar<bool>(false);224 private readonly TMVar<bool> _waitingInHut = new

TMVar<bool>(false);225

226 public Reindeer(int id, Queue<Reindeer> buffer)227 {228 ID = id;229 _reindeerBuffer = buffer;230 }231

232 public Task Start()233 {234 return Task.Run(() =>235 {


236 while (true)237 {238 Thread.Sleep(100 * _randomGen.Next(10));239

240 STMSystem.Atomic(() =>241 {242 _reindeerBuffer.Enqueue(this);243 _waitingInHut.Value = true;244 });245

246 Console.WriteLine("Reindeer {0} is back",ID);247

248 //Waiting in the warming hut249 STMSystem.Atomic(() =>250 {251 if (_waitingInHut)252 {253 STMSystem.Retry();254 }255 });256

257 //Wait for santa to be ready258 STMSystem.Atomic(() =>259 {260 if (_waitingAtSleigh)261 {262 STMSystem.Retry();263 }264 });265


268 //Wait to be released by santa269 STMSystem.Atomic(() =>270 {271 if (_workingForSanta)272 {273 STMSystem.Retry();274 }275 });276 }277 });278 }279

280 public void CallToSleigh()281 {282 STMSystem.Atomic(() =>283 {284 _waitingInHut.Value = false;285 _waitingAtSleigh.Value = true;286 });287 }288

289 public void HelpDeliverPresents()


290 {291 STMSystem.Atomic(() =>292 {293 _waitingAtSleigh.Value = false;294 _workingForSanta.Value = true;295 });296 }297

298 public void ReleaseReindeer()299 {300 _workingForSanta.Value = false;301 }302 }


The STM concurrent queue implementation is implemented as a linked listof nodes. As with the lock-based concurrent queue implementation, thelibrary-based STM concurrent queue implementation uses a dummy node,allowing the enqueue and dequeue operations to operate only on the tail andhead respectively. The head and tail, defined on lines 3 and 4, are oftype TMVar<Node> allowing the STM system to track assignments to theencapsulated variables. Similarly the Next field of the Node class defined online 40 is of type TMVar<Node>. The Dequeue method defined in line 24,skips over the dummy item when reading the node to remove from the queueon line 28. The call to STMSystem.Retry on line 32 causes a thread callingDequeue on an empty queue to block until the queue is no longer empty. Theassignment on line 32 sets the node representing the item to dequeue as thenew dummy node, as it is no longer needed and already points to the nextitem in the queue.

Listing C.7: STM Library Based Concurrent Queue Implementation

1 public class Queue<T>2 {3 private readonly TMVar<Node> _head = new TMVar<Node>(null);4 private readonly TMVar<Node> _tail = new TMVar<Node>(null);5

6 public Queue()7 {8 var node = new Node(default(T));9 _head.Value = node;

10 _tail.Value = node;11 }12

13 public void Enqueue(T value)14 {15 STMSystem.Atomic(() =>16 {17 var node = new Node(value);


18 var curTail = _tail.Value;19 curTail.Next.Value = node;20 _tail.Value = node;21 });22 }23

24 public T Dequeue()25 {26 return STMSystem.Atomic(() =>27 {28 var node = _head.Value.Next.Value;29

30 if (node == null)31 {32 STMSystem.Retry();33 }34

35 _head.Value = node;36 return node.Value;37 });38 }39

40 private class Node41 {42 public readonly TMVar<Node> Next = new TMVar<Node>(null);43 public readonly T Value;44



The library-based STM hashmap defines the collision list as a linked list ofinstances of the Node class defined on line 243. The backing array, defined online 4, of type TMVar<TMVar<Node>[]>, that is a transactional variable toan array of transactional variables to instances of the node Node class. Thisallows the STM system to track the assignment of a new backing array as aresult of resizing the hashmap as well as track the assignment of the first nodein each bucket trough the TMVars in the backing array. As the Next propertyof the Node class, defined on line 247, is type TMVar<Node> the STM systemis able to track changes to the collision list. As the Add method defined online 97 updates the value of a node if the key to add is already present in thehashmap, the Value property of the Node class is of type TMVar<V> whereV is the type parameter, defining the type of the values added to the hashmap.The size variable is of type TMInt and is increment and decremented usingthe supplied ++ and -- overloads defined in the STM library.


Listing C.8: STM Library Based Concurrent Hashmap Implementation

1 public class STMHashMap<K,V> : BaseHashMap<K,V>2 {3 //TMVar to (array of TMVars to Node )4 private readonly TMVar<TMVar<Node>[]> _buckets = new

TMVar<TMVar<Node>[]>();5 private readonly TMInt _threshold = new TMInt();6 private TMInt _size = new TMInt();7

8 public STMHashMap() : this(DefaultNrBuckets)9 {

10

11 }12

13 public STMHashMap(int nrBuckets)14 {15 _buckets.Value = MakeBuckets(nrBuckets);16 _threshold.Value = CalculateThreshold(nrBuckets);17 }18

19 /// <summary>20 /// Creates and initializes the backing array21 /// </summary>22 /// <param name="nrBuckets"></param>23 /// <returns></returns>24 private TMVar<Node>[] MakeBuckets(int nrBuckets)25 {26 var temp = new TMVar<Node>[nrBuckets];27 for (var i = 0; i < nrBuckets; i++)28 {29 temp[i] = new TMVar<Node>();30 }31

32 return temp;33 }34

35


38 private Node CreateNode(K key, V value)39 {40 return new Node(key,value);41 }42

43 private int GetBucketIndex(K key)44 {45 return GetBucketIndex(_buckets.Value.Length, key);46 }47

48 private Node FindNode(K key)49 {50 return FindNode(key, GetBucketIndex(key));51 }


52

53 private Node FindNode(K key, int bucketIndex)54 {55 return FindNode(key, _buckets.Value[bucketIndex].Value);56 }57

58 private Node FindNode(K key, Node node)59 {60 while (node != null && !key.Equals(node.Key))61 node = node.Next.Value;62 return node;63 }64

65 private void InsertInBucket(TMVar<Node> bucketVar, Node node)66 {67 var curNode = bucketVar.Value;68 if (curNode != null)69 {70 node.Next.Value = curNode;71 }72 bucketVar.Value = node;73 }74


77 public override bool ContainsKey(K key)78 {79 return STMSystem.Atomic(() => FindNode(key) != null);80 }81

82 public override V Get(K key)83 {84 return STMSystem.Atomic(() =>85 {86 var node = FindNode(key);87 if (node == null)88 {89 //If node == null key is not present in dictionary90 throw new KeyNotFoundException("Key not found. Key: " +

key);91 }92

93 return node.Value.Value;94 });95 }96

97 public override void Add(K key, V value)98 {99 STMSystem.Atomic(() =>

100 {101 var bucketIndex = GetBucketIndex(key);102 //TMVar wrapping the immutable chain list103 var bucketVar = _buckets.Value[bucketIndex];104 var node = FindNode(key, bucketVar.Value);


105

106 if (node != null)107 {108 //If node is not null key exist in map. Update the value109 node.Value.Value = value;110 }111 else112 {113 //Else insert the node114 InsertInBucket(bucketVar, CreateNode(key,value));115 _size++;116 ResizeIfNeeded();117 }118 });119 }120

121 public override bool AddIfAbsent(K key, V value)122 {123 return STMSystem.Atomic(() =>124 {125 var bucketIndex = GetBucketIndex(key);126 //TMVar wrapping the immutable chain list127 var bucketVar = _buckets.Value[bucketIndex];128 var node = FindNode(key, bucketVar.Value);129

130 if (node == null)131 {132 //If node is not found key does not exist so insert133 InsertInBucket(bucketVar, CreateNode(key,value));134 _size++;135 ResizeIfNeeded();136 return true;137 }138

139 return false;140 });141 }142

143 private void ResizeIfNeeded()144 {145 if (_size.Value >= _threshold.Value)146 {147 Resize();148 }149 }150

151 private void Resize()152 {153 //Construct new backing array154 var newBucketSize = _buckets.Value.Length * 2;155 var newBuckets = MakeBuckets(newBucketSize);156

157 //For each key in the map rehash158 for (var i = 0; i < _buckets.Value.Length; i++)


159 {160 var bucket = _buckets.Value[i];161 var node = bucket.Value;162 while (node != null)163 {164 var bucketIndex = GetBucketIndex(newBucketSize, node.Key);165 InsertInBucket(newBuckets[bucketIndex],

CreateNode(node.Key, node.Value));166 node = node.Next.Value;167 }168 }169

170 //Calculate new resize threshold and assign the rehashedbacking array

171 _threshold.Value = CalculateThreshold(newBucketSize);172 _buckets.Value = newBuckets;173 }174

175 public override bool Remove(K key)176 {177 return STMSystem.Atomic(() =>178 {179 var bucketIndex = GetBucketIndex(key);180 //TMVar wrapping the immutable chain list181 var bucketVar = _buckets.Value[bucketIndex];182 var firstNode = bucketVar.Value;183

184 return RemoveNode(key, firstNode, bucketVar);185 });186 }187

188 private bool RemoveNode(K key, Node node, TMVar<Node> bucketVar)189 {190 if (node == null)191 {192 return false;193 }194

195 if (node.Key.Equals(key))196 {197 _size--;198 bucketVar.Value = node.Next;199 return true;200 }201

202 while (node.Next != null && !key.Equals(node.Next.Value.Key))203 node = node.Next.Value;204


208 _size--;209 node.Next.Value = node.Next.Value.Next;210 return true;


211 }212

213 public override IEnumerator<KeyValuePair<K, V>> GetEnumerator()214 {215 return STMSystem.Atomic(() =>216 {217 var list = new List<KeyValuePair<K, V>>(_size.Value);218 for (var i = 0; i < _buckets.Value.Length; i++)219 {220 var bucket = _buckets.Value[i];221 var node = bucket.Value;222 while (node != null)223 {224 list.Add(new KeyValuePair<K, V>(node.Key, node.Value));225 node = node.Next.Value;226 }227 }228 return list.GetEnumerator();229 });230 }231

232 public override V this[K key]233 {234 get { return Get(key); }235 set { Add(key, value); }236 }237

238 public override int Count239 {240 get { return _size.Value; }241 }242

243 private class Node244 {245 public K Key { get; private set; }246 public TMVar<V> Value { get; private set; }247 public TMVar<Node> Next { get; private set; }248

249 public Node(K key, V value)250 {251 Key = key;252 Value = new TMVar<V>(value);253 Next = new TMVar<Node>();254 }255 }256 }

C.3 AC#

This section presents the AC# implementations used for the evaluation. As thestrategies employed for the AC# implementations are similar to those of thelibrary-based STM implementations the descriptions focus on the di↵erences


between the implementations as well as how the abstractions provided by AC#are utilized.


The AC# dining philosophers implementation represents the forks as instancesof the Fork class defined on line 58. Five forks are created on lines 8 to 12 andpassed to the StartPhilosopher method on lines 14 to 18. The transactiondefined in line 33 acquires both forks by calling the forks AttemptToPickUpwhich is itself defined using a transaction. If a fork is unavailable the callingthread will block until the state changes as a result of the retry statement online 74. The transaction defined on line 43 puts down both forks after eatingon lines 39 to 41 has finished.

Listing C.9: AC# Based Dining Philosophers Implementation

1 public class DiningPhilosopher2 {3 private static readonly int MAX_EAT_COUNT = 1000;4 private static atomic int eatCounter = 0;5

6 public void Start()7 {8 var fork1 = new Fork();9 var fork2 = new Fork();

10 var fork3 = new Fork();11 var fork4 = new Fork();12 var fork5 = new Fork();13

14 var t1 = StartPhilosopher(fork1, fork2);15 var t2 = StartPhilosopher(fork2, fork3);16 var t3 = StartPhilosopher(fork3, fork4);17 var t4 = StartPhilosopher(fork4, fork5);18 var t5 = StartPhilosopher(fork5, fork1);19


27 private Thread StartPhilosopher(Fork left, Fork right)28 {29 var t1 = new Thread(() =>30 {31 while (eatCounter < MAX_EAT_COUNT)32 {33 atomic34 {35 left.AttemptToPickUp();36 right.AttemptToPickUp();


37 }38

39 Console.WriteLine("Thread: " +Thread.CurrentThread.ManagedThreadId + " eating.");

40 Thread.Sleep(100);41 Console.WriteLine("Eat count: " + ++eatCounter);42

43 atomic44 {45 left.PutDown();46 right.PutDown();47 }48

49 Thread.Sleep(100);50 }51 });52

53 t1.Start();54

55 return t1;56 }57

58 public class Fork59 {60 private atomic61 private bool State { get; set; }62

63 public Fork()64 {65 State = true;66 }67

68 public void AttemptToPickUp()69 {70 atomic71 {72 if (!State)73 {74 retry;75 }76

77 State = false;78 }79 }80

81 public void PutDown()82 {83 atomic84 {85 State = true;86 }87 }88 }89 }



As with the library-based Santa Claus implementation AC# uses an atomicblock with an associated orelse block, defined on line 65, to wake Santawhen either the reindeer or elf condition are true. On line 178 the retrystatement is used to ensure that no more than three elfs can go to Santa at anygiven point. Similarly, transactions utilizing retry statement are us ensurethat the elfs and reindeer do not proceeded to next state before the requiredcondition is true throughout the implementation.

Listing C.10: AC# Based Santa Claus Implementation

1 public class SantaClausProblem2 {3 public const int NR_REINDEER = 9;4 public const int NR_ELFS = 6;5 public const int MAX_ELFS = 3;6

7 public static void Main()8 {9 var rBuffer = new Queue<Reindeer>();

10 var eBuffer = new Queue<Elf>();11 var santa = new Santa(rBuffer, eBuffer);12 santa.Start();13

14 for (int i = 0; i < SantaClausProblem.NR_REINDEER; i++)15 {16 var reindeer = new Reindeer(i, rBuffer);17 reindeer.Start();18 }19

20 for (int i = 0; i < SantaClausProblem.NR_ELFS; i++)21 {22 var elf = new Elf(i, eBuffer);23 elf.Start();24 }25

26 System.Console.WriteLine("Press any key to terminate...");27 System.Console.ReadKey();28 }29 }30

31 public class Santa32 {33 private readonly Queue<Reindeer> _rBuffer;34 private readonly Queue<Elf> _eBuffer;35

36 public Santa(Queue<Reindeer> rBuffer, Queue<Elf> eBuffer)37 {38 _rBuffer = rBuffer;39 _eBuffer = eBuffer;40 }41


42 public Task Start()43 {44 return Task.Run(() =>45 {46 while (true)47 {48 var wakestate = SleepUntilAwoken();49

50 switch (wakestate)51 {52 case WakeState.ReindeerBack:53 HandleReindeer();54 break;55 case WakeState.ElfsIncompetent:56 HandleElfs();57 break;58 }59 }60 });61 }62

63 private WakeState SleepUntilAwoken()64 {65 atomic66 {67 if (_rBuffer.Count != SantaClausProblem.NR_REINDEER)68 {69 retry;70 }71

72 return WakeState.ReindeerBack;73 }74 orelse75 {76 if (_eBuffer.Count != SantaClausProblem.MAX_ELFS)77 {78 retry;79 }80

81 return WakeState.ElfsIncompetent;82 }83 }84

85 private void HandleReindeer()86 {87 Console.WriteLine("All reindeer are back!");88

89 //Call reindeer from the warming hut90 atomic91 {92 foreach (var reindeer in _rBuffer)93 {94 reindeer.CallToSleigh();95 }


96 }97

98 //Setup the sleigh99 atomic

100 {101 foreach (var reindeer in _rBuffer)102 {103 reindeer.HelpDeliverPresents();104 }105 }106


111 //Release reindeer112 atomic113 {114 while (_rBuffer.Count != 0)115 {116 var reindeer = _rBuffer.Dequeue();117 reindeer.ReleaseReindeer();118 }119 }120

121 Console.WriteLine("Reindeer released");122 }123

124 private void HandleElfs()125 {126 Console.WriteLine("3 elfs at the door!");127 atomic128 {129 foreach (var elf in _eBuffer)130 {131 elf.AskQuestion();132 }133 }134


138 //Back to work incompetent elfs!139 atomic140 {141 for (int i = 0; i < SantaClausProblem.MAX_ELFS; i++)142 {143 var elf = _eBuffer.Dequeue();144 elf.BackToWork();145 }146 }147

148 Console.WriteLine("Elfs helped");149 }


150 }151

152 public class Elf153 {154 private Random randomGen = new

Random(Guid.NewGuid().GetHashCode());155 public int ID { get; private set; }156 private Queue<Elf> _buffer;157 private atomic bool _waitingToAsk = false;158 private atomic bool _questionAsked = false;159

160 public Elf(int id, Queue<Elf> buffer)161 {162 _buffer = buffer;163 ID = id;164 }165

166 public Task Start()167 {168 return Task.Run(() =>169 {170 while (true)171 {172 Thread.Sleep(100 * randomGen.Next(21));173

174 atomic175 {176 if (_buffer.Count == SantaClausProblem.MAX_ELFS)177 {178 retry;179 }180

181 _buffer.Enqueue(this);182 _waitingToAsk = true;183 }184

185 Console.WriteLine("Elf {0} at the door", ID);186

187 //Waiting on santa188 atomic189 {190 if (_waitingToAsk)191 {192 retry;193 }194 }195

196 //Asking question197

198 //Done asking199 atomic200 {201 if (!_questionAsked)202 {


203 retry;204 }205

206 _questionAsked = false;207 }208 }209 });210 }211

212 public void AskQuestion()213 {214 _waitingToAsk = false;215 }216

217 public void BackToWork()218 {219 _questionAsked = true;220 }221 }222

223 public class Reindeer224 {225 private readonly Random randomGen = new


228 private Queue<Reindeer> reindeerBuffer;229 private atomic bool _workingForSanta = false;230 private atomic bool _waitingAtSleigh = false;231 private atomic bool _waitingInHut = false;232

233 public Reindeer(int id, Queue<Reindeer> buffer)234 {235 ID = id;236 reindeerBuffer = buffer;237 }238

239 public Task Start()240 {241 return Task.Run(() =>242 {243 while (true)244 {245 Thread.Sleep(100 * randomGen.Next(10));246

247 atomic248 {249 reindeerBuffer.Enqueue(this);250 _waitingInHut = true;251 }252

253 Console.WriteLine("Reindeer {0} is back",ID);254

255 //Waiting in the warming hut


256 atomic257 {258 if (_waitingInHut)259 {260 retry;261 }262 }263

264 //Wait for santa to be ready265 atomic266 {267 if (_waitingAtSleigh)268 {269 retry;270 }271 }272


275 //Wait to be released by santa276 atomic277 {278 if (_workingForSanta)279 {280 retry;281 }282 }283 }284 });285 }286

287 public void CallToSleigh()288 {289 atomic290 {291 _waitingInHut = false;292 _waitingAtSleigh = true;293 }294 }295

296 public void HelpDeliverPresents()297 {298 atomic299 {300 _waitingAtSleigh = false;301 _workingForSanta = true;302 }303

304 }305

306 public void ReleaseReindeer()307 {308 _workingForSanta = false;309 }


310 }311

312 public enum WakeState313 {314 ReindeerBack,315 ElfsIncompetent316 }


As with the lock and library-based implementations the AC# implementationis implemented as a linked list of nodes and uses a dummy node. The headand tail nodes, defined on lines 3 and 4, are declared atomic along with theNext property of the Node class defined on 40. The retry on line 32 is usedto block a thread calling Dequeue on an empty queue.

Listing C.11: AC# Based Concurrent Queue Implementation

1 public class Queue<T>2 {3 private atomic Node _head;4 private atomic Node _tail;5

6 public Queue()7 {8 var node = new Node(default(T));9 _head = node;

10 _tail = node;11 }12

13 public void Enqueue(T value)14 {15 atomic16 {17 var node = new Node(value);18 _tail.Next = node;19 _tail = node;20 }21 }22

23

24 public T Dequeue()25 {26 atomic27 {28 var node = _head.Next;29

30 if (node == null)31 {32 retry;33 }34


35 _head = node;36 return node.Value;37 }38 }39

40 private class Node41 {42 public atomic Node Next { get; set; }43 public readonly T Value;44



As with the other concurrent hashmap implementations, the AC# concurrenthashmap implementation utilizes a linked list of nodes for the collision lists.The Node class is defined on line 239. As seen on line 243 its next propertyis declared with the atomic modifier, allowing the STM system to detectchanges to the list. Similarly the Node class’s Value property is declared withthe atomic modifier, as a call to the Add method with a key already presentin the hashmap causes the value of the node representing the key/value pairto be updated. The backing array, seen on line 3, is of type Bucket[] andis declared with the atomic modifier. The atomic modifier allows the STMsystem to track the assignment of a new backing array as a result of resizingthe hashmap. The Bucket class is defined on line 234 and has an atomicValue property to which the first node in the buckets linked list is assigned, ifsuch an item exists. Together with the atomic Next property of the Nodeclass, this allows the STM system to detect any changes to the collision list.

Listing C.12: AC# Based Concurrent Hashmap Implementation

1 public class StmHashMap<K,V> : BaseHashMap<K,V>2 {3 private atomic Bucket[] _buckets;4 private atomic int _threshold;5 private atomic int _size;6

7 public StmHashMap() : this(DefaultNrBuckets)8 {9

10 }11

12 public StmHashMap(int nrBuckets)13 {14 _buckets = MakeBuckets(nrBuckets);


15 _threshold = CalculateThreshold(nrBuckets);16 }17

18 private Bucket[] MakeBuckets(int nrBuckets)19 {20 var temp = new Bucket[nrBuckets];21 for (int i = 0; i < nrBuckets; i++)22 {23 temp[i] = new Bucket();24 }25 return temp;26 }27


30 private Node CreateNode(K key, V value)31 {32 return new Node(key, value);33 }34

35 private int GetBucketIndex(K key)36 {37 return GetBucketIndex(_buckets.Length, key);38 }39

40 private Node FindNode(K key)41 {42 return FindNode(key, GetBucketIndex(key));43 }44

45 private Node FindNode(K key, int bucketIndex)46 {47 return FindNode(key, _buckets[bucketIndex].Value);48 }49

50 private Node FindNode(K key, Node node)51 {52 while (node != null && !key.Equals(node.Key))53 node = node.Next;54 return node;55 }56

57 private void InsertInBucket(Bucket bucketVar, Node node)58 {59 var curNode = bucketVar.Value;60 if (curNode != null)61 {62 node.Next = curNode;63 }64 bucketVar.Value = node;65 }66



69 public override bool ContainsKey(K key)70 {71 return FindNode(key) != null;72 }73

74 public override V Get(K key)75 {76 atomic77 {78 var node = FindNode(key);79 if(node == null)80 {81 //If node == null key is not present in dictionary82 throw new KeyNotFoundException("Key not found. Key: " +

key);83 }84 return node.Value;85 }86 }87

88 public override void Add(K key, V value)89 {90 atomic91 {92 var bucketIndex = GetBucketIndex(key);93 //TMVar wrapping the immutable chain list94 var bucketVar = _buckets[bucketIndex];95 var node = FindNode(key, bucketVar.Value);96

97 if (node != null)98 {99 //If node is not null key exist in map. Update the value

100 node.Value = value;101 }102 else103 {104 //Else insert the node105 InsertInBucket(bucketVar, CreateNode(key, value));106 _size++;107 ResizeIfNeeded();108 }109 }110 }111

112 public override bool AddIfAbsent(K key, V value)113 {114 atomic115 {116 var bucketIndex = GetBucketIndex(key);117 //TMVar wrapping the immutable chain list118 var bucketVar = _buckets[bucketIndex];119 var node = FindNode(key, bucketVar.Value);120

121 if (node == null)


122 {123 //If node is not found key does not exist so insert124 InsertInBucket(bucketVar, CreateNode(key, value));125 _size++;126 ResizeIfNeeded();127 return true;128 }129

130 return false;131 }132 }133 private void ResizeIfNeeded()134 {135 if (_size >= _threshold)136 {137 Resize();138 }139 }140

141 private void Resize()142 {143 //Construct new backing array144 var newBucketSize = _buckets.Length * 2;145 var newBuckets = MakeBuckets(newBucketSize);146

147 //For each key in the map rehash148 for (var i = 0; i < _buckets.Length; i++)149 {150 var bucket = _buckets[i];151 var node = bucket.Value;152 while (node != null)153 {154 var bucketIndex = GetBucketIndex(newBucketSize, node.Key);155 InsertInBucket(newBuckets[bucketIndex],

CreateNode(node.Key, node.Value));156 node = node.Next;157 }158 }159

160 //Calculate new resize threshold and assign the rehashedbacking array

161 _threshold = CalculateThreshold(newBucketSize);162 _buckets = newBuckets;163 }164

165 public override bool Remove(K key)166 {167 atomic168 {169 var bucketIndex = GetBucketIndex(key);170 //TMVar wrapping the immutable chain list171 var bucketVar = _buckets[bucketIndex];172 var firstNode = bucketVar.Value;173


174 return RemoveNode(key, firstNode, bucketVar);175 }176 }177

178 private bool RemoveNode(K key, Node node, Bucket bucketVar)179 {180 if (node == null)181 {182 return false;183 }184

185 if (node.Key.Equals(key))186 {187 _size--;188 bucketVar.Value = node.Next;189 return true;190 }191

192 while (node.Next != null && !key.Equals(node.Next.Key))193 node = node.Next;194


198 _size--;199 node.Next = node.Next.Next;200 return true;201 }202

203 public override IEnumerator<KeyValuePair<K, V>> GetEnumerator()204 {205 atomic206 {207 var list = new List<KeyValuePair<K, V>>(_size);208 for (var i = 0; i < _buckets.Length; i++)209 {210 var bucket = _buckets[i];211 var node = bucket.Value;212 while (node != null)213 {214 var keyValuePair = new KeyValuePair<K, V>(node.Key,

node.Value);215 list.Add(keyValuePair);216 node = node.Next;217 }218 }219 return list.GetEnumerator();220 }221 }222

223 public override V this[K key]224 {225 get { return Get(key); }226 set { Add(key, value); }


227 }228

229 public override int Count230 {231 get { return _size; }232 }233

234 private class Bucket235 {236 public atomic Node Value { get; set; }237 }238

239 private class Node240 {241 public K Key { get; private set; }242 public atomic V Value { get; set; }243 public atomic Node Next { get; set; }244

245 public Node(K key, V value)246 {247 Key = key;248 Value = value;249 }250 }251 }

D SummaryThis master thesis investigates whether language integrated STM is a validalternative to locking in terms of usability, and provides additional benefitscompared to library-based STM. To do so, an extension of C# called AC# wasimplemented. AC# provides integrated support for STM, including conditionalsynchronization using the retry and orelse constructs as well as nesting oftransactions.

In order to develop AC# a set of requirements were defined, detailing howthe underlying STM system should behave in relation to for example trackinggranularity, atomicity level and nesting. Based on the requirements AC# wasdesigned. The design includes a description of new language constructs as wellas a description of modifications to existing language features. A number ofSTM implementations were investigated. Based on this investigation as wellas the requirements and design, an STM library, utilizing the TLII algorithm,was implemented. The STM library was tested using a number of unit tests,ensuring that transactions are executed correctly. In order to perform theactual integration of STM into C#, the open source Roslyn C# compilerwas extended. This required a deep knowledge of the Roslyn project andits structure, which initiated an investigation of Roslyn. Due to the limitedavailability of literature with regards to Roslyn, much of the knowledge obtainedin this area was acquired by reading and debugging Roslyn’s source code. Anumber of unit tests were defined for the Roslyn extension, in order to ensurethat all STM constructs and the integration with the existing language featureswork correctly.

For each of the concurrency approaches: AC#, the STM library and lockingin C#, implementations of the Dining Philosophers problem, the Santa Clausproblem, a concurrent queue, and a concurrent hashmap were created. Theseimplementations were analyzed according to a set of usability characteristicsfacilitating a conclusion upon the usability of language integrated STM. Ourevaluation concludes that AC# is a valid alternative to locking, and providesbetter usability than library-based STM.

172

List of Acronyms

CPU Central Processing Unit

STM Software Transactional Memory

DSTM Dynamic Software Transactional Memory

TL Threads & Locks

CLR Common Language Runtime

CIL Common Intermediate Language

FIFO First In First Out

OOP Object Oriented Programming

IO Input/Output

CAS Compare-And-Swap

API Application Programming Interface

JIT Just-In-Time Compilation

GC Garbage Collection

VB Visual Basic

DSL Domain Specific Language

REPL Read-Eval-Print Loop

MVCC Multiversion Concurrency Control

TRP Transactional Reference Parameter

TOP Transactional Output Parameter

XML Extensible Markup Language

LALR Look Ahead LR

173

Bibliography[1] H. Sutter, “The free lunch is over: A fundamental turn toward concurrency

in software,” Dr. Dobb’s journal, vol. 30, no. 3, pp. 202–210, 2005.

[2] H. Sutter and J. Larus, “Software and the concurrency revolution,” Queue,vol. 3, no. 7, pp. 54–62, 2005.

[3] T. Ugleholdt Hansen, A. Pørtner Karlsen, and K. Breinholt Laurberg,Investigation of trending concurrency models by comparison of performanceand characteristics: Software Transactional Memory, Actor Model, andThreads & Locks. Aalborg University. Department of Computer Science,2015.

[4] Microsoft. (2015, 2) Roslyn wiki. [Online]. Available: https://github.com/dotnet/roslyn

[5] ——. (2015, 2) Core clr repository. [Online]. Available: https://github.com/dotnet/coreclr

[6] T. Harris, S. Marlow, S. Peyton-Jones, and M. Herlihy, “Composablememory transactions,” in Proceedings of the tenth ACM SIGPLAN sym-posium on Principles and practice of parallel programming. ACM, 2005,pp. 48–60.

[7] M. Herlihy, V. Luchangco, M. Moir, and W. N. Scherer III, “Softwaretransactional memory for dynamic-sized data structures,” in Proceed-ings of the twenty-second annual symposium on Principles of distributedcomputing. ACM, 2003, pp. 92–101.

[8] M. Herlihy, V. Luchangco, and M. Moir, “Obstruction-free synchronization:Double-ended queues as an example,” in Distributed Computing Systems,2003. Proceedings. 23rd International Conference. IEEE, 2003, pp. 522–529.

[9] T. Harris, J. Larus, and R. Rajwar, “Transactional memory,” SynthesisLectures on Computer Architecture, vol. 5, no. 1, pp. 1–263, 2010.

[10] M. Herlihy, V. Luchangco, and M. Moir, “A flexible framework for im-plementing software transactional memory,” in ACM SIGPLAN Notices,vol. 41, no. 10. ACM, 2006, pp. 253–262.

[11] T. Harris and K. Fraser, “Language support for lightweight transactions,”in ACM SIGPLAN Notices, vol. 38, no. 11. ACM, 2003, pp. 388–402.

174



https://github.com/dotnet/coreclr

https://github.com/dotnet/coreclr

BIBLIOGRAPHY 175

[12] D. Dice, O. Shalev, and N. Shavit, “Transactional locking ii,” in DistributedComputing. Springer, 2006, pp. 194–208.

[13] A. S. Tanenbaum, Modern operating systems. Prentice Hall Press, 2008.

[14] J. Du↵y, “A (brief) retrospective on transactional memory,” 2010,Located February 2015. [Online]. Available: http://joedu↵yblog.com/2010/01/03/a-brief-retrospective-on-transactional-memory/

[15] N. Shavit and D. Touitou, “Software transactional memory,” DistributedComputing, vol. 10, no. 2, pp. 99–116, 1997.

[16] T. Harris, “Exceptions and side-e↵ects in atomic blocks,” Science ofComputer Programming, vol. 58, no. 3, pp. 325–343, 2005.

[17] M. Corporation, “C# language specification version 5.0,” 2013, LocatedMarch 2015. [Online]. Available: https://www.microsoft.com/en-us/download/details.aspx?id=7029

[18] Microsoft. (2015, 2) Core clr blog. [Online]. Available: http://blogs.msdn.com/b/dotnet/archive/2015/02/03/coreclr-is-now-open-source.aspx

[19] M. M. Michael and M. L. Scott, “Simple, fast, and practical non-blockingand blocking concurrent queue algorithms,” in Proceedings of the fifteenthannual ACM symposium on Principles of distributed computing. ACM,1996, pp. 267–275.

[20] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introductionto algorithms. MIT press, 2009.

[21] R. W. Sebesta, Concepts of Programming Languages, 10th Edition. Pear-son, 2012, ISBN: 978-0-13-139531-2.

[22] Microsoft, “Overview of synchronization primitives,” Located May2015. [Online]. Available: https://msdn.microsoft.com/en-us/library/ms228964%28v=vs.110%29.aspx

[23] ——. (2015, May) Monitor class. [Online]. Available: https://msdn.microsoft.com/en-us/library/System.Threading.Monitor.aspx

[24] ——. (2015, May) Monitor class. [Online]. Available: https://msdn.microsoft.com/en-us/library/System.Threading.Mutex.aspx

[25] ——. (2015, May) Monitor class. [Online]. Available: https://msdn.microsoft.com/en-us/library/System.Threading.SemaphoreSlim.aspx

[26] ——. (2015, May) Monitor class. [Online]. Available: https://msdn.microsoft.com/en-us/library/System.Threading.SpinLock.aspx

http://joeduffyblog.com/2010/01/03/a-brief-retrospective-on-transactional-memory/

http://joeduffyblog.com/2010/01/03/a-brief-retrospective-on-transactional-memory/

https://www.microsoft.com/en-us/download/details.aspx?id=7029

https://www.microsoft.com/en-us/download/details.aspx?id=7029

http://blogs.msdn.com/b/dotnet/archive/2015/02/03/coreclr-is-now-open-source.aspx

http://blogs.msdn.com/b/dotnet/archive/2015/02/03/coreclr-is-now-open-source.aspx

https://msdn.microsoft.com/en-us/library/ms228964%28v=vs.110%29.aspx

https://msdn.microsoft.com/en-us/library/ms228964%28v=vs.110%29.aspx

https://msdn.microsoft.com/en-us/library/System.Threading.Monitor.aspx

https://msdn.microsoft.com/en-us/library/System.Threading.Monitor.aspx

https://msdn.microsoft.com/en-us/library/System.Threading.Mutex.aspx

https://msdn.microsoft.com/en-us/library/System.Threading.Mutex.aspx

https://msdn.microsoft.com/en-us/library/System.Threading.SemaphoreSlim.aspx

https://msdn.microsoft.com/en-us/library/System.Threading.SemaphoreSlim.aspx

https://msdn.microsoft.com/en-us/library/System.Threading.SpinLock.aspx

https://msdn.microsoft.com/en-us/library/System.Threading.SpinLock.aspx

BIBLIOGRAPHY 176

[27] ——. (2015, May) Monitor class. [Online]. Available: https://msdn.microsoft.com/en-us/library/System.Threading.ReaderWriterLock.aspx

[28] M. Herlihy, “Transactional memory,” in Encyclopedia of Parallel Comput-ing. Springer, 2011, pp. 2079–2086.

[29] H. E. Ramadan, I. Roy, M. Herlihy, and E. Witchel, “Committing con-flicting transactions in an stm,” in ACM Sigplan Notices, vol. 44, no. 4.ACM, 2009, pp. 163–172.

[30] K. Ng, M. Warren, P. Golde, and A. Hejlsberg, “The roslyn project,exposing the c# and vb compiler’s code analysis,” White paper, Microsoft(Oct. 2011), 2012.

[31] M. T. Dustin Campbell, “The future of c#,” 2014, build 2014. [Online].Available: http://channel9.msdn.com/Events/Build/2014/2-577

[32] C. Team, “Roslyn ctp introduces interactive code for c#,” 2012, LocatedMarch 2015. [Online]. Available: http://blogs.msdn.com/b/csharpfaq/archive/2012/01/30/roslyn-ctp-introduces-interactive-code-for-c.aspx

[33] D. Campbell, “Going deeper with project roslyn: Exposing thec# and vb compiler’s code analysis,” 2012, lang.NEXT 2012.[Online]. Available: https://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/Roslyn

[34] C. N. Fischer, R. K. Cytron, and R. J. LeBlanc, Crafting a compiler.Addison-Wesley Publishing Company, 2009.

[35] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design patterns:elements of reusable object-oriented software. Pearson Education, 1994.

[36] V. Sadov(VSadov), “Roslyn’s performance,” 2014, Located March 2015.[Online]. Available: https://roslyn.codeplex.com/discussions/541953

[37] A. D. G. (ADGreen). (2014, 4) Provide access to the binder. [Online].Available: https://roslyn.codeplex.com/discussions/541303

[38] N. Gafter, “Flow analysis in the roslyn c# com-piler,” 2011, Located March 2015. [Online]. Avail-able: https://github.com/dotnet/roslyn/blob/master/src/Compilers/CSharp/Portable/FlowAnalysis/Flow%20Analysis%20Design.docx

[39] E. Lippert, “Persistence, facades and roslyn’s red-green trees,” 2012, Located March 2015. [Online].Available: http://blogs.msdn.com/b/ericlippert/archive/2012/06/08/persistence-facades-and-roslyn-s-red-green-trees.aspx

https://msdn.microsoft.com/en-us/library/System.Threading.ReaderWriterLock.aspx

https://msdn.microsoft.com/en-us/library/System.Threading.ReaderWriterLock.aspx

http://channel9.msdn.com/Events/Build/2014/2-577

http://blogs.msdn.com/b/csharpfaq/archive/2012/01/30/roslyn-ctp-introduces-interactive-code-for-c.aspx

http://blogs.msdn.com/b/csharpfaq/archive/2012/01/30/roslyn-ctp-introduces-interactive-code-for-c.aspx

https://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/Roslyn

https://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/Roslyn

https://roslyn.codeplex.com/discussions/541953

https://roslyn.codeplex.com/discussions/541303

https://github.com/dotnet/roslyn/blob/master/src/Compilers/CSharp/Portable/FlowAnalysis/Flow%20Analysis%20Design.docx

https://github.com/dotnet/roslyn/blob/master/src/Compilers/CSharp/Portable/FlowAnalysis/Flow%20Analysis%20Design.docx

http://blogs.msdn.com/b/ericlippert/archive/2012/06/08/persistence-facades-and-roslyn-s-red-green-trees.aspx

http://blogs.msdn.com/b/ericlippert/archive/2012/06/08/persistence-facades-and-roslyn-s-red-green-trees.aspx

BIBLIOGRAPHY 177

[40] Microsoft. (2015, 1) .net compiler platform (”roslyn”) overview. [Online].Available: https://github.com/dotnet/roslyn/wiki/Roslyn%20Overview

[41] R. Hickey. (2015, March) Concurrent programming. [Online]. Available:http://clojure.org/concurrent programming

[42] C. J. Rossbach, O. S. Hofmann, and E. Witchel, “Is transactional pro-gramming actually easier?” ACM Sigplan Notices, vol. 45, no. 5, pp.47–56, 2010.

[43] V. Pankratius, A.-R. Adl-Tabatabai, and F. Otto, Does transactionalmemory keep its promises?: results from an empirical study. Univ., Fak.fur Informatik, 2009.

[44] C. Blundell, E. C. Lewis, and M. M. Martin, “Subtleties of transactionalmemory atomicity semantics,” Computer Architecture Letters, vol. 5, no. 2,2006.

[45] M. F. Spear, V. J. Marathe, L. Dalessandro, and M. L. Scott, “Priva-tization techniques for software transactional memory,” in Proceedingsof the twenty-sixth annual ACM symposium on Principles of distributedcomputing. ACM, 2007, pp. 338–339.

[46] B. Hindman and D. Grossman, “Atomicity via source-to-source transla-tion,” in Proceedings of the 2006 workshop on Memory system performanceand correctness. ACM, 2006, pp. 82–91.

[47] J. Bloch, “E↵ective java (the java series),” 2008.

[48] G. J. Reuter and J. Gray, “Transaction processing: Concepts and tech-niques,” MorganKaufmann, San Mateo, CA, 1993.

[49] J. Han, M. Kamber, and J. Pei, Data Mining: Concepts and Techniques.San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2011.

[50] R. Kumar and K. Vidyasankar, “Hparstm: A hierarchy-based stm protocolfor supporting nested parallelism,” in the 6th ACM SIGPLAN Workshopon Transactional Computing (TRANSACT’11), 2011.

[51] R. Guerraoui and M. Kapalka, “Opacity: A correctness condition fortransactional memory,” Tech. Rep., 2007.

[52] P. Sestoft and H. I. Hansen, C# precisely. MIT Press, 2011.

[53] M. Herlihy and N. Shavit, The Art of Multiprocessor Programming, RevisedReprint. Elsevier, 2012.

[54] M. Mohamedin, B. Ravindran, and R. Palmieri, “Bytestm: Virtualmachine-level java software transactional memory,” in Coordination Mod-els and Languages. Springer, 2013, pp. 166–180.

https://github.com/dotnet/roslyn/wiki/Roslyn%20Overview

http://clojure.org/concurrent_programming

BIBLIOGRAPHY 178

[55] M. Herlihy, “Wait-free synchronization,” ACM Transactions on Program-ming Languages and Systems (TOPLAS), vol. 13, no. 1, pp. 124–149,1991.

[56] R. Ennals, “Software transactional memory should not be obstruction-free,” Technical Report IRC-TR-06-052, Intel Research Cambridge TechReport, Tech. Rep., 2006.

[57] S. Al Bahra, “Nonblocking algorithms and scalable multicore program-ming,” Queue, vol. 11, no. 5, p. 40, 2013.

[58] S. M. Fernandes and J. Cachopo, “Lock-free and scalable multi-versionsoftware transactional memory,” in ACM SIGPLAN Notices, vol. 46, no. 8.ACM, 2011, pp. 179–188.

[59] B. Saha, A.-R. Adl-Tabatabai, R. L. Hudson, C. C. Minh, andB. Hertzberg, “Mcrt-stm: a high performance software transactionalmemory system for a multi-core runtime,” in Proceedings of the eleventhACM SIGPLAN symposium on Principles and practice of parallel pro-gramming. ACM, 2006, pp. 187–197.

[60] J. M. P. Cachopo, “Development of rich domain models with atomicactions,” Ph.D. dissertation, Universidade Tecnica de Lisboa, 2007.

[61] R. Zhang, Z. Budimlic, and W. N. Scherer III, “Commit phase intimestamp-based stm,” in Proceedings of the twentieth annual sympo-sium on Parallelism in algorithms and architectures. ACM, 2008, pp.326–335.

[62] Y. Lev, V. Luchangco, V. Marathe, M. Moir, D. Nussbaum, and M. Ol-szewski, “Anatomy of a scalable software transactional memory,” in Proc.4th ACM SIGPLAN Workshop on Transactional Computing, 2009.

[63] H. Avni and N. Shavit, “Maintaining consistent transactional stateswithout a global clock,” in Structural Information and CommunicationComplexity. Springer, 2008, pp. 131–140.

[64] T. Riegel, C. Fetzer, and P. Felber, “Time-based transactional memorywith scalable time bases,” in Proceedings of the nineteenth annual ACMsymposium on Parallel algorithms and architectures. ACM, 2007, pp.221–228.

[65] R. Elmasri, Fundamentals of database systems. Addison-Wesley, 2011.

[66] R. C. Martin, Clean code: a handbook of agile software craftsmanship.Pearson Education, 2008.

[67] M. Fowler. (2015, May) Unit testing. [Online]. Available: http://martinfowler.com/bliki/UnitTest.html

http://martinfowler.com/bliki/UnitTest.html

http://martinfowler.com/bliki/UnitTest.html

BIBLIOGRAPHY 179

[68] A. Nunes-Harwitt, “Cps recursive ascent parsing,” in Proc. of Internat.LISP Conf, 2003.

[69] D. Graham, E. Van Veenendaal, I. Evans, and R. Black, Foundationsof software testing: ISTQB certification. Course Technology CengageLearning, 2008.

[70] Microsoft. (2015, April) Compiler warning (level 1) cs0420. [Online].Available: https://msdn.microsoft.com/en-us/library/4bw5ewxy(VS.80).aspx

[71] M. L. Scott, “Synchronization,” in Encyclopedia of Parallel Computing.Springer, 2011, pp. 1989–1996.

[72] Microsoft. (2015, May) Monitor class. [Online]. Available: https://msdn.microsoft.com/en-us/library/de0542zz.aspx

[73] ——. (2015, 2) Roslyn wiki. [Online]. Available: https://github.com/dotnet/roslyn/wiki

[74] ——. (2015, 2) Roslyn samples and walktroughs. [Online]. Available:https://github.com/dotnet/roslyn/wiki/Samples-and-Walkthroughs

[75] M. Torgersen. (2015, April) Languages features in c# 6 and vb 14. [Online].Available: https://www.codeplex.com/Download?ProjectName=roslyn&DownloadId=930852

[76] M. Michaelis, “A c# 6.0 language preview,” MSDN Magazine, vol. May,pp. 16–23, 2014.

[77] ——, “The new and improved c# 6.0,” MSDN Magazine, vol. Oktober,pp. 18–24, 2014.

[78] M. Torgersen. (2015, April) Upcoming features in c#. [Online].Available: https://www.codeplex.com/Download?ProjectName=roslyn&DownloadId=930852

[79] S. Imam and V. Sarkar, “Savina-an actor benchmark suite,” in 4th In-ternational Workshop on Programming based on Actors, Agents, andDecentralized Control, AGERE, 2014.

[80] R. Guerraoui, M. Kapalka, and J. Vitek, “Stmbench7: a benchmark forsoftware transactional memory,” Tech. Rep., 2006.

[81] C. A. R. Hoare, “Communicating sequential processes,” Communicationsof the ACM, vol. 21, no. 8, pp. 666–677, 1978.

[82] J. A. Trono, “A new exercise in concurrency,” ACM SIGCSE Bulletin,vol. 26, no. 3, pp. 8–10, 1994.

https://msdn.microsoft.com/en-us/library/4bw5ewxy(VS.80).aspx

https://msdn.microsoft.com/en-us/library/4bw5ewxy(VS.80).aspx

https://msdn.microsoft.com/en-us/library/de0542zz.aspx

https://msdn.microsoft.com/en-us/library/de0542zz.aspx

https://github.com/dotnet/roslyn/wiki

https://github.com/dotnet/roslyn/wiki

https://github.com/dotnet/roslyn/wiki/Samples-and-Walkthroughs

https://www.codeplex.com/Download?ProjectName=roslyn&DownloadId=930852




language integrated stm in c# using the roslyn...

Documents