advanced c language for engineering

Advanced C Language for Engineering © V De Florio

ATHENS CourseATHENS Course

Advanced C Languagefor Engineering

Vincenzo De Florio17 – 21 November 2003

V.ppt-1.12003-11-10

2

Advanced C Language for Engineering ( V De Florio)

Overall structureOverall structure

• An introduction to C• The FILE methodology. Classes. Data hiding.

Examples (class assoc, objalloc, vf. . . )• Literate programming. The cweb tools• The GNU autotools• Other classes for embedded systems (class tom,

the DepAuDE framework)• Linguistic support using C, Lex and YACC.

Examples (the icgi interpreter, class FN, PvmLinda, the Ariel language...)

• Exercise sessions, case studies, project works

3


StructureStructure

• Introduction• First examples• Variables• Fundamental data

types• Loops• Conditional statements

& other instructions• The C preprocessor• Functions• Variables again• Operators

• Standard I/O functions• Derived types• Pointers and arrays• Use of pointers• Type casts &

conversions• Bitwise operators• Standard streams, files• Structures, bitfields,

unions• LLP• Data hiding

http://www.esat.kuleuven.ac.be/~deflorio/c/athens03.prn.gz

4


First examplesFirst examples

main ( ){printf(“hello\n”);}

Function name

Function withno arguments

Program starts here...

...and ends here

Print string “hello” and

character ‘\n’

5



• Log into the system login namepassword

• Type in the hello program in file first.cActivate an editor, e.g., xedit, pico, or vi for instance xedit first.csave the file at the end

• Compile the programcc -o first first.c (or gcc -o first first.c)

• Execute the program ./first

• Modify the program so that it prints another string / two strings / … jump to “Compile the program”

6


First ExamplesFirst Examplesprintffollowedby “(“means“Call function printf”

7



main() { int inf, sup, step; float fahr, celsius; inf = 0; sup = 300; step = 20; fahr = inf; while (fahr <= sup) { celsius = (5.0/9.0) * (fahr-32.0); printf("%4.0f %6.1f\n", fahr, celsius); fahr = fahr + step; }}

Integer and real numbers

Loopstart,loopend

Formatstring

8



9



• Type in the conversion program in file second.c• Compile the program• Execute the program• Modify the program:

Change the format stringsChange stepsDownto vs. To (5 / 9.0) (5 / 9) fahr - 32 vs. fahr - 32.0 fahr += fahr jump to “Compile the program”

10


Definition of Variables (1)Definition of Variables (1)

• To define a variable or a list of variables, one has to mention:

TYPE LIST ;• For instance:

int a, b_b, c2, A;double f, F=1.3, dimX;

• With LIST one can also initialize variables, like it has been done for F in the above example

• One can define variables only at the beginning of a compound instruction such as { TYPE LIST; … ; INSTRUCTIONS }

11



main() { int inf, sup, step; float fahr, celsius; inf = 0; sup = 300; step = 20; fahr = inf; while (fahr <= sup) { celsius = (5.0/9.0) * (fahr-32.0); printf("%4.0f %6.1f\n", fahr, celsius); fahr = fahr + step; }}

init test

increment

for (fahr = inf; fahr <= sup; fahr += step) {...

12



• Fahr += step == Fahr = Fahr + step Fahr++ == Fahr = Fahr + 1

• Compound statements in C: { s1; s2; …; sn; } s1, s2, …, sn;

• Relational operators <=, <, !=, ==, >, >= ... Return 0 when false, 1 when true

• “test” is any operation• 0 = false, non-0 = true• Both these statements are valid:

while ( x == y ) { … while ( x = y ) { …

13



1. Let a be 0

2. Repeatthe loopwhile “a = 0”is “true”

3. So thewhile loopis simplyignored!

14



1. Let a be 1 (returns 1)

2. Repeatthe loopwhile “a = 1”returns “true”

3. So thewhile loopnever ends!

15



The initialvalue of ais undefined!

Here is -85899…

Here is 0 (cygwin)

16


Fundamental data typesFundamental data types

• Four types of integers:char, short, int, long

• Two types of floating point numbers:float, double

• Corresponding formats:%c, %d, %d, %ld%f, %lf

• Sizes:sizeof(char) <= sizeof(short) <= sizeof(int) <= sizeof(long)sizeof(float) <= sizeof(double)

• Practically speaking:char==1, short==2, int==2 or 4, long==4float==4, double==8

17



• Check the actual size of type on your workstations with function sizeof()

18



Same result on:

Linux / gccSunOS / (g)ccHP-UX / ccWin / m$ visual c

But this is not 100% guaranteedon all systems!

19


Type Type ““charchar””

• A char in C is just an integer• The only difference between a char and, e.g., an int

lies in the number of bytes that are used• In C, we can write, e.g.

int i; char c;i = ‘A’;c = 65;if ( i != c ) printf(“not an “);printf(“ASCII system\n”);

• A char is a small number that, when sent, e.g., to the screen, corresponds to a character

• In C there is no difference between A characterThe code of character

20



• Symbols such as ‘A’ are just symbolic constants• Think of them as a #define statement:

#define ‘A’ 65#define ‘B’ 66 . . .

• Every time you encounter a APOSTROPHE CHARACTER APOSTROPHEyou are dealing with a symbolic constant thatdefines a small integer, that when printed can be interpreted as a character

21



• Example• A = 66;

printf(“ As a number, A is \t%d\n”, A);printf(“ As a character, A is \t%c\n”, A);

%c ==“print as acharacter”

%d ==“print asan integer”

22


Types: some formatting charactersTypes: some formatting characters

• %d : print as an integer%c : print as a character%ld : print as a long%f : print as a float%lf : print as a double

23



24



25


Types: an interpretation of a same Types: an interpretation of a same ““substancesubstance”” –– bytes! bytes!

(To be explained later on)

The first four bytes ofA are interpreted indifferent ways

26


Loops in CLoops in C

While loops: while ( op1 ) op2;• No initialisation• Both op1 and op2 are operations

returning integers• while op1 is non zero, do op2• op2 can be a compound instruction, e.g.,

{ op21; op22; … ; op2N }

• relational operations return an integer!• a <= b is 1 when a<=b and 0

otherwise

27



• Which of these loops are correct? And what do they mean? while ( -25 ) a++ ; while ( read_next_character ( ) ) car = car + 1; while ( a + 1 ) { a = a + 1; b = 0; } while ( a ) a = b, b = tmp, tmp = a;

28



While loops: do op2 while ( op1 );• No initialisation• Both op1 and op2 are operations

returning integers• Do op2; then, while op1 is not zero, do op2 again

• op2 can again be a compound instruction, e.g., { op21; op22; … ; op2N }

29



• For loops: for ( op1; op2; op3 ) op4;• Operation op1 is the initialisation• Operation op2 is the condition• Operation op3 is the conclusive part• Operation op4 is the body of the loop, • Both op1 and op3 can be compound (comma-

based!) instructions • Usually used for iterations, e.g.for (i=0; i<n; i++) { do_that(); }

• Exercise: modify second.c so to change the while into a for.

• Exercise: modify the previous example so to print the table in inverse order.

30



• Which of these loops are correct? And what do they mean? for (;;) a++; for (; a<5; a++) ; for ( i=0; i<n; { i++; n--; } ) … /* do something */ for ( i=0, j=n; i<n; i++, j-- ) … /* do sthg */

31



• Loops are an important source of performance for the compiler and for optimizers

• Some computer architectures and compilers can exploit them to achieve a very high speed-up

• This is called “loop level parallelism (LLP) exploitation”

32


Conditional Conditional statementsstatements

• Statement “if”: if (condition) action1; else action2;

• “?:” (condition)? action1 : action2;

• Statement “switch”: switch(const integer expression) {

case value1: op1; break;case value2: op2; break;…case valuen: opn; break;default: opn+1;

}

33


Other instructionsOther instructions

• break;• continue;• goto label;y:while (a < b) {

break;b--;

}x:…

a++;…break; continue;

goto x;

goto y;

34


The C PreprocessorThe C Preprocessor

• The C compiler was originally composed of four components: cpp | cc | as | ld .c -> .i -> .s -> .o

• Component cpp is the C preprocessor cpp looks for preprocessor commands (#cmd’s)#cmd’s start with “#” on column 1 cpp converts a text file f into a text file f’ All the valid “#”-statements are removed and

substituted with C strings or text files

35



• Examples:• #define BEGIN {• #define END }• #define IF if(• #define THEN )• #define INT32 long• #define MAX(A, B) ((A) > (B)? (A):(B))• #define SQR(x) x * x

• IF SQR(x) == x THEN printf(“x==1\n”); is translated intoif( x * x ==x ) printf(“x==1\n”);

Dangerous

36



• Inclusion of external files1. #include <stdio.h>2. #include “assoc.h”

• #ifdef … [ #else … ] #endif• #ifndef … [ #else … ] #endif

• Differences between 1. and 2.• Use of ifdef/ifndef.

37



• #include <stdio.h> is equivalent to• #include “prefix/stdio.h”• On many UNIX systems, prefix == /usr/include

• /usr/include contains the header files of the C standard library stdio.h, string.h, time.h, ctype.h, math.h, …

• A header file is the user interface of a library or a part of a library

• stdio.h defines the interface to a subset of the C standard library

• stdio.h interfaces the standard I/O functions and defines symbols thereof

38


FunctionsFunctions

• Functions are names that points to some memory location where code has been stored by the compiler

• They take arguments and return a value of some typeE.g., double cos(double a);This is a function prototype, with which we declare a

function, called cos, that expects and returns a double• To call a function, we just name it and pass an

argument to itcos (3.1415); /* cos (pi) */

39


FunctionsFunctions

• The name of a function is just a number – its address in the code segment

40


FunctionsFunctions

• Functions can beDeclaredDefined

• A prototype declares a function “There’s a function that looks like that and that…”

• Defining a function means specifying what the function shall do “The function does this and this…”Memory is allocated in the code segment

41


FunctionsFunctions

• To define a function one has to supply the compound instruction that it has to execute:

double addd (double a, double b);double addd (double a, double b){

return a + b;}

Prototype (declaration)

Definition

42


FunctionsFunctions

• Recursion is allowed• C supports a single-level of functions that

can be compiled separately

43


FunctionsFunctions

• return closes the function and returns an output value (default: integer) to the caller.

• Arguments are passed by value: this means that the arguments are copied in temporary variables.

• The only way to let a function modify an argument is by passing the address of the object to be modified.

• Operator & returns the address of a variable.

44


FunctionsFunctions

• Functions have the following structure:• [ type ] name ( [ args ] ) {

[ declarations ] [ instructions ] }

• int main() {int i; int power (int, int);for (i=0; i<10; ++i)printf("%d %d\n", i, power(2,i));

}int power (int x, int n) { int i, p;

for (i = p = 1; i <= n; ++i)p=p*x;

return (p);}

45


• Two classes of variablesDynamically allocatedStatically allocated

• A variable is dynamically allocated when it is local to a function or a compound instruction and is not declared as “static”

• Examplesmain () {

int i;for (i=0; i<n; i++) {

int j;j = i * i;

}}

Again, on VariablesAgain, on Variables

Automaticvariables(hold undefinedvalue)

46



• If the variable is an automatic one, then the initialisation is done each time the variable is allocated (each time the function in which the variable resides is called and the corresponding compound instruction is executed).

47



• A variable is defined at compile time when it is a global variable (I.e., a variable defined outside any

function) It is a variable defined as “static”

• Exampleint fd;int open(int tmp) {

int j;static int first;

}

Global

Automatic

Static

48



• C is case sensitive: int J; float j; is valid• The scope of variables is such that a new name

overrides an old one:int h = 3;{ int h = 4; printf(“h == %d\n”, h);}printf(“h == %d\n”, h);

49


OperatorsOperators

• binary +, -, *, /, %• unary -• precedences: • (+,-) Ð (*,/,%) Ð (-) Ð (||)

Ð (&&) Ð (==, !=) Ð (> >= < <=)

• Note:pa Ð opb if opb has higher precedence than opa

50


OperatorsOperators

• Expressions such as:i < lim && (c=getchar()) != ’\n’ && c != EOFdo not require extra parentheses: = Ð !=, hence parentheses are required around c=getchar().

• Logic clauses are evaluated left-to-right. Evaluation stops when the truth value of a logic expression is found.

51


Standard functions for input/outputStandard functions for input/output

• Defined in stdio.h• Require the stdio.h file to be included• This defines functions such as:

int getchar();int putchar(int c);

• getchar reads the next character in the standard input stream or an end-of-file value (EOF)

• getchar returns the character as one byte stored into a 2-byte or 4-byte integer (an int value)

• Putchar puts on the standard output stream the character we pass as an argument

52



void main() { char c;

c = getchar(); while ( c != EOF ) { putchar ( c ); c = getchar(); }}

Are anyfaults

present?Whichones?

c is declaredas a char

getchar eitherreturns a

char or EOF

53



void main() { char c;

c = getchar(); while ( c != EOF ) { putchar ( c ); c = getchar(); }}

c is a chargetcharreturnsan int!

This loopmay never be

verified

54


void main() { int c;

while ( c = getchar() != EOF ) { putchar ( c ); }}


Are anyfaults

present?Whichones?

( )

implicitparenthesesOK

If stdin = ‘h’ ‘e’ ‘l’ ‘l’ ‘o’, EOFwhat goes to stdout?

55


ExercisesExercises

• Exercise: the just seen program can be easily adapted to work, e.g., as a “word counter” (reports the number of characters, words, and lines in the input), or as a filter to remove blanks or tabs, and so forth

• Input and output can be redirected with < and >. Pipelines of programs can be built by chaining the programs with |

56


Derived typesDerived types

• The following operators define complex types derived from the basic types of C:

* operator pointer-to,[] operator vector-of,() operator function-pointer-to

57



• char *p; : p is of type “pointer-to-chars”• float v[10]; : v is a vector, i.e., a pointer to the

beginning of a memory area allocated by the system and consisting of 10 floats, i.e., v[0], . . . , v[9].

• int getX(); : getX is a pointer to a function returning an int.

58



• Operator [] has higher priority with respect to operator *. This means that

int *v[10]

declares v as a vector of ten pointers-to-int. Parentheses are necessary to change the meaning:

int (*v)[10]

means that v is a pointer to a vector of ten integers

• What’s the difference in terms of sizes?• What if short instead of int?

59



1. int *pi; pointer to integer;2. char **argv; pointer to pointer-to-char;3. float *(fvp)[5]; pointer to vectors-of-5-floats4. long (*fp)(int); pointer to function reading an

int and returning a long.

60



• The address-of (&) operator returns the address of a variablechar c; char *pc; pc = &c;

• Operator *, applied to a pointer, returns the pointed object.char c1 = ’a’; char *p = &c1;char c2 = *p2; /* c2 == ’a’ */void func(char *s) { printf("bye %s\n", s) }main() { void (*pfunc)(char*) = func;

*(pfunc)("hello");}

61


void swap (int a, int b) { int t; t = a; a = b; b = t;}

void main() { int a, b; a = 5, b = 6; swap (a, b); printf(“a = %d, b = %d\n”, a, b);}


Are anyfaults

present?Whichones?

These arenot the same

variables!

62



Step Var Address Contents SPint a, b; a 1024 ? 1028 b 1028 ? 1032a = 5, b = 6; a 1024 5 b 1028 6swap(a, b)int a, int b a 1032 5 1036 b 1036 6 1040int t; t 1040 ? 1044 t = a;a = b;b = t a 1032 6 b 1036 5 t 1040 5} 1032 a 1024 5 b 1028 6

63



• Pointers solve the problem of the lack of “call-by-reference” in C functions. For instance, in order to realize a function that swaps its argument, one may use the following strategy:

int swap(int *a, int *b) { int t;t = *a, *a = *b, *b = t;

}

• The caller then needs to specify the addresses of the operands it wants to swap, as in

swap(&i, &j).

64



void swap (int* a, int* b) { int t; t = *a; *a = *b; *b = t;}

void main() { int a, b; a = 5, b = 6; swap (&a, &b); printf(“a = %d, b = %d\n”, a, b);}

65



Step Var Address Contents SPa = 5, b = 6; a 1024 5 1028 b 1028 6 1032swap(&a, &b)int *a, int *b a 1032 1024 1036 b 1036 1028 1040int t; t 1040 ? 1044 t=*a;*a=*b;*b=t *a 1024 6 *b 1028 5 t 1040 5} 1032 a 1024 6 b 1028 5

66


Pointers and arraysPointers and arrays

• Note: declaring an array means1. declaring a pointer2. allocating memory for the pointed objects.

• The name of the array is indeed a pointer to the first element of the array. In C lingo, this iswritten as vect == &vect[0].

• Exercise: write a program, called report, that reports the occurrences of each digit char in the input stream. Use an array of ten elements corresponding to the ten decimal digits. Produce a histogram at end-of-input.

67



• Exercise: use two programs, one that outputs the ten integer numbers that

count the occurrences of each digit char in the input stream,

the other one that creates a histogram of its input values.

• Then use a pipeline to hook the two programs:report2 | histogram

68



• Arrays and pointers are strictly related to each other: In particular, if int a[10]; then

a == &a[0], a+1 == &a[1], … and so forth. • In other words, *(a+i) == a[i]• Any indexed array is equivalent to a pointer plus

some offset, and vice-versa• Big difference: the array is a constant, i.e., it can

never appear on the left of the = sign, as in

a = pa;

or in

a ++;

69



• When passing an array to a function we are indeed passing the address of its first element; this address is copied (call-by-value) in a temporary variable. This variable may be a pointer.

• Example:char s[7] = "hello!"; /* s is an array */int i = strlen(s);int strlen (char *x) { /* x is a pointer */ int n; for (n=0; *x; x++) /* hence, can be modified */

n++; return (n);}

70



• If p is a pointer, p++ lets p point to the next item. It is the language that takes care of moving p of the right amount of memory.

• For instance (let us assume an int is 2 bytes and a double is 8 bytes):

int *p; p == 1222 p+1 == 1224double *p; p == 5644 p+1 == 5660

and so forth

• In other words: if p is a pointer to an object of type t, then p+n points n objects further and p-n points n objects before.

• The actual size of the object doesn’t matter.

71


Pointers to charsPointers to chars

• Strings are available in C as arrays of characters. Any sentence enclosed between two quotes (“) is an array of characters ending with character ’\0’ (NULL).

• For instance, "hello" means ’h’, ’e’, ’l’, ’l’, ’o’, 0• A very common mistake in C:

char s1[10] = "Hello ";char s2[10] = "World";s1=s2; /* ERROR */

72


Pointers to charsPointers to chars

• As strings are arrays, we can easily pass a string to a function by passing its name, which points to its characters:

char a[] = “Kennedy";printf("Vote %s for president.\n", a);/* a = &a[0] */

• Variables defined within a function cannot be “seen” from other functions.

73


Pointers to functionsPointers to functions

• Exercise: write a program that uses an array of pointers-to-functionInput char == number i -> call array[i ]

74


Pointers and arrays (2)Pointers and arrays (2)

• Function strcpy(char *a, char *b);• Assumption: NULL-terminated strings. Note that

NULL is (int)0, i.e., “false”• Function strcmp(char *s, char *t): returns a negative

number if s < t, 0 if s == t, a positive number if s > t:

strcmp(char *s, char *t) {for ( ; *s == *t; s++, t++)

if (*s == ’\0’) return (0);return (*s - *t);

}

75



• Is this #define OK?• #define STRCPY(a,b) while (*a++ = *b++) ;

76



• To declare multidimensional arrays one declares arrays of arrays. For instance,

static int day_of_month[2][13] = { { 0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}, { 0, 31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31} };

int day_of_year(int day, int month, int year) { int i, leap =year%4 == 0 && year%100 != 0 || year%400 == 0;

for (i=1; i<month; i++)day += day_in_month[leap][i];

return (day);}

77



• int v[i][j] vs. int v[i,j]• Entries are stored “by row”: the rightmost index is

the one that varies the most when entries are referenced in the order they are stored.

• Initialisation: using curly brackets.• When passing a bidimensional array to a function, it

is mandatory that the number of columns be specified. For instance:

f(int day_in_month[2][13]), orf(int day_in_month[][13]), orf(int (*day_in_month)[13]),

• i.e., pointer to array-of-13 integer entries.

78



• “Entries are stored by rows” means that, e.g.,int v[100][200];

• is allocated the same way as aint v[100 × 200];

i.e., as if it were a mono-dimensional array of 20000 int’s.

79



• Fortran stores objects the opposite way with respect to C:

for (i..) for (j..) a[i][j]• is equivalent to

DO I.. DO J.. A(J,I)• Accessing element (i, j) in a n × m matrix means

accessing element k = i × m + j in the associated mono-dimensional array.

• The same applies for N-dimensional matrices, N > 2.

80



• Note how, in order to compute value k for an N dimensional matrix whose dimensions are (d1, d2, . . . , dN), it is necessary to know the values d2, . . . , dN:

k = f(d2, . . . , dN).• Note also that when we need to access sequentially

all the elements of a multidimensional matrix, it is preferable to use a pointer initialised to the first entry of the matrix.

81



• #define N 500#define M 500main() { int a[N][M];int i, j, c;int *pa = & (a[0][0]);int *pl = & (a[N-1][M-1]);

while (pa < pl) { for (i=0; i<N; i++) *pa = 0; for (j=0; j<M; j++) c = *pa + 1; { a[i][j] = 0; *pa = c; c = a[i][j] +1; pa++; a[i][j] = c;} }HP9K 0.7–0.8s 1.2–1.3 s

SUN3 1.1 s 2.4 s

82



• Even when the access is not sequential, it is possible to exploit specific access patterns (e.g., constant stride access).

• An interesting alternative with respect to multidimensional arrays is by using pointers. For instance, the computational cost to access entry (i, j) in a n × m matrix is the one for computing multiplication (i * m) and addition (i * m + j).

83



• If we change fromint a[100][100];

toint** a;

and if we properly allocate the 100 pointers in the row and, for each of them, the memory required for storing 100 int’s, then

accessing entry (i, j) means executing *((*(a+i)+j))

that has a computational cost of only two additions.• Furthermore, no dimension information is required

anymore to access any element of the array.

84



• Array of pointers – an example• char *name_of_month (int n) {

static char *names[] = {"illegal","January","February",. . ."December“};

return ((n < 1 || n > 12)? names[0] : names[n] ;}

85



• Argc and argv: a mechanism that allows a program to inspect the strings on the command

• line.• For instance, command echo:

main(int argc, char *argv[]) {while (--argc > 0)printf("%s%c", *++argv, (argc>1)?’ ’:’\n’ );

}• Exercise: compute 2 3 4 + * (in inverse Polish

notation).• Exercise: write a function that associates user

functions to the command options.

86



• Exercise: write a function that sorts the entries of an array of integers. Modify the function so that it requires a pointer-to-function,

int (*confr)(int,int), to realize sortings in increasing vs. decreasing order.

• Exercise: “opaque” function, working with pointers to object of unknown size.

• Exercise (array of functions): design a scanner of a simple grammar. Tokens must correspond to the entries in an array of functions.

87


Using pointersUsing pointers

• Using pointers in C may be described as a connection-oriented service

• One has to open a connection to the memory service, then use the service (read/write mode), then disconnect (close) the service

• Example: int *pi;

pi = malloc(4); /* open:memory-alloc:4 bytes */ *pi = 5; /* use:write-memory:store-in(p,5) */

j = 1 + *pi; /* use:read-memory:get-from(p) */

free(pi); /* close:free-memory-referred-in(p) */

88



1) Definition of a pointer int *pi;

Note: by doing so, one allocates memory for the pointer, not for the pointed

2) Memory allocationpi = malloc(4); or better malloc(sizeof(int));

Note: this is a request for memory allocation.malloc returns NULL (def:stdio.h) when the request cannot be fulfilled, a value different from NULL when the request can be fulfilled

3) Memory access*pi = … or … = … *pi …

Note: the valid address returned by malloc points to some memory location where the requested memory resides

89



4) Memory releasefree(pi);

Note: the memory pointed to by pi is freedNote: pi is not “erased”! Still it holds the stale reference (be careful…)

90



• A common mistake:

char *p;*p = …

• This is faulty (use-before-connect fault)• Weird consequences…

91



• A common mistake:

free(p);*p = *p + 1;

• This is faulty (use-after-disconnect fault)• Weird consequences…

92


void main() { int *a, *b;

int c[10], d[10];

a = malloc(10*sizeof(int));b = calloc(10, sizeof(int));a = b;a = c;d = a;

}


No checkon return

values

The areapointed by

a is lostIllegal:

arrays areconst

93


Working with the debuggerWorking with the debugger

#include <stdio.h>

main()

{

char *p = NULL;

printf("dereferencing a NULL pointer!\n");

*p = 'A';

printf("done.\n");

}

Big, big mistake…

94



95


A useful tool!A useful tool!

96



Where did the fault take place?

97



98


Commands:

l=list

b=break- point

r=run

s=step(alsodisplay var)

99


Command Command ““printprint””

100


Command Command ““printprint””

101



102


Command“set”

103


ConstantsConstants

• scientific notation, e.g., 1.5e3 -> double• postfix notation, e.g., 145L -> long• prefix notation:

’0x44’ -> unsigned int, hexadecimal, ’044’ -> unsigned int, octal

• constant char: ’x’ = character x -> char• special constants, e.g., \n, \t, \0, \\, \" -> char• “bit patterns”: \ddd, ddd being an octal number.• string constants, e.g., "string" or "".

104


Type casts and conversionsType casts and conversions

• Implicit type casts occur when, in expressions, operands belong to different types.

• Conversions obey the following rules:char and short Þ int , float Þ doubleif an operand is double , the other becomes

double and the result is doubleotherwise, if an operand is long , the other

becomes long and the result is a longotherwise, if an operand is unsigned, the other

one becomes unsigned and the result is unsigned.

otherwise, operands and result are int .

105



• Converting a string of digits into a number, and vice-versa, requires specific support. Functions are available for this, e.g., atoi():

int atoi(char s[]) { int i, n;n = 0;for (i=0; s[i]>=’0’ && s[i]<=’9’; ++i)

n=10*n + s[i] -’0’;return (n);

}

• Note how expression s[i] - ’0’ converts numeric character s[i] into the digit it represents.

106



• The following function can be used to convert an uppercase character into its lowercase counterpart

int lower( int c){

if (c >=’A’ && c <= ’Z’)return (c + ’a’ - ’A’);

elsereturn (c);

}• Note: this program only works for code tables in

which ’a’ follows ’A’. This is true with ASCII and false with, e.g., EBCDIC.

107



• Explicit cast:• The cast operator changes the type of an object.

For instance, the following expression:celsius = ( (double)5 /9) * (fahr-32.0);

is equivalent tocelsius = (5.0/9.0) * (fahr-32.0);

• Casting is invoked by specifying a type between parentheses.

108


IIncrement/decrement operatorsncrement/decrement operators

• IN C: int i = 0; in Assembly:i++; DATA segment

i DB 0. . .INC i

• Operators such as ++ or -- may have a direct counterpart in the machine language.

• Operator ++ increments the contents of a variable. x = n++ is not equivalent to x = ++n.

• Operator -- decrements the contents of a variable. x = n-- is not equivalent to x = --n.

109


ExercisesExercises

• Exercise: function purge(char s[], int c) removes all occurrences of c from s[].

• Exercise: functions strcpy() and strcat().

110


ExercisesExercises

• int strlen (char *p) { int i=0;while (*p++) i++;return i;

}• *p returns the char pointed to by p. *p++ does the

same, but increments the pointer afterwards.• When does the while loop ends?• This is an alternative way to write function strlen:

int strlen (char *p) { char *q = p;while (*p++) ;return q - p - 1;

}

111


Bitwise operatorsBitwise operators

• The following six operands can be applied to any integer expression:& : bitwise AND| : bitwise ORˆ : bitwise exclusive OR<< : left shift>> : right shift˜ : unary complement.

112



• Bitwise AND can be used to set up “masks”, e.g.,c = n & 0177;

which zeroes all the bits from bit 8 onward. • Bitwise OR sets bits:

x = x | MASK sets to 1 the bits that are set in MASK.

• << and >> are respectively arithmetical shifts to the left and to the right (multiplication re: division by 2).

• Operator ˜ turns each 1 into 0 and vice-versa; it is used in expressions like, e.g.,

x & ~ 077,which zeroes the last bits of x. Note that this is independent of the actual size of x, and hence it is “more portable” than, e.g., x & 0177700.

113



• Let’s consider the following function:unsigned int MIDbit (unsigned x, unsigned start, unsigned num)

{return((x >> (start+1-num)) & ~(~0 << num));}

• For instance, MIDbit(x, 4, 3) returns the three bits at position 4, 3, and 2.

• x >> (p+1-n) shifts the bits of interest on the rightmost position in the word.

• ~ 0 is 11...1; • n shifts to left lead to 11...1 00..0• complementing this value we reach 00...0 11..1

n zeroes

n ones

114



• Binary operators+ . / % << >> & ˆ and |

can use notation e1 op= e2

instead of e1 = (e1) op (e2).

• Note that x *= y+1 is equivalent to x = x*(y+1).• An example based on botwise operators follows:

int bitcount(unsigned n) { int b;for (b=0; n != 0; n >>= 1)

if (n & 01) b++;return (b);

}

115


A curiosityA curiosity

• Let us consider the following two programs:

• main() { int n; main() { int n; n = 0; n = 0; n = n+2+n*n; n += 2+n*n; } }

116


A curiosityA curiosity

• When compiled with option "-Qproduce .s" on a Sun workstation, the output Assembly files differ in the following lines:

16,19c16,17< movl a6@(-0x4),d1< addql #0x2,d1< addl d1,d0< movl d0,a6@(-0x4)---> addql #0x2,d0> addl d0,a6@(-0x4)

117


The FILE classThe FILE class

• Using files in C may be described as a connection-oriented service

• One has to open a connection to the file service, then use the service (read/write mode), then disconnect (close) the service

• Example: FILE *pf;

pf = fopen(“readme”, “r”); /* openfile:(readme,rm) */ fprintf(pf, “hi\n”); /* usefile:store-chars(pf,”hi\n”) */

fread(buffer,1,1,pf); /*usefile:read-chars(pf,1,buf) */

fclose (pf); /* closefile(pf) */

118


StructuresStructures

• Keyword struct defines a “record.” An example follows:

struct cd {char author[30];int year;char producer[20];

};• This is a declaration of a type: no element of this

type has been defined so far. No memory has been allocated.

• It is now possible to declare objects of type struct cdstruct cd x, y, z;

119



• The members of a struct can be accessed via the “dot-operator” (“.”). For instance,

struct cd d;leap = d.year%4 == 0 &&

d.year%100 != 0 ||d.year%400 == 0;

• Nesting of structures is allowed: struct a { struct disco c; } d;

• Access: d.c.year.• Typedefs.

120



• A pointer to a structure can be declared, e.g., as follows:

struct disco *a;• Access:

(*a).year;

• a->year is equivalent to (*a).year

121


TypedefTypedef

• What does, e.g., this statement do?

float complex[2];

• Consider now

typedef float complex[2];

• We have defined a new type• Example:

complex a;a[0] = 0.0, a[1] = 1.0;

122


TypedefTypedef

• General rule:

consider the definition of a complex object, called x

write “typedef x”

Now x is no more an identifier. It’s a type

123


TypedefTypedef

• Example:

int func_t (int, int, float);

This defines func_t, a pointer-to-function

typedef int func_t (int, int, float);

This defines a new type, called func_t. Now

func_t myF;

is equivalent to

int myF(int int, float);

124


TypedefTypedef

• There are two valid reasons for encouraging the use of typedef:parametrizing a program with its types may turn

into semplifying the solution of portability problems: for instance,

typedef short integer;Moving the code to a machine where the role of short is played by int we only need to change one line of code:

typedef int integer;enhancing a program’s readability: a type called

LENGTH brings with its name also a hint at its usage and meaning within the program.

125



• Arrays of structures:• struct key {

char *keyword;int keycount;

} keytab[100];

• Ortypedef struct key { … } key_t;

and thenkey_t keytab[100];

126



• key_t keytab[100];• Initialisation:

key_t Keywords[] = {"break", 0,"case", 0,"char", 0, …"while", 0

};

• Exercise: Write a program that writes a static arrays of struct’s to be used as look-up table for another program.

127



• Structures can reference themselves:

struct tnode {char *word; int count;struct tnode *left;struct tnode *right;

};

• Another example:struct nlist {

char *name; char *def;struct nlist *next;

};

128



• Exercise: Write a set of functions dealing with linked listsStructure the set of functions as a serviceOpen a connection to the file service, then use

the service (read/write/delete…), then disconnect (close) the service

129


Bitfield structuresBitfield structures

• Flag registers:#define KEYWORD 01#define EXTERN 02#define STATIC 04#define AUTOMT 010…flags |= EXTERN | STATIC;…if ( flags & (EXTERN | STATIC) ) ...

• It is possible to store in a same variable, flags, a series of conditions that can be “turned on” through a bitwise OR (|) and can be tested via the bitwise AND (&).

• Clearly the definitions must be powers of 2. Octal implies unsigned.

130



• Structures can be used to specify ag registers via so-called “bitfields”:

struct {unsigned is_keyword : 1;unsigned is_extern : 1;unsigned is_static : 1;unsigned is_regist : 1;

} flags;

• Accessing a field: standard way (“.” operator). For instance: flags.is_extern.

131



• Bitfields can be used as lvalues and rvalues as any other integer variable.

• This means that bitwise OR’s and AND’s are not necessary:

flags.is extern = 0if (flags.is extern == 0) ...

132



• Bitfields can only be unsigned or int• Asking the address of a bitfield is illegal• Bitfields can also be “unnamed,” e.g., for padding• The standard doesn’t specify if MSB-first or LSB-

first is used.

133


UnionsUnions

• An union is a variable having many identifiers associated to it.

• These identifiers may be of different type and hence different sizeof’s. Each identifier refer to the same amount of memory, i.e., as many bytes as the “largest” type requires. For instance:

union point_x {char *pc;float *pf;double *pd;long regarded_as_an_int;

} object;

134


UnionsUnions

• Variable object has many “aliases”: it can be regarded as a pointer-to-char, if referred to as object.pc, or as pointer-to-double (object.pd), or as a long (object.regarded as an int).

• The amount of memory is always the same, the one that is enough in order to store the “largest” among a (char*), a (float*), a (double*), or a (long).

135


UnionsUnions

• Typical use of unions:

struct {int its_type;union {

int ival;float fval;char *pval;

} uval;} symbol_table[500];

• If we store in symbol table[i].its type a code that represents the type of object i, then we can set up, e.g., arrays of objects of different types, or linked lists of different objects, and so forth.

136


UnionsUnions

• A switch on its type is the typical way to execute diverse actions depending on the nature of the current object, as it’s done in the following example:

for (i=0;i<500;++i)switch(symbol_table[i].its_type) {case ’i’: printf("%d",

symbol_table[i].uval.ival);break;

…}

137


Standard librariesStandard libraries

• In C many functions are defined in the so-called “standard library”, libc.a. A set of headers refer to the functions and definitions in the standard library. The header file stdio.h contains the basic functions and definitions for I/O:

#include <stdio.h>

138


Standard streamsStandard streams• C and UNIX define three standard I/O streams:

stdin, i.e., the standard input stream, normally given by the characters typed at the keyboard. As in DOS, the redirection character < allows to specify an alternative standard input stream as follows:

prog < inputfilestdout, the standard output stream, normally

given by the characters typed onto our display. Character > redirects stdout on an other file, as in

prog > outputfile.stderr, the standard error stream, is again by

default our display. Is the stream where the programmer does (should) send the error messages. Can be redirected via 2 >, e.g.,

prog 2> /dev/null.

139


PipesPipes

• Besides the stream redirection facilities, UNIX provides the concept of pipe: if we type

prog1 | prog2

the stdout of prog1 becomes the stdin of prog2.

140


PipesPipes

TASK 1 TASK 2stdin stdoutstdin stdout

pipe in pi pe out

int pipeline[2];pipein = pipeline[STDIN], pipeout = pipeline[STDOUT];pipe(pipeline);

• pipe() creates an I/O mechanism called “pipe” and returns two file descriptors, fildes[0] and fildes[1]. The files associated with fildes[0] and fildes[1] are streams and are both opened for reading and writing.

• A read from pipeline[0] accesses the data written to pipeline[1] and a read from pipeline[1] accesses the data written to pipeline[0] (both on a FIFO basis.)

141


PipesPipes

• dup2() duplicates an open file descriptor

• dup2(int fildes, int fildes2)

• The dup2() function causes the file descriptor fildes2 to refer to the same file as fildes. The fildes argument is a file descriptor referring to an open file.

void main() { dup2(1,2);printf(“hello”);fprintf(stderr, “ world\n”);

}$ ./a.outhello world

142


PipesPipes


pipe in pipe out

dup2(pipeout, STDOUT); dup2(pipein, STDIN);

puts(”hello”); gets(msg); // msg is “hello”

h e l l o \0

• STDOUT == 1, STDIN ==0

• Task 1’s stdout is redirected to task 2’s stdin

143


PipesPipes

• Pipes are also available as FILE*: for instance,

FILE *popen(char *command, const char *type); int pclose (FILE *stream);

• the popen() function creates a pipe between the calling program and the command to be executed. command consists of a shell command line. type is an I/O mode, either r for reading or w for writing

• The value returned is a stream pointer such that one can write to the standard input of the command, if the I/O mode is w, by writing to the file stream; and one can read from the standard output of the command, if the I/O mode is r, by reading from the file stream.

144


PipesPipes

1. FILE *f = popen(”date”, ”r”);2. FILE *g=popen(”/bin/sort”, ”w”);

1. with the first one we can, e.g., read the output of command date.

2. The second one connects to service /bin/sort so to ask that external service (in this case, to sort a stream)

145


The UNIX manThe UNIX man

• The UNIX command “man” is an important tool• If you don’t remember how a certain functions is to

be invoked, or what is does, just type

man function

• Try for instance man printf man putc man strcpy man tolower

146


MakefilesMakefiles

• A makefile is a script that tells how to compile an application

• It specifies the dependencies among a number of resources (header and C files)

• An example follows:

all: myfile.exemyfile.exe: myfile.o myobj.o

cc –o myfile.exe myfile.o myobj.omyfile.o: myfile.c common.h

cc –c myfile.cmyobj.o:myobj.c common.h myobj.h

cc –c myobj.c

147


Performance issuesPerformance issues

• How much can the choice of a data structure influence a property such as locality in sequential data accesses? In order to evaluate this we run a simple experiment on a Win2000/Pentium3 PC

148


Performance issuesPerformance issues

• Experiment: the following structure:typedef struct { int a;

char b[STRIDE];} stru_t;and the following access scheme:for (i=0; i<itera-1; i++)

v[i].a = v[i+1].a + 1;are compared with the following two data structures: typedef int stru_ta;typedef char stru_tb[STRIDE];and access scheme: for (i=0; i<itera-1; i++)

a[i] = a[i+1] + 1;

149


STRIDE == 4 (y axis: us; x axis: (x+1)*100000)

150


STRIDE == 8 (y axis: usecs; x axis: (x+1)*100000)

151


STRIDE == 12

152


STRIDE == 16

153


The following pictures plot a vertical line that

represent the gain in performance vs. the best “-O3”

cases for different values of STRIDE:

Stride = 4

154


Stride = 16

155

Advanced C Language for Engineering ( V De Florio)Stride = 32

156


Stride = 64

157


Stride = 128

158


ReferencesReferences

• Kernighan & Ritchie,The C programming Language, Addison-Wesley

159


Optimisations in COptimisations in C

• Amdhal Law• Applications in C: gprof

160


ExerciseExercise

• Write a program that renames a set of files by substituting all the occurrences of a given string with another one in the filename

161


ExercisesExercises

• Write a program that displays a Julia set of a given function in a certain “window” of the Complex plane

• Write a program that zooms in the Julia set and produces an mpeg animationYou need to find some resources in the Internet

• Project work: write a language that controls the aboveFunctional input Input parametersZoom parametersMpeg production parametersEtc!

162


ExerciseExercise

• Problem: you want to organize a set of mp3 CDs• Input: mp3 CDs containing files such as

/Pink Floyd/Meddle/One of these days.mp3• Output: a web-based application that manages a DB

of mp3 CDsClassifying their contents

One of these days PART-OF Meddle PART-OF Pink Floyd…Building relationships among items of different CDsEasy addition of new CDsSearch of items


CWEB : a system for CWEB : a system for ““structured documentationstructured documentation””

Axiom: programmers who want to provide the best possible documentation for their program need twothings simultaneously:

• a language like TeX for formatting,

• a language like C or C++ for programming.


The ideaThe idea

A program can be thought of as a WEB that is made ofmany interconnected pieces. CWEB makes the user able:

• to explain the local structure of each part,• to explain how each entity relates to its neighbors,• to attach documentation / code to each part.


CWEBCWEB

In essence, CWEB is a tool by means of which one can use a style of programming that maximizes his/her ability to perceive the structure of a complex piece of software.

Involved actions and file types


As a Makefile:As a Makefile:

myprog : myprog.c cc -o myprog myprog.cmyprog.c : myprog.w ctangle myprog.w

myprog.ps : myprog.dvi dvips -o myprog.ps myprog.dvimyprog.dvi: myprog.tex tex myprogmyprog.tex: myprog.w cweave myprog.w


CWEB modulesCWEB modules

CWEB is made of two modules:

• cweave (w2tex translator), and

• ctangle (w2c or w2c++ translator.)


CTANGLECTANGLE

A WEB file is a collection of program/documentation piecesthat may appear in any order. This order reflects the idea theprogrammer has of his/her program:

ctangle takes a given “web” and moves the sections from their web structure into the order required by C.

<Header Files><Global Variables><Functions><The Main program><etc.>

<The Main...>= <variables local to main> <set up option selection> <process all the files> <print the grand totals if there were multiple files>

<etc.>= ...


CWEAVECWEAVE

cweave takes a given “web” and creates a structured TeX document in which:

• all sections are numbered,• every relationship between sections are highlighted,• all places where something is defined are highlighted,• the C program is “beautified.”


DrawbacksDrawbacks

• TeX is needed (a huge software system), including fonts and tools like DVI viewers and converters;

• some knowledge of TeX is needed!


How to define a SectionHow to define a Section

The basic part of a web document is the section.A section can be named or unnamed. Unnamedsections begin with “@*” (at-star) or “@ ” (at-space):

@* An example of {\tt CWEB}. Thisunnamed section describes...

@ Another unnamed section.


How to define a Section (2)How to define a Section (2)

A named section has a different syntax based onthe characters @, <, >, and =

@<Variables local to main@>

@<Variables local to main@>= int a, b; int (*fptr)(void);


Using CWEAVEUsing CWEAVE


Using CTANGLEUsing CTANGLE

177


Deepenings: the TIRAN frameworkDeepenings: the TIRAN framework

• With “TIRAN framework” we mean:a set of mechanisms fault masking, containment, error

detection, isolation and recovery a software library

a basic services librarya ‘backbone’, integrating and coordinating the

mechanisms in distributed environmentsa distributed application

some configuration tools, with which the user fulfils the requirements of her/his applicationa “recovery language”, namely a language for programming

error recovery strategies (ARIEL)a “configuration language”, with which the user defines,

e.g., some instances of the FT mechanisms

178


ArchitectureArchitecture

Run-time System(Drivers, Kernel, …)

Use

r ap

plic

atio

nA

RIE

L

DetectionMasking

BackboneManagement of recovery

ContainmentReconfiguration

High level mechanismsFa

ult

Inje

ctio

n

Monitoring

179


MechanismsMechanisms

• Watchdog timer• Voting system• Task Synchronizer• Distributed Memory• Memory Stabilizer

• User-defined mechanisms

• Integrated or stand-alone

180


Basic services libraryBasic services library

• “Basic services library” (BSL) intra-node, inter-node communication via group-based

communication primitives task managementaccess to local clock semaphoresnode shutdown, restart...

181


BackboneBackbone

• The backbone (DB, or BB...): a distributed application managing the following tasks:

management of a database of data describing the topology and the current state of the application, plus error notifications dispatched bythe mechanismsthe user applicationthe “basic service library”

filtering of error notification (via a-count)identification of the fault class (transient vs.

permanent/intermittent)this allows to set up fault-specific strategies

invocation of error recovery (executing the interpreter of the recovery language)

DB

a-count

RINT

182


Backbone Backbone ® typical scenario typical scenario

• A watchdog notifies its local BB agent via function RaiseEvent

RaiseEvent(TIRAN_WD_KICK, TIRAN_WD, TIRAN_ThisTaskUniqueId(), 0)

• The local component of the BB receives that notification and:stores it in its copy of the DB feeds an alpha-count filter forwards the notification to the remote agents (if it's an error notification) initiates recovery... (described

later)

183


BackboneBackbone

• Recovery may ask specific services to basic tools for isolation or recovery

• Prototypal implementation in C and the BSL for NT and C/EPX for Parsytec PowerXplorer

184


BackboneBackbone

• Backbone: distributed applicationmade of “agents”: (task D, task I, task R) " node i in { 0,..,n-1 } : D[i], I[i], R[i] run on node i.

D : database / a-count,I: “I’m Alive” task (watchdog)R: recovery interpreter

D-I-R => “DIR-net”• Agents: one manager and a set of assistants in

hierarchical order• Based on an algorithm for tolerating node and task

crashes

185


BackboneBackbone

• BB is compliant to the timed-asynchronous distributed system model (Cristian & Fetzer)

• In particular, BB tolerates system partitioning during periods of instability

• Partitions are merged back at the end of a period of instability

• As a special case, when a crashed node is rebooted, it is re-entered in the BB

186


BackboneBackbone

• These features are provided by a distributed algorithm, called “algorithm of mutual suspicion” because agents mutually question their state

• This algorithm has been designed by means of the TOM class

187


Backbone Backbone ® AMS AMS

• Twofold error detection strategy:LOCAL PART: " i : task I[i] watches task D[i]

D[i] periodically clears an “I’m Alive” flag, I[i] periodically checks and sets it.

If I[i] finds a set flag, it broadcasts a TEIF (“this entity is faulty”)

REMOTE PART: manager expects TAIA messages (“this assistant is alive”)assistants expect MIA messages (“manager is alive”)WHEN A MESSAGE DOES NOT COME IN WITHIN A

CERTAIN DEADLINE, A SUSPICION PERIOD IS ENTERED

• In absence of faults:LOCAL PART: flag is set and clearedREMOTE PART: MIA’s and TAIA’s are sent around resp.

by the manager and the assistants

188


Backbone Backbone ® ““Mutual SuspicionMutual Suspicion””

• When a message does not come in within a certain deadline, a suspicion period is entered

• A suspicion period ends either returning to the normal situation or by applying a graceful degradation or a recovery strategy

189


Backbone Backbone ® time-out management time-out management

• Based on application-level time-outsObjects that schedule an event (called alarm) a given

amount of seconds in the futureCyclic or non-cyclic In the BB, alarms send a message to task D with a

notification “a time-out has occurred”So it turns time-related transitions into message arrivalsA special task, task A, runs on each node as time-out

managerTask A just need to monitor the top entry in the time-out list

Class of C functions for NT and EPX

190


Backbone Backbone ® time-out management time-out management

191



• Four classes of time-outs IA_SET_TIMEOUT, IA_CLEAR_TIMEOUT

“It’s time to set/clear the I’m Alive flag!”Alarm: send D[this node] message IA_SET or IA_CLEAR

MIA_SEND_TIMEOUT, MIA_RECV_TIMEOUT“It’s time to send a MIA message / to check whether a MIA

message has come in”Alarm: send D[this node] message MIA_SEND or

MIA_RECVTAIA_SEND_TIMEOUT, TAIA_RECV_TIMEOUT

“Time to send a TAIA / to check whether a TAIA has come in”Alarm: send D[this node] message TAIA_SEND or

TAIA_RECVTEIF_TIMEOUT

“Time to check whether a TEIF message has come in”Alarm: send D[this node] message TEIF_CHECK

192



MIA: Manager Is Alive: sent by the manager to all its assistants

TAIA: This Assistant Is Alive: sent by each assistant to the manager

TEIF: This Entity Is Faulty: sent by a task I to signal that its task D is faulty

MIA_SEND, TAIA_RECV: sent from task A[m] to task D[m] (the manager)

TAIA_SEND, MIA_RECV: sent from A[i] to D[i] (an assistant)

TEIF_CHECK: sent from A[i] to D[i] (either the manager or an assistant)

IA_SET (resp. IA_CLEAR): sent from A[i] to D[i] (resp. from A[i] to I[i])

193


Backbone Backbone ® AMS AMS ® coordinator coordinator

194


Backbone Backbone ® AMS AMS ® assistant assistant

195



• When a crash fault affects D[i]: I[i] detects this as a set flag Þ broadcasts a TEIFThe others detect this as a lacking TAIA or MIA and a

coming TEIF• When a crash fault affects node i:

The others detect this as a lacking TAIA or MIA and a lacking TEIF

196


Pseudo-code of the coordinatorPseudo-code of the coordinator

coordinator(){ /* the alarm of these alarms simply sends a message of id <alarm-type>, subid = <subid> to the coordinator */ insert alarms (IA-flag-alarm, MIA_SEND-alarms, TAIA_RECV-alarms)

clear all entries of sus[] /* suspicion periods are all off */ set IA-flag /* ``I'm alive'' flag */ activate IAT /* I'm Alive Task */

loop (wait for incoming messages) { switch (message.type) { /* time to set the IA-flag! */ case IA-flag-alarm: set IA-flag, break;

197



/* time to send a MIA to an assistant */ case MIA_SEND-alarm: send MIA to assistant <subid> break;

/* a message which modifies the db */ case DB: if (local) { renew MIA_SEND alarm on all <subid>s broadcast modifications to db break; } else /* if it's a remote message, it's also a piggybacked TAIA */ { update your copy of the db /* don't break */ }

198


Pseudo-code of the coordinatorPseudo-code of the coordinator /* if a TAIA comes in or a remote DB message comes in... */ case TAIA: if ( ! ispresent(TAIA_RECV-alarm, <subid> ) { insert TAIA_RECV-alarm, <subid> broadcast NIUA /* node is up again! */ } renew TAIA_RECV-alarm, <subid> /* if you get a TAIA while expecting a TEIF, then no problem, simply get out of the suspicion period */ if (sus[ <subid> ] == TRUE) sus[ <subid> ] = FALSE; break; /* no heartbeat from a remote component... enter a suspicion period */ case TAIA_RECV-alarm: sus[ <subid> ] = TRUE insert alarm (TEIF_RECV-alarm, NON_CYCLIC, IMALIVE_RECV_TIMEOUT, <subid>) delete alarm (TAIA_RECV-alarm, <subid>); break;

199



/* a TEIF message has been sent from a IAT: as its last action, the IAT went to sleep */ case TEIF: if (sus[ <subid> ] == TRUE) { delete alarm (TEIF_RECV-alarm, <subid>) sus[ <subid> ] = FALSE /* agent recovery will spawn a clone of the assistant <subid>. If no assistant clones are available, the entire node will be rebooted. */ Agent-Recovery( <subid> ) } else { if (local) { set IA-flag enable IAT } else send ENIA (i.e., ``ENable IAt'') to <subid> } break;

200



case TEIF_RECV-alarm: if (sus[ <subid> ] == TRUE) { delete alarm (TAIA_RECV-alarm, <subid>) sus[ <subid> ] = FALSE /* the entire node will be rebooted. */ Node-Recovery( <subid> ) } break;

case ENIA: set IA-flag activate IAT /* if an IAT gets an “activate” message while it's active, the message is ignored */ break; default: deal with other messages } /* end switch (message.type) */ set IA-flag renew IA-flag-alarm } /* end loop */} /* end coordinator */

201


Rendering via a hypermedia monitorRendering via a hypermedia monitor

• Non-parsing header CGI script (written in C)

• Remotely controlled Netscape browser

202


Backbone Backbone ® Monitor Monitor

203


SimulationsSimulations

204


Simulations (2)Simulations (2)

205



206



207


Deepenings: RDeepenings: ReeL: adoption of a L: adoption of a configuration languageconfiguration language

• A configuration language is a linguistic framework to

define system, middleware, and application parametersdefine tasks, groups, nodesconfigure the basic toolsconfiguration of parameters (e.g., for alpha-count)

208


RReeL L ® Config Config ® ExampleExample

# Defines are importable from C header# files via the INCLUDE statement INCLUDE "phases.h" INCLUDE ”../backbone.h"# Definitions NPROCS = 4 Define 0 = MANAGER Define 1-3 = ASSISTANTS MIA_SEND_TIMEOUT = 800000 # Manager Is # Alive -- manager side TAIA_RECV_TIMEOUT = 1500000 # This # Agent Is Alive timeout--manager side

209


ReL ReL ® Config Config ® ExampleExample

TASK 1 = "Backbone0" IS NODE 1, TASKID {BACKBONE_TASKID}




LOGICAL 1 = "LBack0" IS TASK 1 ENDLOGICAL 2 = "LBack1" IS TASK 2 ENDLOGICAL {ALL} ="all" IS TASK 1,2,3,4 END

210



ALPHA-COUNT 1 IS threshold = 3.0, factor = 0.4 END ALPHA-COUNT

# Tool ConfigurationWATCHDOG TASK 13 WATCHES TASK 14 HEARTBEATS EVERY 100 MS ON ERROR REBOOT # or WARN TASK x...END WATCHDOG

output: a configured instance of a watchdog in file TIRAN_Task10.c

211


• A linguistic framework to

configure replicated tasks, retry blocks, multiple-version software fault tolerance means (NVP, CRB, …)

using multicasts to replicate stdinusing pipelining and stream redirection for transparent

forward of the output valuesusing voting for de-multiplexing output values

ReL ReL ® Config Config

212



N-VERSION TASK 15 VERSION 1 IS TASK 16 TIMEOUT 100 ms VERSION 2 IS TASK 17 TIMEOUT 100 ms VERSION 3 IS TASK 18 TIMEOUT 100 ms VOTING ALGORITHM IS MAJORITY METRIC "task20_cmp" ON SUCCESS TASK 19 ON ERROR TASK 20END N-VERSION

• Output: source files TIRAN_Task1[678].c

213



#include "TIRAN.h"/* Task 18 of NVersion Task 15 Version 3 / 3 */

int TIRAN_task_18(void) { TIRAN_Voting_t *dv; size_t size; double task20_cmp(const void*, const void*);

dv = TIRAN_VotingOpen(task20_cmp); if (dv == NULL) {

TIRAN_exit(TIRAN_ERROR_VOTING_CANTOPEN); } ...

214


ReL ReL ® Config Config ® Support towards NVPSupport towards NVP

215


ReL ReL ® Config Config ® Pipelines+redirectionPipelines+redirection


pipe in pi pe out

int pipeline[2];pipein = pipeline[STDIN], pipeout = pipeline[STDOUT];pipe(pipeline);


pipe in pipe out

dup2(pipeout, STDOUT); dup2(pipein, STDIN);

puts(”hello”); gets(msg); // msg is “hello”

h e l l o \0

216



bash-2.02$ art -s -i .ariel Ariel translator, v2.0f 03-Mar-2000, (c)

K.U.Leuven 1998, 1999, 2000.Parsing file .ariel......doneOutput written in file .rcode.Watchdogs configured.N-version tasks configured.Logicals written in file LogicalTable.csv.Tasks written in file TaskTable.csv.Preloaded r-codes written in file trl.h.Time-outs written in file timeouts.h.Identifiers written in file identifiers.h.Alpha-count parameters written in file

../alphacount.h.

217



• Translator written in C and Lex & YACC

• Still work to be doneC output filesProper syntaxTesting…

Þ DepAuDE

218


ReL ReL ® Recovery Recovery ® ExampleExample

INCLUDE "my_definitions.h" TASK {VOTER1} IS NODE {NODE1}, TASKID {VOTER1} TASK {VOTER2} IS NODE {NODE2}, TASKID {VOTER2} TASK {VOTER3} IS NODE {NODE3}, TASKID {VOTER3} TASK {SPARE} IS NODE {NODE4}, TASKID {SPARE}

IF [ PHASE (T{VOTER1}) == {HAS_FAILED} ] THEN STOP T{VOTER1}

SEND {WAKEUP} T{SPARE} SEND {VOTER1} T{SPARE}

SEND {SPARE} T{VOTER2} SEND {SPARE} T{VOTER3} FI

219



rcode_t rcodes[] = { /* HEADER FILE PRODUCED BY THE ARIEL TRANSLATOR */

• /*line#*/ /* opcode */ /* operand 1 */ /* operand 2 */

• /*0*/ { R_SET_ROLE, 1, 2000 },• /*1*/ { R_SET_ROLE, 2, 2000 },• /*2*/ { R_SET_ROLE, 3, 2000 },• /*3*/ { R_SET_ROLE, 1, 3000 },• /*4*/ { R_SET_DEF_ACT, 666, -1 },• /*5*/ { R_INC_NEST, -1, -1 },• /*6*/ { R_STRPHASE, 0, -1 },• /*7*/ { R_COMPARE, 1, 9999 },• /*8*/ { R_FALSE, 10, -1 },• /*9*/ { R_KILL, 18, 0 },• /*10*/ { R_PUSH, 18, -1 },• /*11*/ { R_SEND, 18, 3 },• /*12*/ { R_PUSH, 0, -1 },• /*13*/ { R_SEND, 18, 3 },• /*14*/ { R_PUSH, 3, -1 },• /*15*/ { R_SEND, 18, 1 },• /*16*/ { R_PUSH, 3, -1 },• /*17*/ { R_SEND, 18, 2 },• /*18*/ { R_DEC_NEST, -1, -1 },

220



Art translated Ariel strategy file: .... provainto rcode object file : ............... .rcode

line rcode opn1 opn2-----------------------------------------------00000 SET_ROLE 1 Assistant00001 SET_ROLE 2 Assistant00002 SET_ROLE 3 Assistant00003 SET_ROLE 1 Manager00004 SET_DEFAULT_ACTION 66600005 IF00006 STORE_PHASE... Thread 000007 ...COMPARE == 999900008 FALSE 1000009 KILL Thread 000010 PUSH... 1800011 ...SEND Thread 300012 PUSH... 000013 ...SEND Thread 300014 PUSH... 300015 ...SEND Thread 100016 PUSH... 300017 ...SEND Thread 200018 FI

223


ReL ReL ® Recovery Recovery ® Run-timeRun-time

• The backbone awakes a recovery interpreter (RINT)• Virtual machine executing r-codes• Stack machine• Currently, r-codes are read either from a file or from

memory (static version, rcodes[] array)

while ( ! End of r-codes ){

get next r-code;goto? Skip r-codes;does it deal with backbone?

Push( query (backbone) );execute generic r-code;

}

224


ExampleExample

• NVP + spares + fault-specific recovery

• IF [ FAULTY TASK10 ]THEN IF [ TRANSIENT TASK 10 ]

THEN RESTART TASK 10 … ELSE … reconfigurationENDIF

ENDIF

225


ExampleExample

226


ExampleExample

227


ExampleExample

228


Results Results ® Time to detect a crash failure Time to detect a crash failure

229


Results Results ® recovery times recovery times

230


Deepening: the redundant watchdogDeepening: the redundant watchdog

• Past: automation solutions for electricity were based onHardware FT tools and approachesAd-hoc developed platforms and architectures

• Today: need for higher flexibility flexible FT solutionsportablebased on standard architectures and heterogeneous

platforms

• TIRAN addressed this problem

231


IntroductionIntroduction

• Example domain: electrical substation automations• Auto-exclusion functionality

requirement: in case of a failure, one must guarantee that the equipment be left in an “acceptable” state, e.g., forcing a “safe” output value, alerting the operators and so forth (for instance: traffic light in safe mode)

The auto-exclusion functionality must guarantee high availability and high integrity

• Typical solution: watchdog timer

232


WatchdogsWatchdogs

• Aim: watching a user task

• The user task is requested to send periodically a “sign of life” (a heartbeat msg) to the watchdog

• The watchdog resets a counter each time a heartbeat is received

• The counter is incremented linearly with time

• When the counter reaches a threshold, an alarm is issued

watchdogUser taskHB

clearHB

clearAlarm!

233


Redundant watchdogRedundant watchdog

• Traditional solution: hardware watchdogdetects crash/performance failures typical availability and integrity of hardware systems low degree of flexibilityoften a non-standard, non-portable solution

• TIRAN solution: software, though redundant, watchdog replication of tasks on several nodesuser-defined replication levelhigher flexibilityexploiting the inherent redundancy of standard distributed

architectures (e.g., nodes in a cluster of workstations)higher portability

234



• ReL strategy and the ariel language to specify the involved parameters: frequency of sending heartbeat messagesparameters of the a-count filternumber of nodes to use task-to-node mapping recovery strategy

235


The TIRAN redundant watchdogThe TIRAN redundant watchdog

• TIRAN tools that we have used:TIRAN framework

Library of mechanisms (WD)The backboneariel and the ReL methodologyBasic services library (communication,

management of tasks, synchronization…)In particular:

configuration of a mechanism (WD)composition of a sophisticated mechanisms

through the use of a simpler one

236


The RThe ReeL methodology in TIRANL methodology in TIRAN

Recoverypseudo-code

Applicationdescription

Systemdescription

D-tooldescription

Configuration code

Recoverydescription

Recovery code

Configuredinstances

Applicationsource code

Application executable Backbone/RINT

ariel ariel

art art

cc cc

237


Problem: how to improve the availability Problem: how to improve the availability and integrity of the TIRAN watchdogand integrity of the TIRAN watchdog

Watchdog

User task

Controlleralarm!

Watchdog

User task

Controller

no alarm

• Problem: what if a WD “forgets” to fire?• Solution: redundant WD with voting in ariel

238



• Problem: what if a WD fires without reason?

Watchdog

User task

Controlleralarm!

• Solution: redundant WD and voting in ariel

Watchdog

ControllerWatchdog

Watchdog

239


Demo: redundant watchdogDemo: redundant watchdog

• Aim of the demo: showing how to use the TIRAN framework the recovery language approach

in order to realize, at low costs, a prototype of a redundant watchdog characterized byÞ either a higher availabilityÞ or a higher integrity

240


RWD: Demo: step 1RWD: Demo: step 1

• ConfigurationSystem composed of 3 “nodes” (node 1, 2 and 3)A client process connects to “watchdog W”W is indeed a “logical”: W = { W1, W2, W3 }The cyclic heartbeat is sent to W in multicast modeFor each task t in W:

if the heartbeat does not show up in time, t firesRaiseEvent(… FIRED!…)The source codes of W1, W2 e W3 are created by the ariel

translator (TIRAN_task_16.c, TIRAN_task17.c, …18.c)These source codes are automatically imported in the C++

user workspace

241


242


Demo: step 2Demo: step 2

• Compiling• W1, W2 and W3 are compiled • Three executables are produced: watchdog16.exe,

watchdog17.exe and watchdog18.exe

243


244


Demo: step 3Demo: step 3

• ExecutionThe TIRAN backbone, the user client and the three

watchdogs are launchedThe three watchdogs wait for an activation message, after

which they expect heartbeatsAs soon as a heartbeat is not received =>

RaiseEvent(… FIRED…) that is, the alarm condition is triggered

245


The RThe ReeL methodology in TIRANL methodology in TIRAN

Recoverypseudo-code

Applicationdescription

Systemdescription

D-tooldescription

Configuration code

Recoverydescription

Recovery code

Configuredinstances

Applicationsource code

Application executable Backbone/RINT

ariel ariel

art art

cc cc

246


Scheme of execution of the recovery Scheme of execution of the recovery actionsactions

DBUser application Recoveryapplication

ErrorDetection

StoreRecovery starts

Query

Skip/executeactions

Result

Recovery endsOK

247


Demo: recovery strategiesDemo: recovery strategies

• In our demo, error recovery is the execution of a system-level alarm

• This alarm is triggered when a certain condition occurs:

• Possible strategies:OR strategy:IF [ FIRED W1 OR FIRED W2 OR FIRED W3 ]

The alarm triggers when any of the watchdogs firesThis compensates up to two watchdog crashesIt lowers the probability that a crashed task goes undetected

because its WD had failed (enhances error detection coverage)

in other words: increases INTEGRITY

248


Demo: recovery strategiesDemo: recovery strategies

• AND strategyIF [ FIRED W1 AND FIRED W2 AND FIRED W3 ]The system alarm triggers when all the WD’s agreeTolerates up to two faulty WD’sLowers the probability that the system alarm is raised

without a very good reason (client still OK)Useful to reduce the probability that the system be

erroneously stopped due to faulty watchdogÞ Better availability

249


W1

W2

W3

BB

client

activationwatchdogFIRE!

Recovery interpretation

System alarm is raised

250


Redundant watchdog: lessons learned and Redundant watchdog: lessons learned and conclusionsconclusions

• AND Þ higher availability• OR Þ higher integrity• 2-out-of-3 : trade-off

• The development is simplified• Aspects related to error recovery are specified

outside the user application• Aspects related to the configuration of the error

detection provisions (in this case, WDs) are specified outside the user application