Open Source Compiler COSCUP'2014 Date : July 19th, 2014 Kito Cheng kito.cheng@gmail.com

2 Compiler Team

3 Open Source Compiler

4 yum update -ysudo apt-get upgrade Compiler?

5

6

7

8 Compiler

9 Compiler Debug!

10 GNU ld.bfd / ld.gold gdb as/objdump libstdc++ libgcc

11 GNU LLVM ld.bfd / ld.gold lld / mclinker gdb lldb as/objdump MC layer in LLVM libstdc++ libc++ libgcc libcompiler-rt

12 binutils vs MC Layer binutils Library , . LLVM MC Layer , assembler disassembler Library . Toolchain

13 libstdc++ vs libc++ Linux C++ libstdc++

14 libstdc++ vs libc++ Linux C++ libstdc++ std::string...

15 libstdc++ vs libc++ Linux C++ libstdc++ std::string... libc++ !

16 libstdc++ vs libc++ Linux C++ libstdc++ std::string... libc++ ! STLPort :

17 libstdc++ vs libc++ Linux C++ libstdc++ std::string... libc++ ! STLPort : STLPort C++ Runtime Library

18 VM/JIT New Programming Language /

19 VM/JIT

20 VM/JIT

21 VM/JIT Kaffe VM : JIT Interpreter

22 VM/JIT Kaffe VM : JIT Interpreter Just in time

23 VM/JIT Kaffe VM : JIT Interpreter Just in time Just too late!

24 Pyston FTLWebKits LLVM based JIT

25 New Programming Language

26 New Programming Language Native Execution ...

27 New Programming Language Native Execution ... code gen

28 New Programming Language Native Execution ... code gen C code Compiler

29 New Programming Language Native Execution ... code gen C code Compiler Compiler

30 New Programming Language GCC ...

31 New Programming Language GCC ... , GCC IR - GIMPLE XD : GPLv3

32 New Programming Language GCC ... , GCC IR - GIMPLE XD : GPLv3 LLVM !

33 Rust

34 / youcompleteme clang static analyzer

35 in Compiler Address-sanitizer Undefined-sanitizer Thread-sanitizer

36 Address-sanitizer :) Valgrind !

37 int main(int argc, char **argv) { int stack_array[100]; stack_array[1] = 0; return stack_array[argc + 100]; // BOOM } ================================================================= ==28706==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fff61e1f344 at pc 0x4a5dfb bp 0x7fff61e1f170 sp 0 READ of size 4 at 0x7fff61e1f344 thread T0 #0 0x4a5dfa in main /home/kito/test.cpp:4 #1 0x7ff11a8a1d64 in __libc_start_main (/lib64/libc.so.6+0x21d64) #2 0x404c98 (/home/kito/a.out+0x404c98) Address 0x7fff61e1f344 is located in stack of thread T0 at offset 436 in frame #0 0x4a5d29 in main /home/kito/test.cpp:1 This frame has 1 object(s): [32, 432) 'stack_array' 0x10006c3bbe60: 00 00 00 00 00 00 00 00[f4]f4 f3 f3 f3 f3 00 00 ... 0x10006c3bbeb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 ...

38 ==12254==ERROR: AddressSanitizer: heap-use-after-free on address 0x60200000eff0 at pc 0x4a5db4 bp 0x7fff3ff57520 sp 0x7fff3ff57518 READ of size 4 at 0x60200000eff0 thread T0 #0 0x4a5db3 in main /home/kito/coscup2014/use-after-free.c:6 #1 0x3c52221d64 in __libc_start_main (/lib64/libc.so.6+0x3c52221d64) #2 0x404c98 (/home/kito/coscup2014/a.out+0x404c98) 0x60200000eff0 is located 0 bytes inside of 4-byte region [0x60200000eff0,0x60200000eff4) freed by thread T0 here: #0 0x476c79 in __interceptor_free /home/kito/gcc/gcc-src/libsanitizer/asan/asan_malloc_linux.cc:63 #1 0x4a5d7c in main /home/kito/coscup2014/use-after-free.c:5 #2 0x3c52221d64 in __libc_start_main (/lib64/libc.so.6+0x3c52221d64) previously allocated by thread T0 here: #0 0x476f19 in __interceptor_malloc /home/kito/gcc/gcc-src/libsanitizer/asan/asan_malloc_linux.cc:73 #1 0x4a5d2b in main /home/kito/coscup2014/use-after-free.c:3 #2 0x3c52221d64 in __libc_start_main (/lib64/libc.so.6+0x3c52221d64) #include int main() { int *a = malloc(sizeof(int)); *a = 100; free(a); return *a; }

39 free/delete/delete[] int main() { int *arr = new int[10]; delete arr; return 0; } ================================================================= ==12421==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new [] vs operator delete) on 0x60400000dfd0 #0 0x478219 in operator delete(void*) /home/kito/gcc/gcc-src/libsanitizer/asan/asan_new_delete.cc:85 #1 0x4a5efb in main /home/kito/coscup2014/mismatch-delete.cpp:3 #2 0x3c52221d64 in __libc_start_main (/lib64/libc.so.6+0x3c52221d64) #3 0x404e58 (/home/kito/coscup2014/a.out+0x404e58) 0x60400000dfd0 is located 0 bytes inside of 40-byte region [0x60400000dfd0,0x60400000dff8) allocated by thread T0 here: #0 0x477e29 in operator new[](unsigned long) /home/kito/gcc/gcc-src/libsanitizer/asan/asan_new_delete.cc:55 #1 0x4a5eeb in main /home/kito/coscup2014/mismatch-delete.cpp:2 #2 0x3c52221d64 in __libc_start_main (/lib64/libc.so.6+0x3c52221d64)

40 Address-sanitizer Valgrind Address Sanitizer Heap Y Y Stack N Y N Y (free/delete) Y Y ( : ) N Y( ) Y N free/delete/delete[] Y Y 10x-30x 1.5x-3x

41 Undefined-Sanitizer Undefined behavior

42 Undefined-Sanitizer Undefined behavior k standard undefined behavior...

43 Undefined-Sanitizer Undefined behavior k standard undefined behavior... c99 UB 1x

44 Undefined-Sanitizer Undefined behavior k standard undefined behavior... c99 UB 1x Undefined behavior !!!!!

45 Undefined-Sanitizer Undefined behavior k standard undefined behavior... c99 UB 1x Undefined behavior !!!!! UB...XD

46 Divde by 0 int main(int argc, const char *argv[]){ return argc/0; } div0.cpp:2:14: runtime error: division by zero Floating point exception

47 Deference Null pointer int main(int argc, const char *argv[]){ int *a = nullptr; return *a; } derefnull.cpp:3:11: runtime error: load of null pointer of type 'int' Segmentation fault

48 Shift int main(int argc, const char *argv[]){ return argc >> 32; } shift.cpp:2:15: runtime error: shift exponent 32 is too large for 32-bit type 'int'

49 Signed Integer Overflow #include int main(int argc, const char *argv[]){ int a = INT_MAX; return a + argc; } overflow.cpp:4:14: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'

50 Thread-Sanitizer Race Condition ! Race Condition , Thread-Sanitizer .

51 Race Condition #include #include int Global; void *Thread1(void *x) { Global++; return NULL; } void *Thread2(void *x) { Global--; return NULL; } int main() { pthread_t t[2]; pthread_create(&t[0], NULL, Thread1, NULL); pthread_create(&t[1], NULL, Thread2, NULL); pthread_join(t[0], NULL); pthread_join(t[1], NULL); }

52 Race Condition #include #include int Global; void *Thread1(void *x) { Global++; return NULL; } void *Thread2(void *x) { Global--; return NULL; } int main() { pthread_t t[2]; pthread_create(&t[0], NULL, Thread1, NULL); pthread_create(&t[1], NULL, Thread2, NULL); pthread_join(t[0], NULL); pthread_join(t[1], NULL); } ================== WARNING: ThreadSanitizer: data race (pid=21757) Write of size 4 at 0x7ffa3e002ef4 by thread T2: #0 Thread2 /home/kito/coscup2014/race.c:12 (race+0x0000000c1a75) Previous write of size 4 at 0x7ffa3e002ef4 by thread T1: #0 Thread1 /home/kito/coscup2014/race.c:7 (race+0x0000000c1a05) Location is global 'Global' of size 4 at 0x7ffa3e002ef4 (race+0x000000e03ef4) Thread T2 (tid=21760, running) created by main thread at: #0 pthread_create /home/kito/llvm/src/projects/compiler- rt/lib/tsan/rtl/tsan_interceptors.cc:842 (race+0x00000005e1f1) #1 main /home/kito/coscup2014/race.c:19 (race+0x0000000c1b03) Thread T1 (tid=21759, finished) created by main thread at: #0 pthread_create /home/kito/llvm/src/projects/compiler- rt/lib/tsan/rtl/tsan_interceptors.cc:842 (race+0x00000005e1f1) #1 main /home/kito/coscup2014/race.c:18 (race+0x0000000c1ad9) SUMMARY: ThreadSanitizer: data race /home/kito/coscup2014/race.c:12 Thread2 ================== ThreadSanitizer: reported 1 warnings

53 Race Condition #include #include int Global; pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; void *Thread1(void *x) { pthread_mutex_lock(&lock); Global++; pthread_mutex_unlock(&lock); return NULL; } void *Thread2(void *x) { Global--; return NULL; } int main() { pthread_t t[2]; pthread_create(&t[0], NULL, Thread1, NULL); pthread_create(&t[1], NULL, Thread2, NULL); pthread_join(t[0], NULL); pthread_join(t[1], NULL); }

54 Race Condition #include #include int Global; pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; void *Thread1(void *x) { pthread_mutex_lock(&lock); Global++; pthread_mutex_unlock(&lock); return NULL; } void *Thread2(void *x) { Global--; return NULL; } int main() { pthread_t t[2]; pthread_create(&t[0], NULL, Thread1, NULL); pthread_create(&t[1], NULL, Thread2, NULL); pthread_join(t[0], NULL); pthread_join(t[1], NULL); } ================== WARNING: ThreadSanitizer: data race (pid=21765) Write of size 4 at 0x7feaa10dcf20 by thread T2: #0 Thread2 /home/kito/coscup2014/race-2.c:15 (race- 2+0x0000000c1ad5) Previous write of size 4 at 0x7feaa10dcf20 by thread T1 (mutexes: write M0): #0 Thread1 /home/kito/coscup2014/race-2.c:9 (race- 2+0x0000000c1a57) Location is global 'Global' of size 4 at 0x7feaa10dcf20 (race- 2+0x000000e03f20) Mutex M0 (0x7feaa10dcef8) created at: #0 pthread_mutex_lock /home/kito/llvm/src/projects/compiler- rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:2956 (race- 2+0x00000007f260) #1 Thread1 /home/kito/coscup2014/race-2.c:8 (race- 2+0x0000000c1a37) Thread T2 (tid=21768, running) created by main thread at: #0 pthread_create /home/kito/llvm/src/projects/compiler- rt/lib/tsan/rtl/tsan_interceptors.cc:842 (race-2+0x00000005e231) #1 main /home/kito/coscup2014/race-2.c:22 (race-2+0x0000000c1b63) Thread T1 (tid=21767, finished) created by main thread at: #0 pthread_create /home/kito/llvm/src/projects/compiler- rt/lib/tsan/rtl/tsan_interceptors.cc:842 (race-2+0x00000005e231)

55 GCC !

56 class T { public: int a; } #include

57 class T { public: int a; } #include In file included from /home/kito/gcc-workspace/native-4.4/lib/gcc/x86_64- unknown-linux-gnu/4.4.7/../../../../include/c++/4.4.7/cstddef:44, from /home/kito/gcc-workspace/native-4.4/lib/gcc/x86_64- unknown-linux-gnu/4.4.7/../../../../include/c++/4.4.7/bits/stl_algobase.h:61, from /home/kito/gcc-workspace/native-4.4/lib/gcc/x86_64- unknown-linux-gnu/4.4.7/../../../../include/c++/4.4.7/vector:61, from test.cpp:5: /home/kito/gcc-workspace/native-4.4/lib/gcc/x86_64-unknown-linux- gnu/4.4.7/include/stddef.h:149: error: two or more data types in declaration of ptrdiff_t ... /home/kito/gcc-workspace/native-4.4/lib/gcc/x86_64-unknown-linux- gnu/4.4.7/../../../../include/c++/4.4.7/bits/vector.tcc:629: error: there are no arguments to difference_type that depend on a template parameter, so a declaration of difference_type must be available gcc 4.4 132 XD

58 class T { public: int a; } #include gcc 4.9 test.cpp:4:1: error: expected ; after class definition } ^ ( clang ?

59 : http://web.archive.org/web/20120622065456/http://people .redhat.com/bkoz/diagnostics/diagnostics.html https://gcc.gnu.org/wiki/ClangDiagnosticsComparison http://tinyurl.com/cxxdiagcmp http://tinyurl.com/clangcmp

60 * Sanitizer Clang/LLVM GCC 4.8 !

61 LTO ! Firefox Linux Kernel

62 libgccjit.so gcc JIT engine !!

63 libgccjit.so gcc JIT engine !! :

64 libgccjit.so gcc JIT engine !! : gcc + dlopen

65 C++

66 LLVM !

67 GCC

68 Named register variables register int *foo asm ("a5");

69 Named register variables , Inline asm register int *foo asm ("a5");

70 Named register variables , Inline asm Linux Kernel register int *foo asm ("a5");

71 Named register variables , Inline asm Linux Kernel Trunk, 3.5 register int *foo asm ("a5");

72 OpenMP Support int main(int argc, char *argv[]) { const int N = 100000; int i, a[N]; #pragma omp parallel for for (i = 0; i < N; i++) a[i] = 2 * i; return 0; }

73 1.x 2.x 3.x

74 1.x 2.x 3.x bug fix back port

75 1.x 2.x 3.x bug fix back port bug? Fix in ToT!

76 1.x 2.x 3.x bug fix back port bug? Fix in ToT! 3.4 3.4.1, 3.4.2!

77 gcc ! http://vmakarov.fedorapeople.org/spec/2014/llvmgcc64.html

78

79 -O -I -l -D -u -fPIC -Wl, -Wa, ... Open Source Project gcc

80 Clang (~3.4.x) Warning, return code

81 Clang (~3.4.x) Warning, return code Clang (trunk) Error abort!

82 Warning -Wall -Werror -Wextra clang gcc ...

83 Code !? Code gcc !

84 Code !? Code gcc ! gcc/g++ gnu89/ gnu++98 c89/c++98 GNU extension

85 Code !? Code gcc ! gcc/g++ gnu89/ gnu++98 c89/c++98 GNU extension clang c99! GNU extension

86 GCC and LLVM collaboration

87 GCC and LLVM collaboration GNU Tools Cauldron 2014 July 18-20, 2014 at Cambridge

88 GCC and LLVM collaboration With LLVM mature enough to feature as the default toolchain in some Unix distributions, and with the inherent (and profitable) share of solutions, ideas and code between the two, we need to start talking at a more profound level. There will always be problems that can't be included in any standard (language, extension, or machine-specific) and are intrinsic to the compilation infrastructure. For those, and other common problems, we need common solutions to at least both LLVM and GCC, but ideally any open source (and even closed source) toolchain. In this BoF session, we shall discuss to what extent this collaboration can take us, how we should start and what are the next steps to make this happen.

89

90

91 Open Source++

92 Toolchain ~ Open Source++

93