My First 100 days with a MySQL DBMS

Presented by : Gustavo René Antúnez DBA Team Lead April, 2015

2

• 15  Years  of  Data  infrastructure  management  consul4ng  

• 200+  Top  brands  • 6000+  databases  under  

management  • Over  200  DBA’s,  in  26  countries  • Top  5%  of  DBA  work  force,  9  

Oracle  ACE’s,  2  MicrosoJ  MVP’s  

• Oracle,  MicrosoJ,  MySQL  partners,  Netezza,  Hadoop  and  MongoDB  plus  UNIX  Sysadmin  and  Oracle  apps

About Pythian

• Flexible  Environment  – Work  remotely  from  anywhere  in  the  world.Flexible  

hours  • Outstanding  People:  

– Work  with  the  industry  top  minds-­‐in  fact,  the  top  5%.  • Career  growth  and  development:  

– Enhance  exis4ng  skills  or  learn  new  ones.Work  with  leading-­‐edge  technologies.  Experiment  and  explore  using  our  in-­‐house  sandbox.  Generous  training  allowance  .  Professional  development  days.  Internal  speaker  series.  

• Fun,  fun  ,fun    – Blog  during  work  hours-­‐please  !  Enjoy  monthly  cheese  

tas4ngs  with  our  resident  cheese  sommelier.  Take  a  day  off  and  volunteer  for  your  favourite  charity.  

3

#PythianLife

4

Welcome to CLV15

Where do I come From–Oracle  DBA  

• Started  with  Version  9.2  in  2004  –Speaker  at  Oracle  Open  World,  Oracle  

Developers  Day  and  Collaborate    –Co-­‐President  of  ORAMEX  (Mexico  Oracle  

User  Group)    –Web  Events  Chair  for  IOUG  Cloud  

Compu4ng  Special  Interest  Group  (SIG);  –Movie  Fana4c  &  Music  Lover  –Bringing  the  best  from  México  (Mexihtli)  to  

the  rest  of  the  world  and  in  the  process  photographing  it  :)  

– rene-­‐ace.com  –@rene_ace  

• #CLV15

5

Where do I come from?

6

How did you get to be a DBA

7

6th Happiest Job of 2014! It  was  #1  in  2014

8

http://www.forbes.com/sites/susanadams/2014/03/20/the-happiest-and-unhappiest-jobs-in-2014/

Work-life balance

Relationship with boss and co-workers

Daily tasksJob resources

Field will grow by 15% between

2012 and 2022

DBA can be the key driver of

success

Happiest Job of 2034?

Oxford University: THE FUTURE OF EMPLOYMENT: HOW SUSCEPTIBLE ARE JOBS TO COMPUTERISATION?

• 47  percent  of  American  jobs  are  at  high  risk  of  being  taken  by  computers  within  the  next  two  decades.  – 1st  Wave    

• Computers  will  start  replacing  people  in  especially  vulnerable  fields  like  transporta4on/logis4cs,  produc4on  labor,  and  administra4ve  support.  

– 2nd  Wave  • Dependent  upon  the  development  of  good  ar4ficial  

intelligence.  This  could  next  put  jobs  in  management,  science  and  engineering,  and  the  arts  at  risk.

9

How can you tell an Oracle DBA has touched your MySQL Installation?

MYSQL_HOME=/home/oracle/products/mysql-­‐version  

10

What is a Relational Database?

11

• Accessible  and  Rela4onal      • Data  Management    • Organised  Data   Store    • Systema4c  Data  Architecture    • Security  Enabled    • Tuneable  Performance    • Ability  to  u4lise  Opera4ng  System  and  its  resources  at  

kernel  level  system  soJware.  

What is MySQL?

12

• Popular  soJware  in  open  source  world  with  GNU  license    

• Easy  to  manage  ,  easy  to  use  features    • Mul4  plaqorm  support    • Adoptable  to  popular  soJware  such  as  PHP  and  Java    • De-­‐facto  standard  for  web  and  e-­‐commerce  

applica4ons  and  unbeatable  choice  for  startups

MySQL Version History

13

MySQL  5.7:  Improvements  in  latest  DMR

• InnoDB  for  beter  transac4onal  throughput,  availability,  IO    

•Replica.on  for  beter  scalability  and  availability  

• Fabric  for  high  availability  and  sharding  •Performance  Schema  for  new  and  improved  

performance  metrics  •Op.mizer  for  beter  EXPLAINing,  parsing,  

query  performance  •GIS  with  na4ve  InnoDB  spa4al  indexes  and  

Boost.Geometry  integra4on

Terminology

14

Oracle MySQL

Database (files) Database Server Instance

Database Instance (memory) Database Server Instance

Schema User Database

User User

Table Space Table Space

Storage Engine

MySQL Architecture

15

MySQL Architecture

16

• SQL  Interface  • Mechanisms  to  receive  commands  and  transmit  

results  to  the  user.  • Parser  

• Lex-­‐YACC  script  that  is  compiled  with  Bison  • Constructs  a  query  structure  used  to  represent  the  

query  statement  (SQL)  in  memory  as  a  tree  structure  (abstract  syntax  tree)

MySQL Architecture

17

• Query  Op.mizer  • Is  a  SELECT-­‐PROJECT-­‐JOIN  

• Query  Execu.on  • Set  of  library  methods  designed  to  implement  a  par4cular  query.  

• mysql_insert()  method  is  designed  to  insert  data.  • mysql_select()  method  designed  to  find  and  return  data  

matching  the  WHERE  clause  • located  in  a  variety  of  source  code  files  under  a  file  of  a  

similar  name  (e.g.,  sql_insert.cc  or  sql_select.cc)

MySQL Architecture

18

• Query  Cache  • Caches  not  only  the  query  structure  but  also  the  query  results  

• Buffer  Manager/Cache  and  Buffers  • Responsible  for  ensuring  that  the  most  frequently  used  data  (or  structures,  as  you  will  see)  are  

available  in  the  most  efficient  manner  possible  • Table  Cache  

• Designed  to  store  metadata  about  the  tables  in  memory.  • Record  Cache  

• Enhances  sequen4al  reads  from  the  storage  engines  • Key  Cache  

• Buffer  for  frequently  used  index  data  (used  exclusively  for  MyISAM  tables  )  • Privilege  Cache  

• used  to  store  grant  data  on  a  user  account  • Hostname  Cache  

• Contains  the  hostnames  of  all  the  connec4ons  to  the  server

MySQL Architecture

19

• The  Storage  Manager  • Interfaces  with  the  opera4ng  system  to  write  data  to  

the  disk  efficiently  • Transac.on  Manager  

• Provides  a  locking  facility  to  ensure  that  mul4ple  simultaneous  users  access  the  data  in  consistent  way

Option Files

20

• Beter  known  as  configura4on  files  

• Lists  specific  startup  op4ons  • Common  Loca4ons  of  Op4on  

File  • Windows  

• INSTALLDIR\my.ini  • INSTALLDIR\my.cnf  

• Linux  &  Mac  • /etc/my.cnf  • /etc/mysql/my.cnf

# The following options will be passed to all MySQL clients[client]

#password = your_password

port = 3306

socket = /tmp/mysql.5.6.16.sock

# Here follows entries for some specific programs# The MySQL server [mysqld] port= 3306 socket= /tmp/mysql.5.6.16.sock log-error = /Users/ligaya/mysql_installs/mysql-5.6.16-osx10.7-x86_64/data/error.log

server-id= 1 table_open_cache = 4000

How to find your Option Files

21

$strace -e stat64 mysqld --print-defaults$lsof -p <mysqld_pid)

Generic Unix/Linux Binary File Layout

22

Directory Contents

bin Client programs and mysqld server

data Log files and databases

docs Manual in Info format

man Unix manual pages

include Include (header) files

lib Libraries

scripts mysql_install_db

share Misc support files including error messages, sample option files and SQL for database installation

sql-bench Benchmarks

Linux RPM File Layout

23

Directory Contents

/usr/bin Client programs and scripts

/usr/sbin mysqld server

/var/lib/mysql log files and databases

/usr/share/info Manual in Info format

/usr/share/man Unix manual pages

/usr/include/mysql

Include (header) files

/usr/lib/mysql Libraries

/usr/share/mysql Misc support files including error messages, character set files, sample option files and SQL for database installation

/usr/share/sql-bench Benchmarks

Disk Space Usage

24

• Database  directories  • Hold  data  files  associated  with  storage  engine  types  

• Table  format  files  (*.frm)  • Keeps  the  defini4on  (schema)  of  the  table  

• Data  and  index  files  • Tables  under  same  top-­‐  level  directory  used  for  database  • MyISAM  storage  engine  creates  one  file  per  table  for  

each  data  and  index.

It can be as easy…

25

[root@localhost ~]# groupadd mysql [root@localhost ~]# useradd -r -g mysql mysql [root@localhost ~]# yum localinstall mysql-community-release-el6-6.noarch.rpm [root@localhost ~]# yum repolist enabled | grep "mysql.*-community.*" [root@localhost ~]# yum repolist all | grep mysql [root@localhost ~]# yum install mysql-community-server [root@localhost ~]# service mysqld start Starting mysqld:[ OK ] [root@localhost ~]# service mysqld status mysqld (pid 407) is running. [root@localhost ~]# mysql_secure_installation

Generic  Binary  Installa4on

Storage Engines

26

• General-­‐purpose  storage  engine  • As  of  5.5  is  the  default  engine  • Main  Features:  

• ACID  compliant  • Row  level  locking  with  consistent  reads  Foreign  keys  • Clustered  primary  key  index  • FullText  Indexes

InnoDB

Storage Engines

27

• Default  storage  engine  prior  to  5.5  Stored  on  disk  as  3  files  •    .frm  (Format  file)  •    .MYD  (Data  file)  •    .MYI  (Index  file)  

• Features  • FullText  indexes  • Geospa4al  datatypes  support  

• There  is  a  limit  of  232  (~4.295E+09)  rows  in  a  MyISAM  table  

MyISAM

Storage Engines

28

• Memory    • CSV    • Archive    • Blackhole    • 3rd  Party

Others

Storage Engines

29

mysql> mysql> SHOW ENGINES\G*************************** 1. row *************************** Engine: MEMORY Support: YES Comment: Hash based, stored in memory, useful for temporary tablesTransactions: NO XA: NO Savepoints: NO*************************** 2. row *************************** Engine: MyISAM Support: YES Comment: MyISAM storage engineTransactions: NO XA: NO Savepoints: NO*************************** 3. row *************************** Engine: InnoDB Support: DEFAULT Comment: Supports transactions, row-level locking, and foreign keysTransactions: YES XA: YES Savepoints: YES

9 rows in set (0.28 sec)

Creating a table with a specific storage engine

30

mysql> CREATE TABLE test1 (col1 INT, col2 CHAR(30), PRIMARY KEY (col1)) ENGINE = INNODB;Query OK, 0 rows affected (0.29 sec)

mysql> CREATE TABLE test2 (col1 INT, col2 CHAR(30)) ENGINE = MYISAM;Query OK, 0 rows affected (0.36 sec)

Creating a table with a specific storage engine

31

mysql> SHOW TABLE STATUS FROM rene;+------------+-----------------+----------+----------------+---------+| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length |Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time |Collation | Checksum | Create_options | Comment |+--------+--------+---------+------------+------+----------------+-------------+------------+-----------------+----------+----------------+---------+| test1 | InnoDB | 10 | Compact | 0 | 0 | 16384 | 0 | 0 | 0 | 1 | 2015-01-08 18:21:05 | NULL | NULL |utf8_general_ci | NULL | | || test2 | MyISAM | 10 | Fixed | 0 | 0 | 0 | 26740122787512319 | 1024 | 0 | NULL | 2015-01-08 18:35:47 | NULL | NULL |utf8_general_ci | NULL | | |+--------+--------+---------+------------+------+----------------+-------------2 rows in set (0.03 sec)

Storage Engines

32

Feature MyISAM Heap BDB InnoDB

Transactional No No Yes Yes

Lock granularity Table Table Page Row

Storage A data file (.MYD) and an index file (.MYI) for each table

In-memory A single data and index file (.db) for each table

A set of data files for all the tables

Portable Yes N/A No Yes

Contrasting Architecture

33

Area MySQL Oracle

Memory Caches • MyISAM key caches

• InnoDB data cache

• InnoDB log cache

• Dictionary cache

Data cache (variants)

Log buffer

Shared Pool

Java Pool Redo/Undo Logs • InnoDB Undo Space

• InnoDB Logs

• Undo Tablespace

• Redo Logs

• Archive Logs Data Storage • Tablespaces

• Table/Index Files

• Format files

• Tablespaces

• Datafiles

Optimizer • Cost-based • Cost-based

Error Log

34

• Default  loca4on:  host_name.err  in  datadir  • Logs  

• Start  and  stops  • Cri4cal  errors  • InnoDB  Monitor  data  (if  turned  on)  • MyISAM  tables  that  need  to  be  checked  or  repaired    • Some  OS’s  -­‐  stack  trace  if  mysqld  crashes

log-­‐error[=file_name]

Slow Query Log

35

• First  line  of  offense  for  tuning  queries    • Why?  

• Performance  usually  • Enable  dynamically  or  with  -­‐-­‐slow-­‐query-­‐log  

• Default  file  loca4on:  host_name-­‐slow.log  in  the  datadir  • Can  also  go  to  a  table  

• Mul4ple  op4ons  for  controlling  it  • Aggregate  its  data  with  mysqldumpslow  u4lity

log-­‐error[=file_name]

Explain

36

• Prior  to  5.6  EXPLAIN  [explain_type]  SELECT  select_options  

• 5.6  EXPLAIN  [explain_type]  explainable_stmt

Displays  query  execu4on  plan  SyntaxColumn Meaning

id The SELECT identifier

select_type The SELECT type

table The table for the output row

type The join type

possible_keys The possible indexes to choose

key The index actually chosen

key_len The length of the chosen key

ref The columns compared to the index

rows Estimate of rows to be examined

Extra Additional information

Explain

37

mysql> explain extended select j1.c1 from j1, j2, j3 where j1.c1 = j2.c1 and j3.c1 = j1.c1;+----+-------------+-------+------+---------------+------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------+------+---------------+------+---------+------+------+-------------+| 1 | SIMPLE | j1 | ALL | NULL | NULL | NULL | NULL | 2 | || 1 | SIMPLE | j2 | ALL | NULL | NULL | NULL | NULL | 3 | Using where || 1 | SIMPLE | j3 | ALL | NULL | NULL | NULL | NULL | 4 | Using where |+----+-------------+-------+------+---------------+------+---------+------+------+-------------+3 rows in set, 1 warning (0.00 sec)

Example

User Accounts

38

They  are  divided  into  2  parts

• username  • hostname  • Eg:  

• ‘root’@‘localhost’    • ‘’@‘127.0.0.1’    • ‘root’@‘192.168.1.%’    • ‘lig’@’%’

• Stage  1:  Connec4ng    • Who  are  you?  

• host  • username  

• Stage  2:  Request  • For  each  request  

• What  are  you  doing  and  are  you  allowed  to  do  that?  (Grants)

Backups

39

They  are  divided  into  2  parts• Logical  Backups  • What  is  Saved?  

• Logical  structure  • Content  

• Machine  Independent  • Slower  • Server  Up/Warm  • Commonly  used  

• mysqldump  • Editable  text  files  • Very  litle  Scalability

[mysql@localhost Desktop]$ mysqldump -u pythian – p $pw --extended-insert=true --add-drop-database--databases rene | gzip > /export/full_dump.gz

Backups

40

They  are  divided  into  2  parts• Physical  Backup  • Raw  copies  • Faster  • Compact  • Usually  server  is  down  or  locked  • Commonly  used  

• Filesystem  Snapshot  • MySQL  Enterprise  Backup    

• Official  solu4on  for  “Hot”  backups  

• XtraBackup  (3rd  party  solu4on  from  Percona)

a)  FLUSH  TABLES  WITH  READ  LOCK;  

b)  Take  the  filesystem  snapshot  c)  UNLOCK  TABLES;

MySQL SYS Schema

41

• SYS  is  similar  to:  • Oracle  V$  catalog  views  • MicrosoJ  SQL  DMVs  (Dynamic  Mgmnt  Views)  • IBM  DB2  SYSIBM  catalog  

• Helps  simplify  DBA  tasks  • Monitor  health,  growth  rates,  and  other  key  metrics  • Spot,  diagnose,  and  tune  performance  problems  

• Provides  easy  to  understand  insights  into  • IO  hot  spots,  Costly  SQL  statements  • Dynamic  table,  index,  and  schema  sta4s4cs  • Wait  4me  analysis,  Locking  • InnoDB  sta4s4cs

Schema Objects in Oracle and MySQL

42

Oracle MySQL

Package N/A

PL/SQL procedure Routine

PL/SQL function Routine

Role N/A

Sequence AUTO_INCREMENT for a column

Synonym N/A

MySQL Case Sensitivity

43

• Oracle:  • Column,  index,  stored  procedure,  

and  trigger  names  as  well  as  column  aliases  are  case  insensi4ve  on  all  plaqorms  

• MySQL  • Databases  correspond  to  directories  

within  the  data  directory.Case  sensi4vity  of  the  database  and  table  names  is  determined  by  the  case  sensi4vity  of  the  underlying  opera4ng  systems

44

MySQL Replication - How it works?

45

MySQL Replication - How it works?

46

• On  the  master  • Replica4on  events  writen  to  a  

special  log  called  binary  log  • stores  data  that  

replica2on  slave  will  be  reading  later  

• Replica2on  slave  connects  to  a  master,  master  creates  a  new  thread  for  the  connec2on  

• On  the  replica  • Two  threads  are  started  on  the  slave  

• IO  thread  • Reads  binary  log  events  from  the  

master  as  they  come  in  and  just  copies  them  over  to  a  local  log  file  called  relay  log  

• SQL  thread  • Reads  events  from  a  relay  log  

stored  locally  on  the  replica4on  slave  and  then  applies  them  as  fast  as  possible.

MySQL Replication Types

47

MySQL Best Features

48

• Free,  Almost  Free,  Lowest  Total  Cost  of  Ownership,  Open  Source    

• Easy  setup  and  low  profile  replica4on  features    • Scalability  and  Flexibility      • High  Performance    • High  Availability    • Robust  Transac4onal  Support    • Works  with  very  limited  resources    • Plaqorm  availability  

MySQL Features - Cons

49

• Tablespace  management    • Role  access  management    • Premature  Stored  Procedures  and  Triggers    • Advanced  Par44oning    • Advanced  Views    • Lack  of  Parallelism      • Struggle  with  Large  and  Cri4cal  Data    • Limited  Backup  and  Recovery  op4ons  for  VLDB    • Some  other  features  in  5.6,  5.7  …

50

FIT-ACER

• F – Focus (SLOW DOWN! Are you ready?)

• I – Identify server/DB name, time, authorization

• T – Type the command (do not hit enter yet)

• A – Assess the command (SPEND TIME HERE!)

• C – Check the server / database name again

• E – Execute the command

• R – Review and document the results

51

52

rene-ace.com

53

To contact us

sales@pythian.com

1-877-PYTHIAN

To follow us

http://www.pythian.com/blog

http://www.facebook.com/pages/The-Pythian-Group/163902527671

@pythian

http://www.linkedin.com/company/pythian

Thank you – Q&A