从 oracle 合并到 mysql - npr 实例分析

7/27/2019 Oracle MySQL - NPR

1/24

npr.org

Migration From Oracle to MySQL

An NPR Case Study

By Joanne Garlow


2/24

OverviewBackground

Database Architecture

SQL Differences

Concurrency Issues

Useful MySQL Tools

Encoding Gotchas


3/24

BackgroundNPR (National Public Radio)

Leading producer and distributor of radio programming

All Things Considered , Morning Edition , Fresh Air , Wait,

Wait, Don t Tell Me, etc .

Broadcasted on over 800 local radio stations nationwide

NPR Digital MediaWebsite (NPR.org) with audio content from radio programs

Web-Only content including blogs, slideshows, editorialcolumns

About 250 produced podcasts, with over 600 in directory

Mobile apps and sites

Syndication


4/24

High-Level System Architecture


5/24

Limitations of the OracleArchitecture

Reached capacity of single system to support our

load

Replication outside our budget

Databases crashes were becoming frequent


6/24

Database Architecture GoalsRedundancy

Scalability

Load balancing

Separation of concerns

Better security


7/24

High-Level System Architecture


8/24

Updated by our Content Management System

Transaction Oriented Resource Contention Highly Normalized

Isolation from main website

Read-only by our webservers Horizontally scalable

Database Architecture

MainInnoDB

AMGMyISAM

PUBLICInnoDB

STATIONSInnoDB

Main

RO slave

MainRO slave

ContentMgmt System

Web Servers

Scripts BackupRO slave

Read and updated only by our website Low resource contention

Small tables or log tables Short Transactions

Updated by a nightly script

Read-only by our ContentManagement System Need fast full text queries(replacing Oracle Text) Large tables

Updated by a quarterly script Read-only from our website Some log type information written Low resource contention No transactions


9/24

Issues When Converting SQLMySQL is case sensitive

Oracle outer join syntax (+) -> OUTER JOIN clause

Oracle returns a zero to indicate zero rows updated MySQL returns TRUE (1) to indicate it

successfully updated 0 rows

MySQL sorts null to the top, Oracle sorts null tothe bottom

Use order by colName desc for sorting asc with

nulls at bottom

M SQL has Limit clause YAY!


10/24

Replacing Oracle SequencesInitialize a table with a single row:

CREATE TABLE our_seq (

id INT NOT NULL

);

INSERT INTO our_seq (id) VALUES (120000000);

Do the following to get the next number in the sequence :

UPDATE our_seq SET id=LAST_INSERT_ID(id+1);

SELECT LAST_INSERT_ID();


11/24

Replacing Oracle SequencesFor updating many rows at once, get the total number of unique IDs you needfirst:

SELECT @totalRows := COUNT(*) FROM...

Then update npr_seq by that many rows:UPDATE npr_seq SET id=LAST_INSERT_ID(id+@totalRows);

and store that ID into another variable:SELECT @lastSeqId := LAST_INSERT_ID();

Then use the whole rownum workaround described above to get a unique value foreach row:INSERT INTO my_table (my_primary_id . . . ) SELECT@lastSeqId - (@rownum:=@rownum+1), . . . FROM (SELECT

@rownum:=-1) r, . . .


12/24

Converting FunctionsNVL() -> IFNULL() or COALESCE()DECODE() -> CASE() or IF()Concatenating strings || -> CONCAT()

test || null returns test in Oracle

CONCAT(

test

,null) returns null in MySQLLTRIM and RTRIM -> TRIM()INSTR() works differently.

Use LOCATE() for Oracle s INSTR() with occurrences =1.SUBSTRING_INDEX() and REVERSE() might also work.


13/24

Converting Datessysdate -> now()

Adding or subtracting

In Oracle 1 subtracts a dayIn MySQL - 1 subtracts a milisecond must

use interval

TRUNC() -> DATE()

TO_DATE and TO_CHAR -> STR_TO_DATE and

DATE_FORMAT


14/24

Update DifferencesYou can't update a table that is used in the WHEREclause for the update (usually in an "EXISTS" or asubselect) in mysql.

UPDATE tableA SET tableA.col1 = NULLWHERE tableA.col2 IN(SELECT tableA.col2

FROM tableA A2, tableBWHERE tableB.col3 = A2.col3 AND

tableB.col4 = 123456);

You can join tables in an update like this (Mucheasier!):

UPDATE tableAINNER JOIN tableB ON tableB.col3 = tableA.col3SET tableA.col1 = NULLWHERE tableB.col4 = 123456;


15/24

RANK() and DENSE_RANK()We really found no good MySQL equivalent for

these functions

We used GROUP_CONCAT() with an ORDER BY andGROUP BY to get a list in a single column over a

window of data


16/24

CollationYou can set collation at the server, database,

table or column level.

Changing the collation at a higher level (say onthe database) won t change the collation for

preexisting tables or column.

Backups will use the original collation unlessyou specify all the way down to column level.


17/24

Concurrency IssuesIn our first round of concurrency testing, our

system ground to a halt!

DeadlocksSlow Queries

MySQL configuration

sync_binlog = 1 // sync to disk, slow but safe

innodb_flush_log_at_trx_commit = 1 // write each

commit

transaction_isolation = READ-COMMITTED


18/24

Useful MySQL ToolsMySQL Enterprise Monitor

http://www.mysql.com/products/enterprise/

MySQL GUI Tools Bundle:

http://dev.mysql.com/downloads/gui-tools/5.0.html

MySQL Query Browser similar to Oracle s SQL

Developer

MySQL Administrator
http://www.mysql.com/products/enterprise/http://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://dev.mysql.com/downloads/gui-tools/5.0.htmlhttp://www.mysql.com/products/enterprise/http://www.mysql.com/products/enterprise/http://www.mysql.com/products/enterprise/http://www.mysql.com/products/enterprise/http://www.mysql.com/products/enterprise/http://www.mysql.com/products/enterprise/http://www.mysql.com/products/enterprise/


19/24

Innotop and innoDB Statusinnotop

http://code.google.com/p/innotop

Helped us identify deadlocks and slow queries(don t forget the slow query log!)

In mysql, use

show engine innodb status\G;

Useful for contention and locking issues
http://code.google.com/p/innotophttp://code.google.com/p/innotophttp://code.google.com/p/innotophttp://code.google.com/p/innotophttp://code.google.com/p/innotophttp://code.google.com/p/innotophttp://code.google.com/p/innotophttp://code.google.com/p/innotop


20/24

Query ProfilingTry the Query Profiler with Explain Plan when

debugging slow queries

http://dev.mysql.com/tech-resources/articles/using-new-query-profiler.html
http://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.htmlhttp://dev.mysql.com/tech-resources/articles/using-new-query-profiler.html


21/24


22/24


23/24

Continuing IssuesBugs with innodb locking specific records (as

opposed to gaps before records)

Uncommitted but timed out transactionsUse innotop or show engine innodb status\G;

and look for threads waiting for a lock but no

locks blocking them

Requires MySQL reboot


24/24

Questions?Joanne Garlow

[email protected] http://www.npr.org/blogs/inside
mailto:[email protected]://www.npr.org/blogs/insidehttp://www.npr.org/blogs/insidehttp://www.npr.org/blogs/insidehttp://www.npr.org/blogs/insidehttp://www.npr.org/blogs/insidehttp://www.npr.org/blogs/insidehttp://www.npr.org/blogs/insidehttp://www.npr.org/blogs/insidemailto:[email protected]:[email protected]:[email protected]

从 oracle 合并到 mysql - npr 实例分析

Documents