web - oracle · •node.js nosql api couples high performance, distributed ......

从WEB 到云

杨鲁 [email protected]

MySQL 技术顾问

当下企业的迫切需要

CLOUD

SOCIAL MOBILE

Text

到 2014年, 90% of 的公司将要支持个人移动设备访问他们的应用

56% 的企业高管认为云计算将会使他们的组织更加灵活和有竞争力

80% 的财富500强企业高管表示，企业范围内的协作是成功的关键

90% 的财富500强企业将会拥有一个大数据的项目部署计划，到2012年年底

BIG

DATA

私有云PRIVATE

公有云PUBLIC

混合云HYBRID

私有云的使用率增加

Source: IOUG ResearchWire member studies on Cloud Computing, conducted in Aug-Sept 2010 and Aug-Sept 2010

Yes, in production at

scale, 17.7%

Yes, in limited use, 12.2%

Yes, in pilot stage, 7.1%

Preliminary planning, 7.5%

Under consideration, 1

0.6%

No, 35.4%

Don’t know/unsure, 9.

5%

Yes, in production at

scale, 11.3%

Yes, in limited use, 12.8%

Yes, in pilot stage, 4.5%

Preliminary planning, 4.9%

Under consideration, 1

0.5%

No, 47.4%

Don’t know/unsure, 8.

7%

20112010

37% have private clouds today

28% increase from 2010 to 2011

20102011

公有云的使用率增加

Source: IOUG ResearchWire member studies on Cloud Computing, conducted in Aug-Sept 2010 and Aug-Sept 2011

20112010

21% use public clouds today

50% increase from 2010 to 2011

Yes, 20.9%

Under consideratio

n, 13.6%No, 52.9%

Don’t know/unsure

, 12.6%

Yes, 13.8%

Under consideratio

n, 11.2%

No, 54.6%

Don’t know/unsure

, 20.4%

20102011

MySQL: 从WEB 到云

各种不同的扩展架构

与WEB开发应用完美结合

在普通硬件上能够获取较佳性能

运行在所有主流平台|与所有主要语言

很多Dev/Ops 的方法支持

WEB

ENTERPRISE

EMBEDDED

CLOUD

UBIQUITOUS IN THE CLOUD

Performance, Scale-Out, High Availability, Self-Healing,

Data Integrity, Provisioning, Monitoring & Resource

Management, Security

MySQL Enterprise Monitor 2.2

MySQL Cluster 7.1

MySQL Cluster Manager 1.0

MySQL Workbench 5.2

MySQL Database 5.5

MySQL Enterprise Backup 3.5

MySQL Enterprise Monitor 2.3

MySQL Cluster Manager 1.1

DRIVING MySQL

INNOVATION

All GA!

MySQL Enterprise Backup 3.7

Oracle VM Template for MySQL

Enterprise Edition

MySQL Enterprise Oracle Certifications

MySQL Windows Installer

New MySQL Enterprise

Commercial Extensions

MySQL Database 5.6 DMR*

MySQL Cluster 7.2 DMR

MySQL Labs!

(“early and often”)

All GA!

MySQL Cluster 7.2

MySQL Utilities 1.0.6

Database Migration Wizard

New Windows Tools/Features

New MySQL Enterprise

Commercial Extensions

MySQL Database 5.6

MySQL Cluster 7.3

MySQL Cluster features

*Development Milestone Release

2010 2011 2012

All GA!

New!

MySQL 5.6 增强了云的领先地位

• Optimizer & InnoDB 更好的性能和扩展性

– 减少了内部预分配的线程

• Performance Schema 更细化的监控单元（Instrumentation）– 统计不同用户, hosts, 帐户, 查询等,资源消耗

• Replication 提供高可用性，扩展性，数据完整性– 灵活性 & 自愈

• “NoSQL” 提供一种选择– 与云服务集成

– 高速摄取和数据的查询，不牺牲ACID

Improving:New!

MySQL 5.6 INNODB

• 提升资源利用率– 去除多余的锁, CPU cache 共享

– 提高并发处理能力

– 优化只读的吞吐能力

– SSD 优化

• 增强了更灵活，更快速的功能扩展– 在线DDL 操作

– 可移动的 Tablespaces

– Dump, restore/warm buffer pool

– NoSQL, Key-value 方式访问InnoDB

– Full-Text Search

New!

通过REPLICATION实现弹性扩展

• 多线程Slaves

• Binary Log Group Commit

• 优化基于行的复制

性能和扩展性

• 自动导换或切换

• Global Transaction Identifiers• Crash Safe Slaves & Binlogs

• Replication Event Checksums

自愈

• Time Delayed Replication

• Remote Binlog Backup• Informational Log Events

DEV/OPS 灵活性

自愈性--MySQL CLOUD REPLICATION

• 简单的跟踪和比较复制整个云实例

– 唯一标识符写入到binlog的每个事务

• 自动检测最新的slave进行切换

• 部署n层复制的层次结构

Global Transaction Identifiers

Master

GTID=123456

GTID=123456

GTID=123456 GTID=123456

• 打开自愈复制集群

• 自动导换和恢复

– mysqlfailover Utility

• 切换&管理

– mysqlrpladmin Utility

• Integrated HA to tolerate instance and region failures

HA UtilitiesMonitoring

Failed

Master

Slaves

Promoted

Master

新的复制工具

5x Higher Replication Performance

• SysBench, running across 10 x schemas

• Oracle Linux 6.1, Oracle Sun Fire x4150 m2 Server

0 5 10

QPS 58 144 283

0

50

100

150

200

250

300Q

ue

rie

s p

er

Seco

nd

Worker Threads

Multi-Threaded Slave Performance

最好的两种方式，没有之一SQL and NoSQL

• Key-Value方式和复杂查询混合

• 事务的完整性保证

• 标准化&很多工具支持

MySQL 5.6 INNODBBest of Both Worlds

Clients and Applications

MySQL ServerMemcached Plug-in

innodb_memcached

local cache(optional)

Handler API InnoDB API

InnoDB Storage Engine

mysqld process

SQL Memcached Protocol

0

10000

20000

30000

40000

50000

60000

70000

80000

8 32 128 512

TP

S

Client Connections

MySQL 5.6: NoSQL Benchmarking

Memcached API

SQL

Up to 9x Higher “SET / INSERT” Throughput

MySQL CLUSTER VISION

―The default database for anyone deploying

rapidly evolving, real-time transactional

services at web-scale, where downtime is

simply not an option‖

WHO’S USING MYSQL CLUSTER?

MySQL CLUSTER 体系结构灵活，弹性，可扩展

Clie

nts

Ap

plic

atio

n

Laye

r

Dat

a N

od

es

Mg

mt

MySQL CLUSTER 体系结构在线扩容

Clie

nts

Ap

plic

atio

n

Laye

r

Dat

a N

od

es

Mg

mt

MySQL CLUSTER 体系结构没有单点故障

Clie

nts

Ap

plic

atio

n

Laye

r

Dat

a N

od

es

Mg

mt

MySQL CLUSTER 7.2Delivering 1 Billion Writes per Minute

70x Faster JOINs

Fastest Ramp Ever

8x Higher Per Node

Performance

NoSQLMemcached

API

70x Faster JOINs

MySQL Cluster 7.3性能提升DBT2 Benchmark

MULTI-REGION AVAILABILITY

• DR and Global Scale

• Replicate complete clusters across regions– Fully active/active

– No passive resources

• Split individual clusters across availability zones– Synchronous replication &

auto-failover between sites

跨站点复制

MySQL CLUSTER 7.3 New Features

• Foreign Keys bring MySQL Cluster to a broader range of

workloads

• Node.js NoSQL API couples high performance, distributed

apps, with high performance distributed database

• Auto-Installer allows fast configuration, auto-discovery

and optimizations to enhance ease of use

• Available Now! Get it here:

– dev.mysql.com/downloads/mysql/

New!

低成本多平台支持性能优容易使用

MySQL 企业版的优势

管理工具高级特性支持和认证

扩展性高可用性安全审计

监控MONITORING备份BACKUP开发支持DEVELOPMENT DBA管理支持迁移

技术支持咨询服务 ORACLE 认证

MySQL ENTERPRISE MONITOR

• 提供对于MySQL环境，单独的，统一的视图，内部部署和云

• 自动化的，以规则为基础的监测和警报（SMTP，SNMP启用的情况）

• 通过监控图表显示，查询采集，监测，分析和调整的结果

• 可视化监控的“热点”的应用程序和服务器

• 自动检测的主从复制拓扑，并添加监控

• 集成了MY ORACLE SUPPORT

Saves you time. Fixes problems you

cannot find yourself.

MySQL ENTERPRISE SCALABILITY

• MySQL thread-handling: excellent

performance, can limit scalability as

connections grow

• MySQL Thread Pool improves

sustained performance/scale as

user connections grow

• Thread Pool API

MySQL Thread Pool gives 20x Better Scalability (RW)

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

Tra

nsacti

on

s P

er

Seco

nd

Simultaneous Database Connections

MySQL 5.5 Sysbench OLTP Read/Write

Reduces the need for Cloud providers to

provision additional MySQL instances to meet performance requirements

• 对所有授权的数据库，进行帐户的集中配置，管理和验证

• 让云供应商使用行业标准的PAM和Windows模块，以实现全球范围内的安全

• 让云消费者对那些在云端访问配置数据库的用户执行集中认证

MySQL Enterprise Security

3. User name/password sent

to the PAM library, yes/no

answer from PAM library

returned to client.

1. Joe logs in using

application user

name/password.

Connected

CREATE USER joe

IDENTIFIED WITH 'authentication_pam'

AS ‘mysql';

App

PAM library

and various

backends

2. Connector sends the user

name/password to the

MySQL server.

Connector

4. PAM library verifies credentials

(using e.g. LDAP or Kerberos

etc) and returns yes/no answer

for delivery to client.

PAM

Authentication

MySQL Enterprise SecurityExample - PAM

MySQL ENTERPRISE AUDIT

• 日志记录连接，登录，查询所有或特定的MySQL服务器活动

• 用户定义的策略，过滤和日志轮换

• 动态地启用，禁用：不需重新启动服务器

• 符合Oracle审计规范的基于XML的审计流

• 通过MySQL的5.5 AUDIT API很容易实现

• MySQL5.5.28或更高版本Adds regulatory compliance to MySQL

applications (IPAA, Sarbanes-Oxley, PCI, etc.)

New!

MySQL ENTERPRISE HIGH AVAILABILITY

• Oracle VM Templates for MySQL

• HA with DRBD and Linux Clustering

• HA with Solaris Clustering

• HA with Windows Clustering

Range of options to make your cloud-based MySQL applications deployments

Highly Available

MySQL ENTERPRISE BACKUP

• 在线，非锁定的备份与恢复

• 表，索引

• 服务器，数据库或对象级

• 逻辑和物理备份

• 全备份或增量备份

• Point-in-time即时恢复

• 压缩备份

• 提供了对MyISAM表的备份和恢复

• 跨平台（Windows，Linux和Unix的）

Ensures quick, online backup and recovery of your cloud-based MySQL

applications

Enhancing DevOps Agility, Reducing Downtime

Self-HealingAutomated

ManagementHA Operations

• Start / Stop node or

whole cluster

• On-Line Scaling

• On-Line

Reconfiguration

• On-Line Upgrades

• Node monitoring

• Auto-recovery

extended to SQL

+ mgmt nodes

• Cluster-wide

configuration

consistency

• Persistent

configurations

• HA Agents

MySQL CLUSTER MANAGER

How Does MySQL Cluster Manager Help?

Example: Initiating upgrade from MySQL Cluster 7.0 to 7.2

• 1 x preliminary check of cluster state

• 8 x ssh commands per server

• 8 x per-process stop commands

• 4 x scp of configuration files (2 x mgmd & 2 x

mysqld)

• 8 x per-process start commands

• 8 x checks for started and re-joined processes

• 8 x process completion verifications

• 1 x verify completion of the whole cluster.

• Excludes manual editing of each configuration

file.

Total: 46 commands - 2.5 hours of attended

operation

Before MySQL Cluster Manager With MySQL Cluster Manager

upgrade cluster --package=7.1 mycluster;

Total: 1 Command -

Unattended Operation

• Results– Reduces the overhead and complexity

of managing database clusters

– Reduces the risk of downtime resulting from administrator error

– Automates best practices in database

cluster management

Copyright 2011 Oracle Corporation 43

ORACLE MySQL PREMIER SUPPORT

• MySQL的最大的工程和支持组织• 由MySQL开发者的支持• 世界一流的支持，以29种语言• 热修复和维护版本• 全天候• 无限事故• 咨询支持• 全球规模和范围

CommunityCommercial

Resource Utilization

CommunityCommercial

Lower TCO

CommunityCommercial

Dev/Ops Productivity

WITH MySQL ENTERPRISE & CGE

• Enhanced Agility:

– Pre-Built Templates

– Automated Scaling & Management

– Flexible, Real-Time Backups

• Improved Service Levels:

– Integrated Security & Auditing

– Self-Healing Failure Recovery

– 24 x 7 Technical Support

Cloud-based Deployments (1)

• Run Only One MySQL Server Per Compute Instance

– MySQL will run faster with dedicated computational units and

memory resources

• Leverage larger instance types for heavy usage

– The largert instance types also have higher I/O throughput

capabilities

• Predictable I/O, Disk and Network Bandwidth

– Define specific I/O performance requirements, and provision

dedicated network resources

Best Practices

0

200

400

600

800

1000

1200

1400

1600

8 16 32 64 128 256

TPS

Threads

Sysbench: Read Only

Large

Extra Large

0100200300400500600700800900

1000

8 16 32 64 128 256

TPS

Threads

Sysbench: Read / Write

Large

Extra Large

Measuring Scalability on AWS

• 2x higher performance with 2x larger instance– Large: 2 x 2.4GHz, 7.5GB RAM, 2 x 20GB EBS volumes

– Extra Large: 4 x 2.4GHz, 15GB RAM, 2 x 20GB EBS volumes

– MySQL 5.5.25a + Oracle Linux 6.1

Measuring Scalability on AWS

• 2x higher performance as CPU + Memory doubles

• 1.6x with double CPU, 50% of the memory

0

5000

10000

15000

20000

25000

30000

10 20 40 80 100

TPM

Threads

DBT2

Large

Extra Large

High CPU

• Large: 2 x 2.4GHz E5645, 7.5GB RAM

• Extra Large: 4 x 2.4GHz E5645, 15GB RAM

• High CPU: 8 x 2.13GHz E5506, 7GB RAM


• Warm up Data Partitions (Amazon Elastic Block Store (Amazon EBS)

– One typically faces a ―first write‖ performance hit when initially

writing to new partitions

• Be Sure to Configure MySQL properly

– Simply running MySQL in the cloud isn't going to make it fast by

default

• Don't Forget Monitoring

– You can and should monitor your MySQL databases running in

the cloud

Best Practices


• Use MySQL Replication– MySQL replication is very important for scale out, load balancing,

and high availability

• Use Templates– You can save time and avoid risks of error by relying on

templates, such as the Oracle VM template for MySQL Enterprise Edition

• Use Good Security Practices– The cloud is no different from any other Internet-connected

device

Best Practices


• Mount Partitions with the noatime/nodiratime Options

– On Linux, should yield up to 10% better I/O performance

• Don’t Forget Backups

– MySQL Enterprise Backup performs online "Hot", non-blocking

backups of your MySQL databases

• Use Load Balancing

– You can use MySQL replication across multiple MySQL slaves to

improve read performance and use sharding to improve write

performance

Best Practices

KEY BUSINESS BENEFIT

MySQL has enabled CERN’s

database group to cut database

provisioning time and

administration costs

WHY MySQL?

“The Cloud model has allowed us

to deliver a self service platform to

our MySQL users, empowering

them while minimizing costs for

CERN.”

DATABASE AS A SERVICE AT CERNhttps://blogs.oracle.com/mysql/entry/with_its_mysql_database_as

CUSTOMER

CERN, the European Organization

for Nuclear Research, is one of the

world’s largest and most respected

centers for scientific research.

CHALLENGES

Better support the scientists who

selected MySQL as their

database. Empower users and

minimize complexity and costs of

CERN’s IT


MySQL Cluster deployed across 5

AWS regions gives global latency of

<1/3rd of a second, enabling real-time

fraud detection

WHY MySQL?

“MySQL Cluster enables users to get

the best of both world’s… agility of

NoSQL systems with the trust,

maturity & reliability of the SQL

model "

PROTECTING $100BN+ TRANSACTIONS https://blogs.oracle.com/mysql/entry/with_its_mysql_database_as

CUSTOMER

One of largest payment providers

on the internet, 30% year-on-year

growth

CHALLENGES

Protect financial transactions with

fraud detection system. Must be

real-time with global reach

servicing 100m+ users

GLOBAL SCALING OF MySQL CLUSTERAt Paypal

5 AWS Data Centers – US-E,US-W, TK, EU, AS


MySQL Cluster has allowed

PlayfulPlay to keep pace with the

rapid success of the game

WHY MYSQL?

“Being the number one Facebook

game in Latin America is a big

responsibility, and we rely on

Oracle’s products and MySQL

support for continued growth.”

EL CHAVO, POPULAR GAMING ON FACEBOOKhttps://blogs.oracle.com/MySQL/entry/mysql_cluster_powers_el_chavo

CUSTOMER

PlayfulPlay has created Latin

America’s #1 Facebook game

based on "El Chavo del 8".

CHALLENGES

Fast time to market and high

scalability at the lowest possible

cost were the leading development

priorities.


ZCS is an innovative application

that is cost-effectively delivered on

premise or as a Service (SaaS)

WHY MySQL?

―MySQL is a key component of

Zimbra Collaboration Suite —

enabling us to deliver a reliable

and scalable platform to millions of

users"

ZIMBRA’S SAAS SOLUTION

APPLICATION

The Zimbra Collaboration Suite

(ZCS) solution unifies email,

contacts, shared calendar, VoIP,

and online document authoring in

a rich browser-based interface


MySQL has enabled facebook to

grow to 1 billion users.

WHY MySQL?

―We are one of the largest MySQL

web sites in production. MySQL

has been a revolution for young

entrepreneurs.‖

SOCIAL NETWORKING AT FACEBOOK

APPLICATION

Facebook is a social networking

site that connects people with

friends and others who work, study

and live around them.

NEXT STEPS

Download the Guidehttp://bit.ly/V1FfhH

Try Out MySQL 5.6

http://dev.mysql.com/downloads/mysql/

Engage MySQL Consulting

http://www.mysql.com/consulting/

web - oracle · •node.js nosql api couples high performance, distributed ......

Documents