www cubrid org(6)

7/30/2019 Www Cubrid Org(6)

1/5


2/5

1. When explicitly removing a node in the management tool, the data stored in the node is migrated to the remaining nodes and then the node is

removed.

2. When a new node is added, which is called bootstrapping, the added node communicates with the seed nodes to report that it has been added.

Based on the configuraons, the new node will bootstrap either to the range on the ring specified in the configuraons, or to the range near the

node with the most disk space used, that does not have another node bootstrapping into that range.

3. The data is migrated from that node to the new node.

4. The new node becomes available once the migraon process is complete.

Adding a New Node aer a Node Failure

The following is the result of adding a new node aer a node failure.

1. When a node is down, the data stored in that node is not migrated to other nodes, and the service is connued with two replicas . In other words,

no error will be returned, even when service requests are received during this me.

2. When a new node is added, the added node is ass igned to a specific area of the ring. However, bootstrapping is not performed.

Bootstrapping can be performed only when the number of data replicaons to be migrated is three .

3. The added node has no data in it, but it handles the request because it can provide service. If a read request is received at this me, the node

returns no data for the key. If the replica factor is 3 and the read consistency level is 1, 1/3 of the read requests may return no data. And if the

consistency level is set to the value of quorum, 1/6 of the read requests may return empty data. In short, no read consistency is assured unl the

fail-over has been recovered. At the actual level 1, the coordinang node is most likely to receive the response from the new node first. This is the

case because there is no I/O from the new node - it has no data. For this reason, a new node has a higher chance of returning empty data than the

exisng nodes.

4. When Read Repair-ing the new node by using the management tool, the node is built with replicaon data read from other nodes. The read

consistency is broken unl the Read Repair is complete.

Cassandra can provide service without error, even when a node fails. Although Cassandra shows good performance when wring data, this is not so

when reading data, because prolonged Rread Repair means prolonged data inconsistency. Therefore, to maintain read consistency during node failure,

he following method should be applied.

Set the read consistency level to 'all' and execute a read. In this case, the latest data from all replicas can be obtained.

If a read is failed, Cassandra retries that read. This is because a Rread Repair at the first read may be used as the source of the restored data at

the second read. However, this method assumes that Rread Repair is completed before the second read. (When the consistency level is low, the

read repair is performed in the background, in a thread separate from the read operaon processing thread.)

Failure Factors and Recovery Methods of HBase

HBase consists of several components, which are shown below (figure from HBase: The Definive Guide):

HRegionServertakes care of data distribuon, while HMastermonitors HRegionServer. HDFSstores and replicates data, and Zookeeperkeeps the

ocaon informaon of HMaster and elects a master. If redundancy is not established for each component, all of the components become SPoF.

HRegionServer can be detailed as follows: HRegionServer distributes data in a unit called a ' region.' A region is the result of dividing a big table where

he sorted data is stored by the sorng key range (like a tablet in a big table). The key range informaon of each region is stored in a separate region,

called the meta region. The region where the locaon of the meta region is stored is called the root region. In short, the region server stores a

hierarchical tree consisng of root regions, meta regions, and data regions. If a region server is down, the region that the failed server covers is

unavailable unl that region is assigned to another server. Therefore, service downme occurs unl the region is recovered.

I f this is the case, then how long wil l this downme be?

converted by Web2PDFConvert.com
http://www.web2pdfconvert.com/?ref=PDFhttp://www.web2pdfconvert.com/?ref=PDFhttp://ofps.oreilly.com/titles/9781449396107/architecture.html


3/5

Let's esmate the downme while examining the failover process.

Region Server Failure

When a failure occurs in a region server, the data is recovered through the steps described below:

1. HMaster detects fai lure and directs one of the other servers to perform the service of the failed server.

2. The directed HRegionServer first reads the WAL (Write Ahead Log) of the new region, and recovers the MemStore of that region.

3. Once MemStore is completely recovered, HMaster modifies the meta region that stores the locaon of the region to restart the service of the

region.

4. The data of the region stored in the disk will be freshly recovered by HD FS.

n the end, the recovery me required is as long as the me it takes to detect a failure, read the log and create a new region. Since the server assigned

o the recovered region can access the data file in the HDFS, no data migraon occurs on the HDFS. Therefore, the downme is not significantly long.

HDFS Failure

The HDFS consists of one name node and several data nodes. Here, the name node is the node that stores meta data. So when this is down, service

ailure will occur. However, if one of the data nodes is down, no service failure will occur because the data has the replica. But the data stored in the

ailed data node will be built by one of the other nodes to recover the replica factor to normal (recovery). At this me, a huge data replicaon may

occur, slowing down any read requests from the service or applicaon. This is because the disk I/O for read is affected by the data replicaon.

Replicaon and Failover of MongoDB

MongoDB asynchronously replicates data from the master to the slave. The advantages of asynchronous replicaon are that it does not degrade

performance of the master and the service performance does not degrade when a slave is added. However, the data will be lost when a failure occurs,

because the data is inconsistent.

MongoDB recovers from failure in a similar way as the HA of DBMS. It elects a master when a failure occurs. Let's take a look at the two scenarios

below.

Node Failure

Configure three nodes with one master and two slave nodes. Stop the master node. One of the two slaves will be automacally elected as a master.The me it takes to elect a new master when a failure occurs is a couple of seconds. This downme is not that long. However, once the nodes are

configured as a master and slaves and then the master is down again, no master is elected again.

Adding a Node

Enter data in the master. Assume that the size of the data is 5 GB, which is smaller than the memory size. Then, add a new slave to the master. In this

case, adding a new slave does not degrade the performance of the master. It takes several minutes for the added slave to replicate all data.

n MongoDB, the degradaon of performance due to a failed or added node is minimal. However, if a failure occurs in the nodes while the replicas of

he master and the slave are inconsistent, the data that has not been replicated by the slave may be lost. In MongoDB, the master writes the operaon

history to the Oplog log in the local server and then the slave reads the log and stores it in its database to replicate. If a failure occurs while the slave

has not yet finished reading the log of the master, the unread data will be lost. In addion, if the master log is full while the slave has not finishedeplicang the content, all the data in the master is read and stored in the slave, rather than being replicated in the log. This is called data sync. If a

ailure occurs in the master in the situaon above, a large amount of data will be lost.

Conclusion

So far, I have reviewed failovers of C assandra, H Base and MongoDB.

Cassandra offers high availability for Write operaons. However, it takes a long me to recover data from a failure. This is because Cassandra

denfies all the data to recover, and then reads and writes the latest version of each data. Also since it responds to service requests while the added

node is sll in the process of data recovery, an incorrect read result may be returned. In addion, Rread Repair is executed twice when reading the

data to recover. Although it provides hinted handoff for operaon execuon failure as well as for node failure, an incorrect result may be returned if the

data to be recovered is read first. Therefore, if the consistency level is not raised, it cannot be used for the services that require read processing.

Because of its configuraon, HBase has many factors that may cause a failure. However, while Cassandra has to recover data during a failure, HBase

does not need to recover data unless a failure occurs in the HDFS. This gives HBase a short downme. The downme during an HDFS failure is not

hat long either. The read performance may be hit while recovering data, but the data consistency is maintained. In this way, higher availability is

offered if the SPoF part can become redundant.

http://www.web2pdfconvert.com/?ref=PDFhttp://www.web2pdfconvert.com/?ref=PDF


4/5

MongoDB provides automac failover and has a short downme. However, its asynchronous replicaon method may cause data loss aer a failover.

Thus, before choosing the database soluon that will suit your purpose, you should consider these characteriscs of each product.

For reference, CUBRID RDBMS provides synchronous High-Availability for data consistency which results in no data loss, though it lacks the

performance of NoSQL soluons.

See also

The Story behind LINE App Development

Dev Plaorm If you have a smartphone, you must be using LINE, KakaoTalk, or Whatsapp messaging app, if not all of them. But unli...

9 months ago by Esen Sagynov 0 10817

NoSQL Benchmarking

Dev Plaorm NoSQL is the talk of the town. And we have already covered what it is for in one of our previous blogs. Today I would ...

2 years ago by Hye Jeong Lee 16 43183

Log Analysis System Using Hadoop and MongoDB

Dev Plaorm I have been searching for a way to process the exponenally increasing service logs at a low cost. For the past several mo...

2 years ago by Jeong Hyun Lee 5 25530

What is NoSQL for?

Dev Plaorm In a social network service where many people form relaonships, a connecon becomes the informaon. A fundamental method ...

2 years ago by Kyu Jae Lee 5 14605

Overview of New High-Availability Features in CUBRID 8.4.0

CUBRID Life Perhaps, the HA (High-Availability) feature is one of the most important in any fully-featured database management system. It ref...

2 years ago by Esen Sagynov 2 14125

http://www.web2pdfconvert.com/?ref=PDFhttp://www.web2pdfconvert.com/?ref=PDFhttp://www.cubrid.org/blog/cubrid-life/overview-of-new-high-availability-features-in-cubrid-3-2/#commentshttp://www.cubrid.org/blog/categories/cubrid-life/http://www.cubrid.org/blog/cubrid-life/overview-of-new-high-availability-features-in-cubrid-3-2/http://www.cubrid.org/blog/cubrid-life/overview-of-new-high-availability-features-in-cubrid-3-2/http://www.cubrid.org/blog/dev-platform/what-is-nosql-for/#commentshttp://www.cubrid.org/blog/categories/dev-platform/http://www.cubrid.org/blog/dev-platform/what-is-nosql-for/http://www.cubrid.org/blog/dev-platform/what-is-nosql-for/http://www.cubrid.org/blog/dev-platform/log-analysis-system-using-hadoop-and-mongodb/#commentshttp://www.cubrid.org/blog/categories/dev-platform/http://www.cubrid.org/blog/dev-platform/log-analysis-system-using-hadoop-and-mongodb/http://www.cubrid.org/blog/dev-platform/log-analysis-system-using-hadoop-and-mongodb/http://www.cubrid.org/blog/dev-platform/nosql-benchmarking/#commentshttp://www.cubrid.org/blog/categories/dev-platform/http://www.cubrid.org/blog/dev-platform/nosql-benchmarking/http://www.cubrid.org/blog/dev-platform/nosql-benchmarking/http://www.cubrid.org/blog/dev-platform/the-story-behind-line-app-development/#commentshttp://www.cubrid.org/blog/categories/dev-platform/http://www.cubrid.org/blog/dev-platform/the-story-behind-line-app-development/http://www.cubrid.org/blog/dev-platform/the-story-behind-line-app-development/http://blog.cubrid.org/cubrid-life/overview-of-new-high-availability-features-in-cubrid-3-2/


5/5

About CUBRID | Contact us |

2012 CUB RI D.org. All rights reserved.

0 comments

What's this? ALSO ONCUBRID OPEN SOURCE DATABASE COMMUNITY

Understanding TCP/IP Network Stack The Principles of Java ApplicationPerformance Tuning

Leave a message...

Discussion Community Share

comment 12 days ago

1 comment a month ago

0

t d b W b2PDFC t
http://www.web2pdfconvert.com/?ref=PDFhttp://www.web2pdfconvert.com/?ref=PDFhttp://www.cubrid.org/contacthttp://www.cubrid.org/about

www cubrid org(6)

Documents