datanode communicate with namenode
Post on 15-Jul-2015
510 Views
Preview:
TRANSCRIPT
NameNode与DataNode通信
Protocol
• register: handshake,register when startup
• heartbeat: liveness,capacity,used,remaining. short time 3s default
• blockreport: diff. long time 6h
Protocol
Inside NameNode
• FSNameSystem : INode,Metadata
• BlockManager:Block,DataNode,Heartbeat
• 两者关系不是割裂，紧密联系
Inside NameNode
• DataNode在NameNode中对应的对象
• DataNode ——> DataNodeDescriptor
• Disk ——> DataNodeStorageInfo
• Block在DataNodeStorageInfo中以List存储，头结点为blockList
INode
• Dir —> INodeDirectory只有meta，没有实际存储
• File —> INodeFile Blocks Replica
INodeFile• INodeFile存储BlockInfo
• BlockInfo存储triplet : save memory, block report
Heartbeat
• the function of heartbeat
BlockReport
• The function of BlockReport
BlockReport• toInvalidate:不在BlockMap中，该Block已经被删除
• toUC:在BlockMap中，状态为UC
• toCorrupt:在BlockMap中，状态为Corruput
• toAdd:在BlockMap中，triplet没有该节点的该磁盘
• toRemove:没有汇报，丢失的Block
BlockReport流程
BlockReport
BlockReport
top related