NameNode与DataNode通信

Protocol

• register: handshake,register when startup

• heartbeat: liveness,capacity,used,remaining. short time 3s default

• blockreport: diff. long time 6h

Protocol

Inside NameNode

• FSNameSystem : INode,Metadata

• BlockManager:Block,DataNode,Heartbeat

• 两者关系不是割裂，紧密联系

Inside NameNode

• DataNode在NameNode中对应的对象

• DataNode ——> DataNodeDescriptor

• Disk ——> DataNodeStorageInfo

• Block在DataNodeStorageInfo中以List存储，头结点为blockList

INode

• Dir —> INodeDirectory只有meta，没有实际存储

• File —> INodeFile Blocks Replica

INodeFile• INodeFile存储BlockInfo

• BlockInfo存储triplet : save memory, block report

Heartbeat

• the function of heartbeat

BlockReport

• The function of BlockReport

BlockReport• toInvalidate:不在BlockMap中，该Block已经被删除

• toUC:在BlockMap中，状态为UC

• toCorrupt:在BlockMap中，状态为Corruput

• toAdd:在BlockMap中，triplet没有该节点的该磁盘

• toRemove:没有汇报，丢失的Block

BlockReport流程

BlockReport

BlockReport