Hadoop学习十八:Hadoop-Hdfs DataXceiverServer源码其它操作及总结

一. 其它操作

public static final byte OP_REPLACE_BLOCK = (byte) 83;
  public static final byte OP_COPY_BLOCK = (byte) 84;
  public static final byte OP_BLOCK_CHECKSUM = (byte) 85;

二.REPLACE_BLOCK

 Hadoop学习十八:Hadoop-Hdfs DataXceiverServer源码其它操作及总结

  • Receive a block and write it to disk, it then notifies the namenode to remove the copy from the source.
  • sourceID:read delete hint.
  • DatanodeInfo proxySource:read proxy source.
  • 拷贝数据块和写数据块不同的是
    • 发起拷贝请求方不提供数据,只提供一个数据源proxySource,接收方需要自己同proxySource建立连接;写请求发起方提供数据。
    • 拷贝请求不需要链式流,因为只发生在两个节点上。

三.copyBlock

Hadoop学习十八:Hadoop-Hdfs DataXceiverServer源码其它操作及总结

  •  Read a block from the disk and then sends it to a destination.读取block,发生给请求方。

四.getBlockChecksum

Hadoop学习十八:Hadoop-Hdfs DataXceiverServer源码其它操作及总结

  • Get block checksum (MD5 of CRC32).

五.总结

  1. readBlock(DataInputStream in):从in里读取blockId和generationStamp,读取此block数据,发送给请求方(Socket s = ss.accept())。
  2. writeBlock(DataInputStream in):从in里读取blockId和generationStamp,将要从此block读数据; 读取DatanodeInfo targets[],组成链式流;读取block数据到targets[0],继续发生给下一个targets[1]...
  3. replaceBlock(DataInputStream in):从in里读取blockId和generationStamp,将要从此block读数据;读取proxySource,与proxySource建立连接开始读取block数据;读取sourceID,notifies the namenode to remove the copy from the source.
  4. copyBlock(DataInputStream in):从in里读取blockId和generationStamp,读取此block的数据发送给请求方。
  5. getBlockChecksum(DataInputStream in):从in里读取blockId和generationStamp,读取此block的检验值和MD5码。

相关推荐