转载

[HADOOP] Unknown error encountered while tailing edits. Shutting down standby NN.

前段时间，standby的NN挂掉了，并且怎么起也起不来，如下日志:

2016-01-03 14:04:19,293 FATAL org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer: Unknown error encountered while tailing edits. Shutting down standby NN. java.io.IOException: Failed to apply edit log operation ReassignLeaseOp [leaseHolder=DFSClient_NONMAPREDUCE_854707399_1, path=/tmp/jrdw/kafka2hdfs/log_mobile_gateway-21-1443245603647--6536501137915724876, newHolde r=HDFS_NameNode, opCode=OP_REASSIGN_LEASE, txid=20790808505]: error File is not under construction: /tmp/jrdw/kafka2hdfs/log_mobile_gateway-21-1443245603647--6536501137915724876        at org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:205)        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:112)        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:771)        at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:227)        at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:321)        at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$200(EditLogTailer.java:279)        at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:296)        at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:456)        at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:292)

刚开始怀疑是不是editlog的下载有问题，后来发现editlog是可以解析出来的，但是不排除editlog的顺序存在bug