Hadoop常见错误及解决办法汇总2

3，错误：java.net.SocketTimeoutException: 480000 millistimeout while waiting for channel to be ready for write4 y# J0 W+ W- l#F o% A4 I
2014-05-06 14:28:09,386 ERRORorg.apache.hadoop.hdfs.server.datanode.DataNode: hadoop-datanode1:50010ataXceiver error processing READ_BLOCKoperation src: /192.168.1.191:48854 dest: /192.168.1.191:50010
java.net.SocketTimeoutException: 480000 millistimeout while waiting for channel to be ready for write. ch :java.nio.channels.SocketChannel[connected local=/192.168.1.191:50010remote=/192.168.1.191:48854]$i w. V4 w9 o& p
atorg.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246) Y$ @: l5 Z9 y L
atorg.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:172)- s$ K8 W) R; |
atorg.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:220)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(BlockSender.java:546)
atorg.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:710)+ N# ]! t7 W8 Q& y
atorg.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:340)
atorg.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:101): y, ~, W* P) K* \1 J
atorg.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:65)
atorg.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:221)1 q- a' m3 A5 f# ` o8 ^
atjava.lang.Thread.run(Thread.java:722)
原因：IO超时
解决方法：4 @3 A( S/ l3 z5 g
修改hadoop配置文件hdfs-site.xml，增加dfs.datanode.socket.write.timeout和dfs.socket.timeout两个属性的设置。0 M9 a' R; h9 J; T' U& u
<property>0 ]2 l+ T8 }3 ]
<name>dfs.datanode.socket.write.timeout</name>" |0 C% k: u1 q. o9 R. O
<value>6000000</value>' p F! j6 g& x! J U
</property>
<property>6 h- q6 G) k/ Z$ S g" V
<name>dfs.socket.timeout</name>1 Z3 N1 Q6 A: g
<value>6000000</value>
</property>
注意: 超时上限值以毫秒为单位。0表示无限制。
4，错误：DataXceiver errorprocessing WRITE_BLOCK operation
2014-05-0615:21:30,378 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:hadoop-datanode1:50010ataXceivererror processing WRITE_BLOCK operation src: /192.168.1.193:34147dest: /192.168.1.191:500100 d3 F/ x) v" t- d/ `1 V' f
java.io.IOException: Premature EOF from inputStream; T4 W) h) A0 Z( o; g
at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:194)1 E7 q" J6 V& D2 ]3 J8 u5 r# Y
atorg.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)9 B/ `& K4 u3 |" N" o; }
atorg.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134), X4 i5 y8 ?, E, F7 a
atorg.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)% M0 O4 f& k. H/ H. \/ a2 j0 I
at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:435)
atorg.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:693)
atorg.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:569)
atorg.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:115)
atorg.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:68)
atorg.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:221); Q& ~$ U/ I' q& ?2 _
at java.lang.Thread.run(Thread.java:722)
原因：文件操作超租期，实际上就是data stream操作过程中文件被删掉了。; v! @( [( o# r# j4 s
$ p/ h6 \+ {9 E+ e
解决办法：
修改hdfs-site.xml （针对2.x版本，1.x版本属性名应该是：dfs.datanode.max.xcievers）：
<property>
<name>dfs.datanode.max.transfer.threads</name> % B |& `$ |. p, S) j7 M; s7 o
<value>8192</value> q G4 `1 b$ |
</property>
拷贝到各datanode节点并重启datanode即可" X%