Cassandra bulkloader在数据上传期间偶尔挂起

Cassandra bulkloader在数据上传期间偶尔挂起,cassandra,bulkloader,Cassandra,Bulkloader,我使用Cassandra bulkloader将数据上载到集群(用于上载数据的节点不是集群的一部分)。我观察到,在极少数情况下,散装过程会挂起 数据上传。挂起是不可预测的,如果下次运行,相同的数据上载将成功 集群详细信息:20节点Cassandra 2.1.11 EC2集群。 散货装载机节点:2.1.2卡桑德拉 下面是一个此类挂起的客户端和服务器日志 客户端日志: 016-01-16 10:40:20,466 INFO [STREAM-OUT-ip-172-31-44-254.us-west-2

我使用Cassandra bulkloader将数据上载到集群(用于上载数据的节点不是集群的一部分)。我观察到,在极少数情况下,散装过程会挂起 数据上传。挂起是不可预测的,如果下次运行,相同的数据上载将成功

集群详细信息:20节点Cassandra 2.1.11 EC2集群。 散货装载机节点:2.1.2卡桑德拉

下面是一个此类挂起的客户端和服务器日志

客户端日志:

016-01-16 10:40:20,466 INFO [STREAM-OUT-ip-172-31-44-254.us-west-2.compute.internal./172.31.44.254] org.apache.hadoop.hdfs.DFSClient: Seek offset: 141643669
2016-01-16 10:40:20,525 INFO [STREAM-OUT-ip-172-31-44-254.us-west-2.compute.internal./172.31.44.254] org.apache.hadoop.hdfs.DFSClient: Seek offset: 152129429
2016-01-16 10:40:21,282 INFO [STREAM-IN-ip-172-31-32-55.us-west-2.compute.internal./172.31.32.55] org.apache.cassandra.streaming.StreamResultFuture: Stream #49f7ad10-bc67-11e5-a004-e70ac8f0e359 Session with ip-172-31-32-55.us-west-2.compute.internal./172.31.32.55 is complete
2016-01-16 10:40:21,368 ERROR [STREAM-IN-ip-172-31-44-254.us-west-2.compute.internal./172.31.44.254] org.apache.cassandra.streaming.StreamSession: Stream #49f7ad10-bc67-11e5-a004-e70ac8f0e359 Streaming error occurred
java.lang.NullPointerException
at org.apache.cassandra.streaming.StreamSession.received(StreamSession.java:547)
at org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:429)
at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:251)
at java.lang.Thread.run(Thread.java:744)
2016-01-16 10:40:21,369 INFO [STREAM-IN-ip-172-31-44-254.us-west-2.compute.internal./172.31.44.254] org.apache.cassandra.streaming.StreamResultFuture: Stream #49f7ad10-bc67-11e5-a004-e70ac8f0e359 Session with ip-172-31-44-254.us-west-2.compute.internal./172.31.44.254 is complete
2016-01-16 10:40:21,780 INFO [STREAM-IN-ip-172-31-32-51.us-west-2.compute.internal./172.31.32.51] org.apache.cassandra.streaming.StreamResultFuture: Stream #49f7ad10-bc67-11e5-a004-e70ac8f0e359 Session with ip-172-31-32-51.us-west-2.compute.internal./172.31.32.51 is complete
2016-01-16 10:40:31,841 INFO [STREAM-IN-ip-172-31-32-53.us-west-2.compute.internal./172.31.32.53] org.apache.cassandra.streaming.StreamResultFuture: Stream #49f7ad10-bc67-11e5-a004-e70ac8f0e359 Session with ip-172-31-32-53.us-west-2.compute.internal./172.31.32.53 is complete
2016-01-16 10:40:34,527 INFO [STREAM-IN-ip-172-31-44-255.us-west-2.compute.internal./172.31.44.255] org.apache.cassandra.streaming.StreamResultFuture: Stream #49f7ad10-bc67-11e5-a004-e70ac8f0e359 Session with ip-172-31-44-255.us-west-2.compute.internal./172.31.44.255 is complete
服务器日志(流失败的节点):

INFO[STREAM-INIT-/172.31.37.149:51647]2016-01-16 10:40:21360 StreamResultFuture.java:109-STREAM"6cbdf570-bc67-11e5-b829-cf763a9770ff ID"0为批量加载创建新的流计划
信息[STREAM-INIT-/172.31.37.149:51647]2016-01-16 10:40:21360 StreamResultFuture.java:116-STREAM#6cbdf570-bc67-11e5-b829-cf763a9770ff,ID#0收到批量加载的流媒体计划
信息[STREAM-INIT-/172.31.37.149:51652]2016-01-16 10:40:21362 StreamResultFuture.java:116-STREAM#6cbdf570-bc67-11e5-b829-cf763a9770ff,ID#0收到批量加载的流媒体计划
信息[STREAM-IN-/172.31.37.148]2016-01-16 10:40:21368 StreamResultFuture.java:180-STREAM#49f7ad10-bc67-11e5-a004-e70ac8f0e359与/172.31.37.148的会话已完成

警告[STREAM-IN-/172.31.37.148]2016-01-16 10:40:21368 StreamResultFuture.java:207-STREAM#49f7ad10-bc67-11e5-a004-e70ac8f0e359 STREAM失败2.1中有很多关于流的错误。如果这些错误仍然发生,您应该更新到最新版本并打开一个。谢谢。我将以2.1.13作为我的主要目标来尝试这个实验。但是,我很难从2.1.2升级bulkloader节点。
INFO [STREAM-INIT-/172.31.37.149:51647] 2016-01-16 10:40:21,360 StreamResultFuture.java:109 - Stream #6cbdf570-bc67-11e5-b829-cf763a9770ff ID#0 Creating new streaming plan for Bulk Load
INFO [STREAM-INIT-/172.31.37.149:51647] 2016-01-16 10:40:21,360 StreamResultFuture.java:116 - Stream #6cbdf570-bc67-11e5-b829-cf763a9770ff, ID#0 Received streaming plan for Bulk Load
INFO [STREAM-INIT-/172.31.37.149:51652] 2016-01-16 10:40:21,362 StreamResultFuture.java:116 - Stream #6cbdf570-bc67-11e5-b829-cf763a9770ff, ID#0 Received streaming plan for Bulk Load
INFO [STREAM-IN-/172.31.37.148] 2016-01-16 10:40:21,368 StreamResultFuture.java:180 - Stream #49f7ad10-bc67-11e5-a004-e70ac8f0e359 Session with /172.31.37.148 is complete
WARN [STREAM-IN-/172.31.37.148] 2016-01-16 10:40:21,368 StreamResultFuture.java:207 - Stream #49f7ad10-bc67-11e5-a004-e70ac8f0e359 Stream failed <<<<<<<<<<<<< Last entry in log for the stream id
INFO [STREAM-IN-/172.31.37.149] 2016-01-16 10:40:21,407 StreamResultFuture.java:166 - Stream #6cbdf570-bc67-11e5-b829-cf763a9770ff ID#0 Prepare completed. Receiving 1 files(29350007 bytes), sending 0 files(0 bytes)
INFO [StreamReceiveTask:230] 2016-01-16 10:40:22,159 StreamResultFuture.java:180 - Stream #507b0100-bc67-11e5-8fa7-e70ac8f0e359 Session with /172.31.37.148 is complete
INFO [StreamReceiveTask:230] 2016-01-16 10:40:22,159 StreamResultFuture.java:212 - Stream #507b0100-bc67-11e5-8fa7-e70ac8f0e359 All sessions completed
INFO [StreamReceiveTask:242] 2016-01-16 10:40:22,204 StreamResultFuture.java:180 - Stream #56ddad90-bc67-11e5-ae72-079f597e5002 Session with /172.31.37.150 is complete
INFO [StreamReceiveTask:242] 2016-01-16 10:40:22,205 StreamResultFuture.java:212 - Stream #56ddad90-bc67-11e5-ae72-079f597e5002 All sessions completed