Amazon ec2 cassandra引导失败，带“0”；出现流式处理错误"；

我在aws ec2中有一个cassandra集群。集群现在有7个节点。我想向这个集群添加一个新节点。但引导程序出现错误。引导程序无法成功完成

我犯了很多错误：

ERROR [STREAM-IN-/172.31.20.223] 2016-01-01 15:37:30,941 StreamSession.java:524 - [Stream #b1cc3600-b054-11e5-80c4-21d1c7c11a01] Streaming error occurred:
java.nio.channels.ClosedChannelException: null
        at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:257) ~[na:1.8.0_66]
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:300) ~[na:1.8.0_66]
        at org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:53) ~[apache-cassandra-2.2.4.jar:2.2.4]

还有这样的错误：

WARN  [STREAM-IN-/172.31.20.223] 2016-01-01 15:37:30,941 StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80
c4-21d1c7c11a01] Retrying for following error
java.lang.RuntimeException: Last written key DecoratedKey(-8466153190082758358, 000000000000356a) >= current key
 DecoratedKey(-9223372036854775808, ) writing into /data_lvm/cassandra/data/feeds/inbox-eb873af0a19711e5ade0432b
31304f95/tmp-la-309-big-Data.db

WARN  [STREAM-IN-/172.31.8.188] 2016-01-01 15:25:05,001 StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80c4-21d1c7c11a01] Retrying for following error
java.lang.IllegalArgumentException: Not enough bytes
        at org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:362) ~[apache-cassandra-2.2.4.jar:2.2.4]
        at org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNa
meType.java:98) ~[apache-cassandra-2.2.4.jar:2.2.4]
        at org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381) ~[apache-cassandra-2.2.4.jar:2.2.4]

WARN  [STREAM-IN-/172.31.4.135] 2016-01-01 15:23:44,228 StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80c
4-21d1c7c11a01] Retrying for following error
java.lang.ArrayIndexOutOfBoundsException: null
ERROR [Thread-571] 2016-01-01 15:23:44,228 CassandraDaemon.java:185 - Exception in thread Thread[Thread-571,5,ma
in]
java.lang.RuntimeException: java.lang.InterruptedException
        at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.jar:na]
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32) ~[apache-cassandra-2.2.4.jar:
2.2.4]
        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66]
Caused by: java.lang.InterruptedException: null
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer
.java:1220) ~[na:1.8.0_66]
        at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[na:1.8.0_66]
        at java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:350) ~[na:1.8.0_66]
        at org.apache.cassandra.streaming.compress.CompressedInputStream$Reader.runMayThrow(CompressedInputStrea
m.java:176) ~[apache-cassandra-2.2.4.jar:2.2.4]
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.2.4.jar:
2.2.4]

还有这样的错误：

WARN  [STREAM-IN-/172.31.20.223] 2016-01-01 15:37:30,941 StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80
c4-21d1c7c11a01] Retrying for following error
java.lang.RuntimeException: Last written key DecoratedKey(-8466153190082758358, 000000000000356a) >= current key
 DecoratedKey(-9223372036854775808, ) writing into /data_lvm/cassandra/data/feeds/inbox-eb873af0a19711e5ade0432b
31304f95/tmp-la-309-big-Data.db

WARN  [STREAM-IN-/172.31.8.188] 2016-01-01 15:25:05,001 StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80c4-21d1c7c11a01] Retrying for following error
java.lang.IllegalArgumentException: Not enough bytes
        at org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:362) ~[apache-cassandra-2.2.4.jar:2.2.4]
        at org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNa
meType.java:98) ~[apache-cassandra-2.2.4.jar:2.2.4]
        at org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381) ~[apache-cassandra-2.2.4.jar:2.2.4]

WARN  [STREAM-IN-/172.31.4.135] 2016-01-01 15:23:44,228 StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80c
4-21d1c7c11a01] Retrying for following error
java.lang.ArrayIndexOutOfBoundsException: null
ERROR [Thread-571] 2016-01-01 15:23:44,228 CassandraDaemon.java:185 - Exception in thread Thread[Thread-571,5,ma
in]
java.lang.RuntimeException: java.lang.InterruptedException
        at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.jar:na]
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32) ~[apache-cassandra-2.2.4.jar:
2.2.4]
        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66]
Caused by: java.lang.InterruptedException: null
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer
.java:1220) ~[na:1.8.0_66]
        at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[na:1.8.0_66]
        at java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:350) ~[na:1.8.0_66]
        at org.apache.cassandra.streaming.compress.CompressedInputStream$Reader.runMayThrow(CompressedInputStrea
m.java:176) ~[apache-cassandra-2.2.4.jar:2.2.4]
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.2.4.jar:
2.2.4]

还有这样的错误：

WARN  [STREAM-IN-/172.31.20.223] 2016-01-01 15:37:30,941 StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80
c4-21d1c7c11a01] Retrying for following error
java.lang.RuntimeException: Last written key DecoratedKey(-8466153190082758358, 000000000000356a) >= current key
 DecoratedKey(-9223372036854775808, ) writing into /data_lvm/cassandra/data/feeds/inbox-eb873af0a19711e5ade0432b
31304f95/tmp-la-309-big-Data.db

WARN  [STREAM-IN-/172.31.8.188] 2016-01-01 15:25:05,001 StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80c4-21d1c7c11a01] Retrying for following error
java.lang.IllegalArgumentException: Not enough bytes
        at org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:362) ~[apache-cassandra-2.2.4.jar:2.2.4]
        at org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNa
meType.java:98) ~[apache-cassandra-2.2.4.jar:2.2.4]
        at org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381) ~[apache-cassandra-2.2.4.jar:2.2.4]

WARN  [STREAM-IN-/172.31.4.135] 2016-01-01 15:23:44,228 StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80c
4-21d1c7c11a01] Retrying for following error
java.lang.ArrayIndexOutOfBoundsException: null
ERROR [Thread-571] 2016-01-01 15:23:44,228 CassandraDaemon.java:185 - Exception in thread Thread[Thread-571,5,ma
in]
java.lang.RuntimeException: java.lang.InterruptedException
        at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.jar:na]
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32) ~[apache-cassandra-2.2.4.jar:
2.2.4]
        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66]
Caused by: java.lang.InterruptedException: null
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer
.java:1220) ~[na:1.8.0_66]
        at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[na:1.8.0_66]
        at java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:350) ~[na:1.8.0_66]
        at org.apache.cassandra.streaming.compress.CompressedInputStream$Reader.runMayThrow(CompressedInputStrea
m.java:176) ~[apache-cassandra-2.2.4.jar:2.2.4]
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.2.4.jar:
2.2.4]

我尝试了很多方法来引导，比如清理所有节点，清理所有节点，修复所有节点。但它仍然错误地失败了

有人能帮忙找出问题所在吗

我用6个节点启动集群，并成功添加了第7个节点。但当我尝试添加第8个节点时，它失败了

每个节点大约有500G数据

---

我使用的DataStax Community Edition 2.2.4似乎是2.2.4版在向现有集群添加节点时出现了问题

签出此jira：

我也遇到了同样的问题，使用了修补过的罐子并开始工作

修复程序将在2.2.5版本上正式发布。但是你可以同时使用这条路径