Java 卡桑德拉·赫克托-不可用例外

Java 卡桑德拉·赫克托-不可用例外,java,cassandra,hector,Java,Cassandra,Hector,我试图使用Hector插入记录,但有时会出现以下错误: me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be enough replicas present to handle consistency level. at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorI

我试图使用Hector插入记录,但有时会出现以下错误:

me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be enough replicas present to handle consistency level.
    at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:59)
    at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:264)
    at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:113)
    at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:243)
    at me.prettyprint.cassandra.service.template.AbstractColumnFamilyTemplate.executeBatch(AbstractColumnFamilyTemplate.java:115)
    at me.prettyprint.cassandra.service.template.AbstractColumnFamilyTemplate.executeIfNotBatched(AbstractColumnFamilyTemplate.java:163)
    at me.prettyprint.cassandra.service.template.ColumnFamilyTemplate.update(ColumnFamilyTemplate.java:69)
    at ustocassandra.USToCassandraHector.consumer(USToCassandraHector.java:271)
    at ustocassandra.USToCassandraHector.access$100(USToCassandraHector.java:41)
    at ustocassandra.USToCassandraHector$2.run(USToCassandraHector.java:71)
    at java.lang.Thread.run(Thread.java:724)
Caused by: UnavailableException()
    at org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:20841)
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
    at org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:964)
    at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:950)
    at me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:246)
    at me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:243)
    at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:104)
    at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258)
    ... 9 more
我知道通常的解释是上面没有足够的节点,但不是这种情况。我的所有节点都已启动:

./nodetool ring
Note: Ownership information does not include topology; for complete information, specify a keyspace

Datacenter: DC1
==========
Address         Rack        Status State   Load            Owns                Token
                                                                               4611686018427388000
172.16.217.222  RAC1        Up     Normal  353.36 MB       25.00%              -9223372036854775808
172.16.217.223  RAC2        Up     Normal  180.84 MB       25.00%              -4611686018427388000
172.16.217.224  RAC3        Up     Normal  260.34 MB       25.00%              -2
172.16.217.225  RAC4        Up     Normal  222.71 MB       25.00%              4611686018427388000
我插入了20个线程的记录(也许我应该少用一点?据我所知,在这种情况下,错误会过载,而不是不可用)。我使用的是一个写一致性。我正在使用AutoDiscoveryStartup和LeastActiveBalancing策略。复制因子为2

我使用的是Cassandra 1.2.8(我试过使用2.0,结果是一样的)

错误从一开始就没有发生。在出现错误之前,我通常会插入大约200万条记录。我的代码设置为发生错误时重试。经过几十次重试后,插入通常会成功。在那之后,它对数百万次插入再次正常工作,然后我再次得到错误,循环继续

可能是因为我设置了gc_grace=60?无论如何,我不是每60秒就收到一次错误,所以我不认为这是原因

关于这个错误的原因,你能给我一些建议吗?我该怎么办

编辑:

“nodetool tpstats”说我删除了一些消息:

Message type           Dropped
RANGE_SLICE                  0
READ_REPAIR                  0
BINARY                       0
READ                         0
MUTATION                    11
_TRACE                       0
我在日志文件中看到以下警告:

 WARN [ScheduledTasks:1] 2013-09-30 09:20:16,633 GCInspector.java (line 136) Heap is 0.853986836999536 full.  You may need to reduce memtable and/or cache sizes.  Cassandra is now reducing cache sizes to free up memory.  Adjust reduce_cache_sizes_at threshold in cassandra.yaml if you don't want Cassandra to do this automatically
 WARN [ScheduledTasks:1] 2013-09-30 09:20:16,634 AutoSavingCache.java (line 185) Reducing KeyCache capacity from 1073741824 to 724 to reduce memory pressure
 WARN [ScheduledTasks:1] 2013-09-30 09:20:16,634 GCInspector.java (line 142) Heap is 0.853986836999536 full.  You may need to reduce memtable and/or cache sizes.  Cassandra will now flush up to the two largest memtables to free up memory.  Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do this automatically
 WARN [ScheduledTasks:1] 2013-09-30 09:20:16,634 StorageService.java (line 3618) Flushing CFS(Keyspace='us', ColumnFamily='my_cf') to relieve memory pressure
这正是Hector抛出不可用异常的时间。所以,这可能是一个与记忆有关的问题。
我想我会尝试一下警告所说的:减小memtable的大小

这可能是因为您的服务器超载,所以某些节点没有响应。没有
重载异常
(重载节点看起来就像不可用的节点)

您应该检查您的Cassandra日志-是否存在堆已满的警告?
nodetool tpstats
中是否列出了丢弃的消息?服务器上的CPU负载是多少?

谢谢,似乎(从日志文件和“tpstats”中)我的节点确实超载了。我已经编辑了我的问题。如果您有其他建议,欢迎您提出:)