Hbase批量上载问题-独立模式

我对Hbase非常陌生。我在独立模式下使用Hbase，并使用google n-grams开发应用程序。我在AmazonEC2实例中使用一个ubuntu盒

我正在通过批处理api使用我的程序进行一百万次写操作

        HTable table = new HTable(conf, tableName);
        conf.set("zookeeper.session.timeout", "180000");
        conf.set("hbase.rpc.timeout", "180000");
        conf.set("hbase.regionserver.logroll.period", "180000");
        conf.set("hbase.hregion.max.filesize","51200");
        table.setAutoFlush(false);
        table.batch(iterateList);
        table.flushCommits();

当我对1M行执行此操作时，我将按以下顺序获得以下异常：

    12/08/28 14:17:13 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x1396d9653bd0004, likely server has closed socket, closing socket connection and attempting reconnect
    12/08/28 14:17:13 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x1396d9653bd0003, likely server has closed socket, closing socket connection and attempting reconnect
    12/08/28 14:17:13 WARN client.HConnectionManager$HConnectionImplementation: Failed all from region=googlegrams,,1346160551634.abc8baf335dee8485d64a6c8b989d6b0., hostname=ip-10-80-74-132.ec2.internal, port=45549

java.util.concurrent.ExecutionException:java.io.IOException:调用ip-10-80-74-132.ec2.internal/10.80.74.132:45549在本地异常上失败：java.io.EOFEException

然后

  12/08/28 14:17:15 WARN zookeeper.ClientCnxn: Session 0x1396d9653bd0004
  for server null, unexpected error, closing socket connection and attempting reconnect
  java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)

我试过的东西很少是， 1.ubuntu的ulimit和nproc设置更改 2.Hbase配置，如zookeeper超时、rpc超时日志滚动周期和hregion文件大小

请帮忙

干杯，

---

Dwarak

尝试增加hbase.client.lease.period