Hbase HDFS集成-Hbase主机未启动
我已经在虚拟机上配置了两个并行运行的linux发行版集群节点 在两个linux发行版中,/etc/hosts文件的内容如下Hbase HDFS集成-Hbase主机未启动,hbase,hdfs,Hbase,Hdfs,我已经在虚拟机上配置了两个并行运行的linux发行版集群节点 在两个linux发行版中,/etc/hosts文件的内容如下 hduser@ubuntu-master:~$ cat /etc/hosts 192.168.56.103 Ubuntu-Master master 192.168.56.102 LinuxMint-Slave slave 10.33.136.219 inkod2lp00100.techmahindra.com inkod2lp00100 hduser@Linu
hduser@ubuntu-master:~$ cat /etc/hosts
192.168.56.103 Ubuntu-Master master
192.168.56.102 LinuxMint-Slave slave
10.33.136.219 inkod2lp00100.techmahindra.com inkod2lp00100
hduser@LinuxMint-Slave ~ $ cat /etc/hosts
192.168.56.103 Ubuntu-Master master
192.168.56.102 LinuxMint-Slave slave
10.33.136.219 inkod2lp00100.techmahindra.com inkod2lp00100
两个linux发行版中hbase-site.xml(Location-/usr/local/hbase/conf)的内容如下-
hduser@ubuntu-master:~$ cat /usr/local/hbase/conf/hbase-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.tmp.dir</name>
<value>file:///usr/local/hbase/hbasetmp/hbase-${user.name}</value>
</property>
<property>
<name>hbase.master</name>
<value>Ubuntu-Master:16000</value>
<description>The host and port that the HBase master runs at.</description>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://Ubuntu-Master:54310/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>Ubuntu-Master,LinuxMint-Slave</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>file:///usr/local/hbase/zookeeperdata</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2222</value>
</property>
</configuration>
hduser@LinuxMint-Slave ~ $ cat /usr/local/hbase/conf/hbase-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.tmp.dir</name>
<value>file:///usr/local/hbase/hbasetmp/hbase-${user.name}</value>
</property>
<property>
<name>hbase.master</name>
<value>Ubuntu-Master:16000</value>
<description>The host and port that the HBase master runs at.</description>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://Ubuntu-Master:54310/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>Ubuntu-Master,LinuxMint-Slave</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>file:///usr/local/hbase/zookeeperdata</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2222</value>
</property>
</configuration>
这里是HMaster服务的日志文件
2015-02-03 12:21:14,168 WARN [Thread-12] hdfs.DFSClient: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplication (=1). There are 2 datanode(s) running and no node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1471) ………………….
……………………..
2015-02-03 12:21:14,185 DEBUG [master:Ubuntu-Master:60000] util.FSUtils: Unable to create version file at hdfs://Ubuntu-Master:54310/hbase, retrying
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplication (=1). There are 2 datanode(s) running and no node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1471)
…………………………………………
2015-02-03 12:21:24,285 WARN [Thread-15] hdfs.DFSClient: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplication (=1). There are 2 datanode(s) running and no node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1471)
……………………………………………………….
2015-02-03 12:21:24,286 DEBUG [master:Ubuntu-Master:60000] util.FSUtils: Unable to create version file at hdfs://Ubuntu-Master:54310/hbase, retrying
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplication (=1). There are 2 datanode(s) running and no node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1471)
……………………………………………..
2015-02-03 12:21:34,312 WARN [Thread-17] hdfs.DFSClient: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplication (=1). There are 2 datanode(s) running and no node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1471)
……………………………………………………………….
2015-02-03 12:21:44,333 FATAL [master:Ubuntu-Master:60000] master.HMaster: Unhandled exception. Starting shutdown.
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplication (=1). There are 2 datanode(s) running and no node(s) are excluded in this operation.
…………………………………….
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
2015-02-03 12:21:44,334 INFO [master:Ubuntu-Master:60000] master.HMaster: Aborting
2015-02-03 12:21:44,334 DEBUG [master:Ubuntu-Master:60000] master.HMaster: Stopping service threads
2015-02-03 12:21:44,335 INFO [master:Ubuntu-Master:60000] ipc.RpcServer: Stopping server on 60000
2015-02-03 12:21:44,335 INFO [RpcServer.listener,port=60000] ipc.RpcServer: RpcServer.listener,port=60000: stopping
2015-02-03 12:21:44,339 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped
2015-02-03 12:21:44,339 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping
2015-02-03 12:21:44,339 INFO [master:Ubuntu-Master:60000] master.HMaster: Stopping infoServer
2015-02-03 12:21:44,364 INFO [master:Ubuntu-Master:60000] mortbay.log: Stopped SelectChannelConnector@0.0.0.0:60010
2015-02-03 12:21:44,508 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2015-02-03 12:21:44,509 INFO [master:Ubuntu-Master:60000] zookeeper.ZooKeeper: Session: 0x14b4e1d0a040002 closed
2015-02-03 12:21:44,510 INFO [master:Ubuntu-Master:60000] master.HMaster: HMaster main thread exiting
2015-02-03 12:21:44,510 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: HMaster Aborted
at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:194)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:135)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2803)
2015-02-03 12:21:44,515 ERROR [Thread-5] hdfs.DFSClient: Failed to close file /hbase/.tmp/hbase.version
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/hbase.version could only be replicated to 0 nodes instead of minReplication (=1). There are 2 datanode(s) running and no node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1471)
我最近也有同样的问题 克服它的方法很简单,但是很危险 您肯定会丢失HDFS上的所有数据。 您应该执行以下操作:
Stop-hbase.sh&&Stop-warn.sh&&Stop-dfs.sh
/etc/hadoop/HDFS site.xml
中找到路径,在我的例子中,我必须删除的文件夹是/home/hadoop/hadoopdata/HDFS/namenode
和/home/hadoop/hadoopdata/HDFS/datanode
李>
相反,您可以简单地删除两台服务器上的/home/hadoop/hadoopdata
目录
以下是您可能需要查找的配置文件:
<property>
<name>dfs.name.dir</name>
<value>file:///home/hadoop/hadoopdata/hdfs/namenode</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>file:///home/hadoop/hadoopdata/hdfs/datanode</value>
</property>
dfs.name.dir
file:///home/hadoop/hadoopdata/hdfs/namenode
dfs.data.dir
file:///home/hadoop/hadoopdata/hdfs/datanode
hadoop namenode-format
(namenode部分可能与您不同)hadoop datanode-format
(datanode部分对您来说可能再次不同)Start-dfs.sh&&Start-warn.sh&&Start-hbase.sh
<property>
<name>dfs.name.dir</name>
<value>file:///home/hadoop/hadoopdata/hdfs/namenode</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>file:///home/hadoop/hadoopdata/hdfs/datanode</value>
</property>