Hadoop copyFromLocalFile不';t在CDH4中工作

Hadoop copyFromLocalFile不';t在CDH4中工作,hadoop,amazon-ec2,cloudera,java,Hadoop,Amazon Ec2,Cloudera,Java,我已经在亚马逊云(1台服务器)的Ubuntu12LTS服务器上成功安装了CDH4。我使用Cloudera Manager免费版安装软件,没有错误) 我有一个程序,它使用JavaAPI将文件从我的家用计算机加载到云中的HDFS。我想知道这个程序失败的原因以及如何修复它 Configuration conf = new Configuration(); conf.set("fs.defaultFS", "hdfs://node01:8020"); FileSystem fs = Fil

我已经在亚马逊云(1台服务器)的Ubuntu12LTS服务器上成功安装了CDH4。我使用Cloudera Manager免费版安装软件,没有错误)

我有一个程序,它使用JavaAPI将文件从我的家用计算机加载到云中的HDFS。我想知道这个程序失败的原因以及如何修复它

Configuration conf = new Configuration();       
conf.set("fs.defaultFS", "hdfs://node01:8020");
FileSystem fs = FileSystem.get(conf);       
Path targetPath = new Path("/users/<username>/myfile.txt");
Path sourcePath = new Path("/home/<username>/myfile.txt");
fs.copyFromLocalFile(false,true,sourcePath,targetPath);

我已经解决了这个问题-

我是从家里的机器连接到hadoop的,而不是在hadoop本地网络上。显然,当您这样做时,namenode会告诉我的家庭计算机使用datanode的私有IP写入datanode。由于不在同一网络上,我的家庭计算机无法连接到创建此错误的datanode

我通过创建一个从家庭网络到hadoop网络的VPN连接解决了这个问题,现在一切都正常了

org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/<username>/myfile.txt could only be replicated to 0 nodes instead of minReplication (=1).  There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
    at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1322)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2170)
    at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:471)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:297)
    at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44080)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687)
Configured Capacity: 95120474112 (88.59 GB)
Present Capacity: 95120474112 (88.59 GB)
DFS Remaining: 95039008768 (88.51 GB)
DFS Used: 81465344 (77.69 MB)
DFS Used%: 0.09%
Under replicated blocks: 177
Blocks with corrupt replicas: 0
Missing blocks: 0

-------------------------------------------------
Datanodes available: 1 (1 total, 0 dead)

Live datanodes:
Name: privateip:port (node01)
Hostname: node01
Rack: /default
Decommission Status : Normal
Configured Capacity: 95120474112 (88.59 GB)
DFS Used: 81465344 (77.69 MB)
Non DFS Used: 0 (0 KB)
DFS Remaining: 95039008768 (88.51 GB)
DFS Used%: 0.09%
DFS Remaining%: 99.91%
Last contact: Sun Jan 27 03:01:53 UTC 2013