使用whirr在amazon ec2上启动hadoop群集:找不到操作处理程序

使用whirr在amazon ec2上启动hadoop群集:找不到操作处理程序,hadoop,amazon-ec2,amazon,apache-whirr,Hadoop,Amazon Ec2,Amazon,Apache Whirr,我想在amazon实例上使用whirr启动一个实例。但是,当我尝试使用标准化whirr命令启动群集时,它会在我的hadoop群集名称后的某个目录名中搜索,该名称为“myclutster”,但该名称不存在。有人能帮我吗 kaustubh@hdv-Kaustubh:~/Downloads$ whirr launch-cluster --config whirrprop.properties Unable to start the cluster. Terminating all nodes. jav

我想在amazon实例上使用whirr启动一个实例。但是,当我尝试使用标准化whirr命令启动群集时,它会在我的hadoop群集名称后的某个目录名中搜索,该名称为“myclutster”,但该名称不存在。有人能帮我吗

kaustubh@hdv-Kaustubh:~/Downloads$ whirr launch-cluster --config whirrprop.properties Unable to start the cluster. Terminating all nodes. java.lang.IllegalArgumentException: java.lang.NullPointerException: Action handler not found at org.apache.whirr.actions.ScriptBasedClusterAction.safeGetActionHandler(ScriptBasedClusterAction.java:245) at org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100) at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:106) at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63) at org.apache.whirr.cli.Main.run(Main.java:64) at org.apache.whirr.cli.Main.main(Main.java:97) Caused by: java.lang.NullPointerException: Action handler not found at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204) at org.apache.whirr.HandlerMapFactory$ReturnHandlerByRoleOrPrefix.apply(HandlerMapFactory.java:66) at org.apache.whirr.HandlerMapFactory$ReturnHandlerByRoleOrPrefix.apply(HandlerMapFactory.java:45) at com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.compute(ComputingConcurrentHashMap.java:355) at com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.compute(ComputingConcurrentHashMap.java:184) at com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.getOrCompute(ComputingConcurrentHashMap.java:153) at com.google.common.collect.ComputingConcurrentHashMap.getOrCompute(ComputingConcurrentHashMap.java:69) at com.google.common.collect.ComputingConcurrentHashMap$ComputingMapAdapter.get(ComputingConcurrentHashMap.java:393) at org.apache.whirr.actions.ScriptBasedClusterAction.safeGetActionHandler(ScriptBasedClusterAction.java:238) ... 5 more Unable to load cluster state, assuming it has no running nodes. java.io.FileNotFoundException: /home/kaustubh/.whirr/mycluster/instances (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.(FileInputStream.java:120) at com.google.common.io.Files$1.getInput(Files.java:100) at com.google.common.io.Files$1.getInput(Files.java:97) at com.google.common.io.CharStreams$2.getInput(CharStreams.java:91) at com.google.common.io.CharStreams$2.getInput(CharStreams.java:88) at com.google.common.io.CharStreams.readLines(CharStreams.java:306) at com.google.common.io.Files.readLines(Files.java:580) at org.apache.whirr.state.FileClusterStateStore.load(FileClusterStateStore.java:54) at org.apache.whirr.state.ClusterStateStore.tryLoadOrEmpty(ClusterStateStore.java:58) at org.apache.whirr.ClusterController.destroyCluster(ClusterController.java:143) at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:118) at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63) at org.apache.whirr.cli.Main.run(Main.java:64) at org.apache.whirr.cli.Main.main(Main.java:97) java.lang.NullPointerException: Action handler not found Usage: whirr launch-cluster [OPTIONS] Option Description ------ ----------- --aws-ec2-spot-price Spot instance price (aws-ec2 specific option) --blobstore-cache-container The name of the container to be used for caching local files. If not specified Whirr will create a random one and remove it at the end of the session. --blobstore-credential The blob store credential --blobstore-identity The blob store identity --blobstore-location-id The blob store location ID --blobstore-provider The blob store provider. E.g. aws-s3, cloudfiles-us, cloudfiles-uk --client-cidrs A comma-separated list of CIDR blocks. E.g. 208.128.0.0/11,108.128.0.0/11 --cluster-name The name of the cluster to operate on. E.g. hadoopcluster. --cluster-user The name of the user that Whirr will create on all the cluster instances. You have to use this user to login to nodes. --config Note that Whirr properties specified in this file should all have a whirr. prefix. --credential The cloud credential. --firewall-rules A comma-separated list of port numbers. E.g. 8080,8181 --firewall-rules-role A comma-separated list of port numbers. E.g. 8080,8181. Replace 'role' with an actual role name --hardware-id The type of hardware to use for the instance. This must be compatible with the image ID. --hardware-min-ram The minimum amount of instance memory. E.g. 1024 --identity The cloud identity. --image-id The ID of the image to use for instances. If not specified then a vanilla Linux image is chosen. --instance-templates The number of instances to launch for each set of roles. E.g. 1 hadoop- namenode+hadoop-jobtracker, 10 hadoop-datanode+hadoop-tasktracker --instance-templates-max-percent- The percentage of successfully started failures instances for each set of roles. E. g. 100 hadoop-namenode+hadoop- jobtracker,60 hadoop-datanode+hadoop- tasktracker means all instances with the roles hadoop-namenode and hadoop- jobtracker has to be successfully started, and 60% of instances has to be succcessfully started each with the roles hadoop-datanode and hadoop- tasktracker. --instance-templates-minimum-number-of- The minimum numberof successfully instances started instances for each set of roles. E.g. 1 hadoop-namenode+hadoop- jobtracker,6 hadoop-datanode+hadoop- tasktracker means 1 instance with the roles hadoop-namenode and hadoop- jobtracker has to be successfully started, and 6 instances has to be successfully started each with the roles hadoop-datanode and hadoop- tasktracker. --location-id The location to launch instances in. If not specified then an arbitrary location will be chosen. --login-user Override the default login user used to bootstrap whirr. E.g. ubuntu or myuser:mypass. --max-startup-retries The number of retries in case of insufficient successfully started instances. Default value is 1. --private-key-file The filename of the private RSA key used to connect to instances. --provider The name of the cloud provider. E.g. aws-ec2, cloudservers-uk --public-key-file The filename of the public key used to connect to instances. --run-url-base The base URL for forming run urls from. Change this to host your own set of launch scripts. --service-name (optional) The name of the service to use. E.g. hadoop. --state-store What kind of store to use for state (local, blob or none). Defaults to local. --state-store-blob Blob name for state storage. Valid only for the blob state store. Defaults to whirr- --state-store-container Container where to store state. Valid only for the blob state store. --terminate-all-on-launch-failure Whether or not to automatically terminate all nodes when cluster launch fails for some reason. --version kaustubh@hdv-Kaustubh:~/Downloads$whirr启动集群--config whirprop.properties 无法启动群集。终止所有节点。 java.lang.IllegalArgumentException:java.lang.NullPointerException:未找到操作处理程序 位于org.apache.whirr.actions.ScriptBasedClusterAction.safeGetActionHandler(ScriptBasedClusterAction.java:245) 位于org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100) 位于org.apache.whirr.ClusterController.launchCluster(ClusterController.java:106) 位于org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63) 位于org.apache.whirr.cli.Main.run(Main.java:64) 位于org.apache.whirr.cli.Main.Main(Main.java:97) 原因:java.lang.NullPointerException:找不到操作处理程序 位于com.google.common.base.premissions.checkNotNull(premissions.java:204) 位于org.apache.whirr.HandlerMapFactory$ReturnHandlerByRoleOrPrefix.apply(HandlerMapFactory.java:66) 位于org.apache.whirr.HandlerMapFactory$ReturnHandlerByRoleOrPrefix.apply(HandlerMapFactory.java:45) 在com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.compute上(ComputingConcurrentHashMap.java:355) 在com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.compute上(ComputingConcurrentHashMap.java:184) 在com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.getOrCompute上(ComputingConcurrentHashMap.java:153) 在com.google.common.collect.ComputingConcurrentHashMap.getOrCompute上(ComputingConcurrentHashMap.java:69) 位于com.google.common.collect.ComputingConcurrentHashMap$ComputingMapAdapter.get(ComputingConcurrentHashMap.java:393) 位于org.apache.whirr.actions.ScriptBasedClusterAction.safeGetActionHandler(ScriptBasedClusterAction.java:238) ... 还有5个 无法加载群集状态,假定它没有正在运行的节点。 java.io.FileNotFoundException:/home/kaustubh/.whirr/mycluster/instances(没有这样的文件或目录) 在java.io.FileInputStream.open(本机方法) 位于java.io.FileInputStream。(FileInputStream.java:120) 位于com.google.common.io.Files$1.getInput(Files.java:100) 位于com.google.common.io.Files$1.getInput(Files.java:97) 位于com.google.common.io.CharStreams$2.getInput(CharStreams.java:91) 位于com.google.common.io.CharStreams$2.getInput(CharStreams.java:88) 位于com.google.common.io.CharStreams.readLines(CharStreams.java:306) 位于com.google.common.io.Files.readLines(Files.java:580) 位于org.apache.whirr.state.FileClusterStateStore.load(FileClusterStateStore.java:54) 位于org.apache.whirr.state.ClusterStateStore.tryLoadOrEmpty(ClusterStateStore.java:58) 位于org.apache.whirr.ClusterController.destroyCluster(ClusterController.java:143) 位于org.apache.whirr.ClusterController.launchCluster(ClusterController.java:118) 位于org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63) 位于org.apache.whirr.cli.Main.run(Main.java:64) 位于org.apache.whirr.cli.Main.Main(Main.java:97) java.lang.NullPointerException:未找到操作处理程序 用法:whirr启动群集[选项] 选项说明 ------ ----------- --aws-ec2-现货价格现货实例价格(特定于aws-ec2 (可选) --blobstore缓存容器要使用的容器的名称 用于缓存本地文件。如果不是 指定的Whirr将创建一个随机 一个,然后将其从 一场 --blobstore凭据blob存储凭据 --blobstore标识blob存储标识 --blob存储位置id blob存储位置id --blobstore提供程序blob存储提供程序。例如aws-s3, cloudfiles美国、cloudfiles英国 --客户端CIDR以逗号分隔的CIDR块列表。 例如208.128.0.0/11108.128.0.0/11 --cluster name要操作的群集的名称。 例如Hadoop集群。 --cluster user将访问的用户的名称 在所有集群实例上创建。 您必须使用此用户登录 到节点。 --请注意,指定了Whirr属性 此文件中的所有文件都应具有 呼呼。前缀 --凭据云凭据。 --防火墙规则以逗号分隔的端口列表 数字。例如808181 --防火墙规则角色以逗号分隔的端口列表 数字。例如808181。代替 具有实际角色名称的“角色” --硬件id要使用的硬件类型
whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 1 hadoop-datanode+hadooptasktracker
whirr.hardware-id=m1.small # or larger - t1.micro is not a good choice