Java Apache Storm-初始化时出错(工作)
风暴-1.0.1 我的风暴。yaml:Java Apache Storm-初始化时出错(工作),java,python,netty,apache-storm,Java,Python,Netty,Apache Storm,风暴-1.0.1 我的风暴。yaml: storm.zookeper.servers: - "zookeper" storm.zookeeper.port: 2181 # nimbus.host: "nimbus" nimbus.seeds: ["nimbus"] storm.local.dir: "/var/log/storm" supervisor.slots.ports: - 6700 - 6701 - 6702 - 6703 worker.childopts: "-
storm.zookeper.servers:
- "zookeper"
storm.zookeeper.port: 2181
# nimbus.host: "nimbus"
nimbus.seeds: ["nimbus"]
storm.local.dir: "/var/log/storm"
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
worker.childopts: "-Xmx768m"
nimbus.childopts: "-Xmx512m"
supervisor.childopts: "-Xmx256m"
storm.messaging.transport: "backtype.storm.messaging.netty.Context"
storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 5242880
storm.messaging.netty.max_retries: 100
storm.messaging.netty.max_wait_ms: 1000
storm.messaging.netty.min_wait_ms: 100
在我的主管日志中,我看到:
2016-08-26 19:22:28.023 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:28.523 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:28.883 o.a.s.d.supervisor [INFO] Worker Process b4a402ba-458d-4c4e-b167-143bd9134200 exited with code: 13
2016-08-26 19:22:28.956 o.a.s.d.supervisor [INFO] Worker Process c9718f4b-131f-4cfe-bce7-e3e448790a4e exited with code: 13
2016-08-26 19:22:28.966 o.a.s.d.supervisor [INFO] Worker Process d30be8b6-6ed0-496e-9d23-6879df1f1bdd exited with code: 13
2016-08-26 19:22:29.023 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:29.040 o.a.s.d.supervisor [INFO] Worker Process e80983c3-1ec6-4b25-9ef1-f29eb42194ae exited with code: 13
2016-08-26 19:22:29.524 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:30.024 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:30.524 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:31.025 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:31.525 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:32.025 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:32.525 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
工作日志包含以下错误:
因为错误表明未找到类
backtype.storm.messaging.netty.Context
原因:java.lang.ClassNotFoundException:backtype.storm.messaging.netty.Context
在Storm1.0.0
中,对许多包进行了重大的重构重命名。我想您现在应该使用org.apache.storm.messaging.netty.Context
storm.messaging.transport: "org.apache.storm.messaging.netty.Context"
请参见问题在于我的JAVA_HOME env var指向错误的文件夹 之前: JAVA_HOME=/usr/lib/jvm/JAVA-1.8.0-openjdk-1.8.0.101-1.b14.fc24.x86_64/jre/ 之后:
storm.zookeper.servers:
- "zookeper"
storm.zookeeper.port: 2181
# nimbus.host: "nimbus"
nimbus.seeds: ["nimbus"]
storm.local.dir: "/var/log/storm"
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
worker.childopts: "-Xmx768m"
nimbus.childopts: "-Xmx512m"
supervisor.childopts: "-Xmx256m"
storm.messaging.transport: "backtype.storm.messaging.netty.Context"
storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 5242880
storm.messaging.netty.max_retries: 100
storm.messaging.netty.max_wait_ms: 1000
storm.messaging.netty.min_wait_ms: 100
JAVA_HOME=/usr/lib/jvm/JAVA-1.8.0-oracle-1.8.0.92.x86_64/
我得到了检查storm是否正确安装的建议,方法是运行storm示例中提供的示例java拓扑
我跟着这位官员
给我这个:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 49.281 s
[INFO] Finished at: 2016-08-27T19:16:54-07:00
[INFO] Final Memory: 21M/391M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project storm-starter: Could not resolve dependencies for project org.apache.storm:storm-starter:jar:1.0.1: Could not find artifact jdk.tools:jdk.tools:jar:1.7 at specified path /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-1.b14.fc24.x86_64/jre/../lib/tools.jar -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
(petrel) [dmitry:/storm-starter]$ dnf list installed | grep java
55:abrt-java-connector.x86_64 1.1.0-8.fc24 @koji-override-0
749:java-1.8.0-openjdk.x86_64 1:1.8.0.101-1.b14.fc24 @updates
750:java-1.8.0-openjdk-headless.x86_64 1:1.8.0.101-1.b14.fc24 @updates
751:java-1.8.0-oracle.x86_64 1.8.0.92-1.fc24.R @russianfedora-nonfree
752:java-1.8.0-oracle-devel.x86_64 1.8.0.92-1.fc24.R @russianfedora-nonfree
753:java-1.8.0-oracle-headless.x86_64 1.8.0.92-1.fc24.R @russianfedora-nonfree
754:javapackages-tools.noarch 4.6.0-14.fc24 @koji-override-0
1846:python3-javapackages.noarch 4.6.0-14.fc24 @koji-override-0
2431:tzdata-java.noarch 2016f-1.fc24 @updates
当我将JAVA_HOME切换到/usr/lib/jvm/JAVA-1.8.0-oracle-1.8.0.92.x86_64/时,我就能够编译样本Tology并将其提交到远程集群(该集群仍然是storm的本地安装,zookeeper/nimbus/supervisor/ui/logviewer正在本地运行)
还请记住,雅虎的一篇文章建议将“netty”作为传输,并提供以下值:
#storm.messaging.transport: "backtype.storm.messaging.netty.Context"
#storm.messaging.netty.server_worker_threads: 1
#storm.messaging.netty.client_worker_threads: 1
#storm.messaging.netty.buffer_size: 5242880
#storm.messaging.netty.max_retries: 100
#storm.messaging.netty.max_wait_ms: 1000
#storm.messaging.netty.min_wait_ms: 100
这会对你的系统产生负面影响,因为storm 1.0+改变了他们的课程结构
类在1.0.1中从backtype.storm重新打包到org.apache.storm。
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 49.281 s
[INFO] Finished at: 2016-08-27T19:16:54-07:00
[INFO] Final Memory: 21M/391M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project storm-starter: Could not resolve dependencies for project org.apache.storm:storm-starter:jar:1.0.1: Could not find artifact jdk.tools:jdk.tools:jar:1.7 at specified path /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-1.b14.fc24.x86_64/jre/../lib/tools.jar -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
(petrel) [dmitry:/storm-starter]$ dnf list installed | grep java
55:abrt-java-connector.x86_64 1.1.0-8.fc24 @koji-override-0
749:java-1.8.0-openjdk.x86_64 1:1.8.0.101-1.b14.fc24 @updates
750:java-1.8.0-openjdk-headless.x86_64 1:1.8.0.101-1.b14.fc24 @updates
751:java-1.8.0-oracle.x86_64 1.8.0.92-1.fc24.R @russianfedora-nonfree
752:java-1.8.0-oracle-devel.x86_64 1.8.0.92-1.fc24.R @russianfedora-nonfree
753:java-1.8.0-oracle-headless.x86_64 1.8.0.92-1.fc24.R @russianfedora-nonfree
754:javapackages-tools.noarch 4.6.0-14.fc24 @koji-override-0
1846:python3-javapackages.noarch 4.6.0-14.fc24 @koji-override-0
2431:tzdata-java.noarch 2016f-1.fc24 @updates
#storm.messaging.transport: "backtype.storm.messaging.netty.Context"
#storm.messaging.netty.server_worker_threads: 1
#storm.messaging.netty.client_worker_threads: 1
#storm.messaging.netty.buffer_size: 5242880
#storm.messaging.netty.max_retries: 100
#storm.messaging.netty.max_wait_ms: 1000
#storm.messaging.netty.min_wait_ms: 100