Apache Can';不要运行giraph SimpleIndegreeCount计算
我正在尝试运行Giraph中包含的SimpleIndegreeCountComputement示例。我的做法如下: SimpleInDegreeCountComputation.java:Apache Can';不要运行giraph SimpleIndegreeCount计算,apache,giraph,Apache,Giraph,我正在尝试运行Giraph中包含的SimpleIndegreeCountComputement示例。我的做法如下: SimpleInDegreeCountComputation.java: public class SimpleInDegreeCountComputation extends BasicComputation <LongWritable, LongWritable, DoubleWritable, DoubleWritable> {
public class SimpleInDegreeCountComputation extends BasicComputation
<LongWritable, LongWritable, DoubleWritable, DoubleWritable> {
.......
结果如下:
14/05/18 18:58:40 INFO utils.ConfigurationUtils: No edge input format specified.
Ensure your InputFormat does not require one.
14/05/18 18:58:40 INFO utils.ConfigurationUtils: No edge output format specified.
Ensure your OutputFormat does not require one.
Exception in thread "main" java.lang.IllegalArgumentException: checkClassTypes: vertex
value types not assignable, computation - class org.apache.hadoop.io.LongWritable,
VertexInputFormat - class org.apache.hadoop.io.DoubleWritable
at org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
at org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
at org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
[0,0,[[1,5],[2,9]]]
[1,0,[[0,5],[3,3]]]
[2,0,[[0,9],[3,3],[4,3]]]
[3,0,[[1,3],[2,3],[4,2]]]
[4,0,[[2,3],[3,3]]]
我不太确定我做错了什么。如果有人能给我指出正确的方向,或者链接到一个资源,解释我正在尝试做的事情的更简单的方式,我将不胜感激!我认为问题可能是格式错误(-vif)。
我使用的输入文件如下所示:
14/05/18 18:58:40 INFO utils.ConfigurationUtils: No edge input format specified.
Ensure your InputFormat does not require one.
14/05/18 18:58:40 INFO utils.ConfigurationUtils: No edge output format specified.
Ensure your OutputFormat does not require one.
Exception in thread "main" java.lang.IllegalArgumentException: checkClassTypes: vertex
value types not assignable, computation - class org.apache.hadoop.io.LongWritable,
VertexInputFormat - class org.apache.hadoop.io.DoubleWritable
at org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
at org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
at org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
[0,0,[[1,5],[2,9]]]
[1,0,[[0,5],[3,3]]]
[2,0,[[0,9],[3,3],[4,3]]]
[3,0,[[1,3],[2,3],[4,2]]]
[4,0,[[2,3],[3,3]]]
从compute和vertex输入类的定义来看,
JsonLongDoubleFloatDoubleVertexInputFormat
似乎与SimpleIndegreeCountComputing
不兼容
:
:
/**
*抽象类,用户应将其子类化以使用自己的基于文本的
*顶点输入格式。
*
*@param顶点索引值
*@param顶点值
*@param边值
*/
@抑制警告(“原始类型”)
公共抽象类TextVertexInputFormat
扩展VertexInputFormat{
你可以看到:
- 顶点id的类型为
longwriteable
- 顶点数据的类型为
doublewriteable
- 边缘数据的类型为
floatwriteable
由于它是
LongWritable
,DoubleWritable
和FloatWritable
而不是Long
,Double
和Float
,这些类型无法自动转换
我找不到任何您可以使用的InputFormat,因此您需要修改现有的
JsonLongDoubleFloatDoubleVertexInputFormat
或修改算法以使用NullWritable
作为边缘数据类型。我看不到任何要使用的边缘数据,因此它也可以为null。在这种情况下,您可以使用。Hi zahorak!谢谢你的回答。我试过这么做,但现在的问题是创建一个包含修改过的文件的新jar。我不知道怎么做。我解包了giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar来查看它是如何制作的。但我仍然无法创建正确的jar文件,因为我没有所有必要的文件(包含在giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar中).有什么建议吗?我在giraph中编写的所有代码,都是在giraph示例中编写的。只需从github签出并编辑它,然后运行mvn clean install-DskipTests
,您就可以获得所需的所有JAR。我的存储库中是否包含我对giraph示例所做的更改,以便能够运行我的代码。谢谢zahorak。根据你的建议我的解决方案是:1)从github获取代码2)修改文件SimpleIndegreeCountComputing 3)使用mvn创建JAR安装现在可以了:))步骤4)可以是发送一个拉回请求,这样其他人就不会再有这个问题了(但我不确定他们是否会接受)
/**
* Computation in which both incoming and outgoing message types are the same.
*
* @param <I> Vertex id
* @param <V> Vertex data
* @param <E> Edge data
* @param <M> Message type
*/
public abstract class BasicComputation<I extends WritableComparable,
V extends Writable, E extends Writable, M extends Writable>
extends AbstractComputation<I, V, E, M, M> {
}
public class JsonLongDoubleFloatDoubleVertexInputFormat extends
TextVertexInputFormat<LongWritable, DoubleWritable, FloatWritable> {
/**
* Abstract class that users should subclass to use their own text based
* vertex input format.
*
* @param <I> Vertex index value
* @param <V> Vertex value
* @param <E> Edge value
*/
@SuppressWarnings("rawtypes")
public abstract class TextVertexInputFormat<I extends WritableComparable,
V extends Writable, E extends Writable>
extends VertexInputFormat<I, V, E> {