Java MapReduceSpring数据配置

Java MapReduceSpring数据配置,java,maven,hadoop,mapreduce,spring-data,Java,Maven,Hadoop,Mapreduce,Spring Data,我试图使用Spring数据运行Map Reduce程序,但我一直得到与找不到mapper类相同的错误。谁能告诉我我做错了什么。我的应用程序上下文看起来 <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" x

我试图使用Spring数据运行Map Reduce程序,但我一直得到与找不到mapper类相同的错误。谁能告诉我我做错了什么。我的应用程序上下文看起来

<?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
        xmlns:hadoop="http://www.springframework.org/schema/hadoop"
        xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
            http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.2.xsd
            http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">


        <hadoop:configuration>
            fs.default.name=hdfs://localhost:9000
            mapred.job.tracker=localhost:9001
        </hadoop:configuration>
        <hadoop:job id="wordcountJob" input-path="/input"
            output-path="/outputlst" mapper="com.mapreduce.spring.WordMapper"
            reducer="com.mapreduce.spring.WordReducer" jar-by-class="com.mapreduce.spring.App" />
            <hadoop:job-runner id="runner" run-at-startup="true" job-ref="wordcountJob"/>
</beans>

fs.default.name=hdfs://localhost:9000
mapred.job.tracker=localhost:9001
我的pom.xml如下所示

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.springframework.samples</groupId>
    <artifactId>SpringDoop</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <properties>

        <!-- Generic properties -->
        <java.version>1.6</java.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

        <!-- Spring -->
        <spring-framework.version>3.2.3.RELEASE</spring-framework.version>

        <!-- Hibernate / JPA -->
        <hibernate.version>4.2.1.Final</hibernate.version>

        <!-- Logging -->
        <logback.version>1.0.13</logback.version>
        <slf4j.version>1.7.5</slf4j.version>

        <!-- Test -->
        <junit.version>4.11</junit.version>

    </properties>

    <dependencies>
        <!-- Spring and Transactions -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>${spring-framework.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-tx</artifactId>
            <version>${spring-framework.version}</version>
        </dependency>

        <!-- Logging with SLF4J & LogBack -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>${slf4j.version}</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>${logback.version}</version>
            <scope>runtime</scope>
        </dependency>

        <!-- Hibernate -->
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-entitymanager</artifactId>
            <version>${hibernate.version}</version>
        </dependency>


        <!-- Test Artifacts -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>${spring-framework.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-hadoop</artifactId>
            <version>1.0.1.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>1.2.0</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                            <mainClass>com.mapreduce.spring.App</mainClass>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>

        </plugins>
    </build>
</project>

4.0.0
org.springframework.samples
斯普林杜普
0.0.1-快照
1.6
UTF-8
UTF-8
3.2.3.1发布
4.2.1.最终版本
1.0.13
1.7.5
4.11
org.springframework
spring上下文
${spring framework.version}
org.springframework
德克萨斯州春季
${spring framework.version}
org.slf4j
slf4j api
${slf4j.version}
编译
回写
回归经典
${logback.version}
运行时
org.hibernate
休眠实体管理器
${hibernate.version}
org.springframework
弹簧试验
${spring framework.version}
测试
朱尼特
朱尼特
${junit.version}
测试
org.springframework.data
spring数据hadoop
1.0.1.1发布
org.apache.hadoop
hadoop内核
1.2.0
org.apache.maven.plugins
maven jar插件
2.3.2
真的
com.mapreduce.spring.App
我得到了以下例外

 Oct 08, 2013 10:46:57 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [application-context.xml]
Oct 08, 2013 10:46:57 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.GenericXmlApplicationContext@7e54864c: startup date [Tue Oct 08 22:46:57 PDT 2013]; root of context hierarchy
Oct 08, 2013 10:46:57 PM org.springframework.core.io.support.PropertiesLoaderSupport loadProperties
INFO: Loading properties file from class path resource [hadoop.properties]
Oct 08, 2013 10:46:57 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@1ffebe2b: defining beans [org.springframework.context.support.PropertySourcesPlaceholderConfigurer#0,hadoopConfiguration,wordcountJob,runner]; root of factory hierarchy
Oct 08, 2013 10:46:58 PM org.springframework.data.hadoop.mapreduce.JobExecutor$2 run
INFO: Starting job [wordcountJob]
Oct 08, 2013 10:46:58 PM org.apache.hadoop.mapred.JobClient copyAndConfigureFiles
WARNING: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
Oct 08, 2013 10:46:58 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
INFO: Total input paths to process : 1
Oct 08, 2013 10:46:58 PM org.apache.hadoop.util.NativeCodeLoader <clinit>
WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Oct 08, 2013 10:46:58 PM org.apache.hadoop.io.compress.snappy.LoadSnappy <clinit>
WARNING: Snappy native library not loaded
Oct 08, 2013 10:46:59 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: Running job: job_201310082141_0002
Oct 08, 2013 10:47:00 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO:  map 0% reduce 0%
Oct 08, 2013 10:47:08 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob
INFO: Task Id : attempt_201310082141_0002_m_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mapreduce.spring.WordMapper
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
    at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:199)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:718)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: com.mapreduce.spring.WordMapper
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
    ... 8 more
2013年10月08日10:46:57 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
信息:从类路径资源[application context.XML]加载XML bean定义
2013年10月8日晚上10:46:57 org.springframework.context.support.AbstractApplicationContext prepareRefresh
信息:刷新org.springframework.context.support。GenericXmlApplicationContext@7e54864c:启动日期[2013年10月8日星期二22:46:57 PDT];上下文层次结构的根
2013年10月8日晚上10:46:57 org.springframework.core.io.support.properties加载程序支持加载属性
信息:从类路径资源[hadoop.properties]加载属性文件
2013年10月8日晚上10:46:57 org.springframework.beans.factory.support.DefaultListableBeanFactory预实例化单例
信息:在org.springframework.beans.factory.support中预实例化单例。DefaultListableBeanFactory@1ffebe2b:定义bean[org.springframework.context.support.propertysourcesplaceplaceconfigurer#0,hadoopConfiguration,wordcountJob,runner];工厂层次结构的根
2013年10月8日晚上10:46:58 org.springframework.data.hadoop.mapreduce.JobExecutor$2运行
信息:正在启动作业[wordcountJob]
2013年10月8日晚上10:46:58 org.apache.hadoop.mapred.JobClient copyAndConfigureFiles
警告:未设置作业jar文件。可能找不到用户类。请参阅JobConf(类)或JobConf#setJar(字符串)。
2013年10月8日晚上10:46:58 org.apache.hadoop.mapreduce.lib.input.FileInputFormat列表状态
信息:要处理的总输入路径:1
2013年10月8日晚上10:46:58 org.apache.hadoop.util.NativeCodeLoader
警告:无法为您的平台加载本机hadoop库。。。在适用的情况下使用内置java类
2013年10月8日晚上10:46:58 org.apache.hadoop.io.compress.snappy.LoadSnappy
警告:未加载Snappy本机库
2013年10月8日晚上10:46:59 org.apache.hadoop.mapred.JobClient monitor和printjob
信息:正在运行作业:作业\u 201310082141\u 0002
2013年10月8日晚上10:47:00 org.apache.hadoop.mapred.JobClient monitor和printjob
信息:地图0%减少0%
2013年10月8日10:47:08 PM org.apache.hadoop.mapred.JobClient monitor和printjob
信息:任务Id:尝试\u 201310082141\u 0002\u m\u000000\u 0,状态:失败
java.lang.RuntimeException:java.lang.ClassNotFoundException:com.mapreduce.spring.WordMapper
位于org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
位于org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:199)
位于org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:718)
位于org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
位于org.apache.hadoop.mapred.Child$4.run(Child.java:255)
位于java.security.AccessController.doPrivileged(本机方法)
位于javax.security.auth.Subject.doAs(Subject.java:415)
位于org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
位于org.apache.hadoop.mapred.Child.main(Child.java:249)
原因:java.lang.ClassNotFoundException:com.mapreduce.spring.WordMapper
在java.net.URLClassLoader$1.run(URLClassLoader.java:366)
在java.net.URLClassLoader$1.run(URLClassLoader.java:355)
位于java.security.AccessController.doPrivileged(本机方法)
位于java.net.URLClassLoader.findClass(URLClassLoader.java:354)
位于java.lang.ClassLoader.loadClass(ClassLoader.java:424)
位于sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
位于java.lang.ClassLoader.loadClass(ClassLoader.java:357)
位于java.lang.Class.forName0(本机方法)
位于java.lang.Class.forName(Class.java:270)
位于org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
位于org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
... 8个以上
我不知道为什么会收到此警告无法使用内置java类为您的平台加载本机hadoop库。请告诉我我在做什么