Apache spark 设置Spark时如何解决以下错误

我正在建立Spark框架

我确实将winutils.exe放在了C:\hadoop\bin文件夹中，并通过编辑PATH变量更新了二进制文件。不过，当我启动Spark时，我会出现以下错误。

在Hadoop二进制文件中找不到可执行文件null\bin\winutils.exe。

然而，尽管存在上述错误，我仍然可以成功地启动Spark。请告知在运行spark作业时，上述错误是否会导致任何问题

接下来，当我尝试运行wordcount.py文件时..出现以下错误：

C:\python-spark\python-spark-tutorial>spark-submit rdd\WordCount.py
19/03/10 11:51:10 ERROR Shell: Failed to locate the winutils binary in the 
hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in 
the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:379)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:394)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:387)
    at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80)
    at org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
    at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
    at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:791)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:761)
    at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:634)
    at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2422)
    at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2422)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2422)
    at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:79)
    at org.apache.spark.deploy.SparkSubmit.secMgr$lzycompute$1(SparkSubmit.scala:359)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$secMgr$1(SparkSubmit.scala:359)
    at org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply(SparkSubmit.scala:367)
    at org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply(SparkSubmit.scala:367)
    at scala.Option.map(Option.scala:146)
    at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:366)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:143)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
19/03/10 11:51:10 WARN NativeCodeLoader: Unable to load native-hadoop 
library for your platform... using builtin-java classes where applicable
Traceback (most recent call last):
File "C:\Users\gabs\AppData\Local\Programs\Python\Python37-32\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "C:\Users\gabs\AppData\Local\Programs\Python\Python37-32\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\python-spark\spark-2.4.0-bin- 
hadoop2.7\python\lib\pyspark.zip\pyspark\worker.py", line 25, in <module>
ModuleNotFoundError: No module named 'resource'
19/03/10 11:51:26 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
org.apache.spark.SparkException: Python worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:170)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:97)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:117)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:108)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:121)
    at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:402)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:408)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(Unknown Source)
    at java.net.AbstractPlainSocketImpl.accept(Unknown Source)
    at java.net.PlainSocketImpl.accept(Unknown Source)
    at java.net.ServerSocket.implAccept(Unknown Source)
    at java.net.ServerSocket.accept(Unknown Source)
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:164)
    ... 14 more
19/03/10 11:51:26 ERROR TaskSetManager: Task 0 in stage 0.0 failed 1 times; aborting job
Traceback (most recent call last):
File "C:/python-spark/python-spark-tutorial/rdd/WordCount.py", line 12, in <module>
wordCounts = words.countByValue()
File "C:\python-spark\spark-2.4.0-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\rdd.py", line 1261, in countByValue
File "C:\python-spark\spark-2.4.0-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\rdd.py", line 844, in reduce
File "C:\python-spark\spark-2.4.0-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\rdd.py", line 816, in collect
File "C:\python-spark\spark-2.4.0-bin-hadoop2.7\python\lib\py4j-0.10.7-src.zip\py4j\java_gateway.py", line 1257, in __call__
File "C:\python-spark\spark-2.4.0-bin-hadoop2.7\python\lib\py4j-0.10.7- 
src.zip\py4j\protocol.py", line 328, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling 
z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 
in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 
(TID 0, localhost, executor driver): org.apache.spark.SparkException: Python 
worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:170)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:97)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:117)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:108)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:121)
    at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:402)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:408)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(Unknown Source)
    at java.net.AbstractPlainSocketImpl.accept(Unknown Source)
    at java.net.PlainSocketImpl.accept(Unknown Source)
    at java.net.ServerSocket.implAccept(Unknown Source)
    at java.net.ServerSocket.accept(Unknown Source)
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:164)
    ... 14 more

Driver stacktrace:
    at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1887)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1875)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1874)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
    at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1874)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
    at scala.Option.foreach(Option.scala:257)
    at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2108)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2057)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2046)
    at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
    at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:737)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2061)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2082)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2101)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2126)
    at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:945)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
    at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
    at org.apache.spark.rdd.RDD.collect(RDD.scala:944)
    at org.apache.spark.api.python.PythonRDD$.collectAndServe(PythonRDD.scala:166)
    at org.apache.spark.api.python.PythonRDD.collectAndServe(PythonRDD.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
    at py4j.Gateway.invoke(Gateway.java:282)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:238)
    at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.spark.SparkException: Python worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:170)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:97)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:117)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:108)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:121)
    at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:402)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:408)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    ... 1 more
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(Unknown Source)
    at java.net.AbstractPlainSocketImpl.accept(Unknown Source)
    at java.net.PlainSocketImpl.accept(Unknown Source)
    at java.net.ServerSocket.implAccept(Unknown Source)
    at java.net.ServerSocket.accept(Unknown Source)
    at  org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:164)
    ... 14 more

C:\python spark\python spark教程>spark提交rdd\WordCount.py
2010年3月19日11:51:10错误Shell:在中找不到winutils二进制文件
hadoop二进制路径
java.io.IOException:在中找不到可执行文件null\bin\winutils.exe
Hadoop二进制文件。
位于org.apache.hadoop.util.Shell.getQualifiedBinPath（Shell.java:379）
位于org.apache.hadoop.util.Shell.getWinUtilsPath（Shell.java:394）
位于org.apache.hadoop.util.Shell（Shell.java:387）
位于org.apache.hadoop.util.StringUtils。（StringUtils.java:80）
位于org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod（SecurityUtil.java:611）
位于org.apache.hadoop.security.UserGroupInformation.initialize（UserGroupInformation.java:273）
位于org.apache.hadoop.security.UserGroupInformation.ensureInitialized（UserGroupInformation.java:261）
位于org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject（UserGroupInformation.java:791）
位于org.apache.hadoop.security.UserGroupInformation.getLoginUser（UserGroupInformation.java:761）
位于org.apache.hadoop.security.UserGroupInformation.getCurrentUser（UserGroupInformation.java:634）
位于org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply（Utils.scala:2422）
位于org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply（Utils.scala:2422）
位于scala.Option.getOrElse（Option.scala:121）
位于org.apache.spark.util.Utils$.getCurrentUserName（Utils.scala:2422）
位于org.apache.spark.SecurityManager（SecurityManager.scala:79）
位于org.apache.spark.deploy.SparkSubmit.secMgr$lzycompute$1（SparkSubmit.scala:359）
位于org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$secMgr$1（SparkSubmit.scala:359）
位于org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply（SparkSubmit.scala:367）
位于org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply（SparkSubmit.scala:367）
在scala.Option.map处（Option.scala:146）
位于org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment（SparkSubmit.scala:366）
位于org.apache.spark.deploy.SparkSubmit.submit（SparkSubmit.scala:143）
位于org.apache.spark.deploy.SparkSubmit.doSubmit（SparkSubmit.scala:86）
位于org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit（SparkSubmit.scala:924）
位于org.apache.spark.deploy.SparkSubmit$.main（SparkSubmit.scala:933）
位于org.apache.spark.deploy.SparkSubmit.main（SparkSubmit.scala）
19/03/10 11:51:10警告NativeCodeLoader:无法加载本机hadoop
您的平台的库。。。在适用的情况下使用内置java类
回溯（最近一次呼叫最后一次）：
文件“C:\Users\gabs\AppData\Local\Programs\Python\Python37-32\lib\runpy.py”，第193行，在运行模块中作为主
“\uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu
文件“C:\Users\gabs\AppData\Local\Programs\Python 37-32\lib\runpy.py”，第85行，在运行代码中
exec（代码、运行\全局）
文件“C:\python spark\spark-2.4.0-bin-
hadoop2.7\python\lib\pyspark.zip\pyspark\worker.py”，第25行，在
ModuleNotFoundError:没有名为“资源”的模块
2010年3月19日11:51:26错误执行者：任务0.0在阶段0.0（TID 0）中出现异常
org.apache.spark.SparkException:Python工作者无法连接回。
位于org.apache.spark.api.PythonWorkerFactory.createSimpleWorker（PythonWorkerFactory.scala:170）
位于org.apache.spark.api.python.PythonWorkerFactory.create（PythonWorkerFactory.scala:97）
位于org.apache.spark.SparkEnv.createPythonWorker（SparkEnv.scala:117）
位于org.apache.spark.api.python.BasePythonRunner.compute（PythonRunner.scala:108）
位于org.apache.spark.api.python.PythonRDD.compute（PythonRDD.scala:65）
在org.apache.spark.rdd.rdd.computeOrReadCheckpoint（rdd.scala:324）
位于org.apache.spark.rdd.rdd.iterator（rdd.scala:288）
位于org.apache.spark.scheduler.ResultTask.runTask（ResultTask.scala:90）
位于org.apache.spark.scheduler.Task.run（Task.scala:121）
位于org.apache.spark.executor.executor$TaskRunner$$anonfun$10.apply（executor.scala:402）
位于org.apache.spark.util.Utils$.tryWithSafeFinally（Utils.scala:1360）
位于org.apache.spark.executor.executor$TaskRunner.run（executor.scala:408）
位于java.util.concurrent.ThreadPoolExecutor.runWorker（未知源）
位于java.util.concurrent.ThreadPoolExecutor$Worker.run（未知源）
位于java.lang.Thread.run（未知源）
原因：java.net.SocketTimeoutException:接受超时
位于java.net.DualStackPlainSocketImpl.waitForNewConnection（本机方法）
位于java.net.DualStackPlainSocketImpl.socketAccept（未知源）
位于java.net.AbstractPlainSocketImpl.accept（未知源）
位于java.net.PlainSocketImpl.accept（未知源）
位于java.net.ServerSocket.implacpt（未知源）
位于java.net.ServerSocket.accept（未知源）
位于org.apache.spark.api.PythonWorkerFactory.createSimpleWorker（PythonWorkerFactory.scala:164）
... 14多
2010年3月19日11:51:26错误TaskSetManager:阶段0.0中的任务0失败1次；中止工作
回溯（最近一次呼叫最后一次）：
文件“C:/pythonspark/pythonspark tutorial/rdd/WordCount.py”，第12行，在
wordCounts=words.countByValue（）
文件“C:\python spark\spark-2.4.0-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\rdd.py”，第1261行，countByValue中
文件“C:\python spark\spark-2.4.0-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\rdd.py”，reduce中第844行
文件“C:\python spark\spark-2.4.0-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\rdd.py”，第816行，在collect中
文件“C:\python spark\spark-2.4.0-bin-had
SET HADOOP_HOME=C:\hadoop
spark-submit rdd\WordCount.py