Java 如何设置Intellij以在scala项目中使用AWS默认配置文件

我有一个scala项目，我想使用s3a从S3 bucket中读取：// 我已经在intellij中配置了AWS配置文件，并且可以从命令行访问我的文件。 但是，如果我从intellij运行它，它会给我错误 我也尝试过使用s3://和s3://n阅读，但没有成功。 还尝试添加aws creds作为环境变量

    at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:141)
    at org.apache.hadoop.fs.s3a.S3AFileSystem.verifyBucketExists(S3AFileSystem.java:332)
    at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:275)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3288)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3337)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3305)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:476)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
    at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary$1.apply(DataSource.scala:547)
    at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary$1.apply(DataSource.scala:545)
    at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
    at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
    at scala.collection.immutable.List.flatMap(List.scala:355)
    at org.apache.spark.sql.execution.datasources.DataSource.org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary(DataSource.scala:545)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:359)
    at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
    at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:615)
    at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:467)
    at com.expediagroup.egdp.mydemoapp.task.S3TaskRunner.getS3Data(S3TaskRunner.scala:24)
    at com.expediagroup.egdp.mydemoapp.task.S3TaskRunner.run(S3TaskRunner.scala:17)
    at com.expediagroup.egdp.mydemoapp.util.TaskInitializer.initialize(TaskInitializer.scala:22)
    at com.expediagroup.egdp.mydemoapp.main$.main(main.scala:30)
    at com.expediagroup.egdp.mydemoapp.main.main(main.scala)
Caused by: com.amazonaws.AmazonClientException: No AWS Credentials provided by BasicAWSCredentialsProvider EnvironmentVariableCredentialsProvider InstanceProfileCredentialsProvider : com.amazonaws.SdkClientException: Unable to load credentials from service endpoint
    at org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:151)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1166)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:762)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:724)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4365)
    at com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:5126)
    at com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:5100)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4349)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4312)
    at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1343)
    at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1283)
    at org.apache.hadoop.fs.s3a.S3AFileSystem.verifyBucketExists(S3AFileSystem.java:320)
    ... 25 more
Caused by: com.amazonaws.SdkClientException: Unable to load credentials from service endpoint
    at com.amazonaws.auth.EC2CredentialsFetcher.handleError(EC2CredentialsFetcher.java:183)
    at com.amazonaws.auth.EC2CredentialsFetcher.fetchCredentials(EC2CredentialsFetcher.java:162)
    at com.amazonaws.auth.EC2CredentialsFetcher.getCredentials(EC2CredentialsFetcher.java:82)
    at com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:164)
    at org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:129)
    ... 41 more
Caused by: java.net.SocketTimeoutException: connect timed out
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
    at sun.net.www.http.HttpClient.New(HttpClient.java:308)
    at sun.net.www.http.HttpClient.New(HttpClient.java:326)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1148)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
    at com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:54)
    at com.amazonaws.internal.EC2CredentialsUtils.readResource(EC2CredentialsUtils.java:116)
    at com.amazonaws.internal.EC2CredentialsUtils.readResource(EC2CredentialsUtils.java:87)
    at com.amazonaws.auth.InstanceProfileCredentialsProvider$InstanceMetadataCredentialsEndpointProvider.getCredentialsEndpoint(InstanceProfileCredentialsProvider.java:189)
    at com.amazonaws.auth.EC2CredentialsFetcher.fetchCredentials(EC2CredentialsFetcher.java:122)
    ... 44 more```

位于org.apache.hadoop.fs.s3a.S3AUtils.translateException（S3AUtils.java:141）
位于org.apache.hadoop.fs.s3a.S3AFileSystem.verifyBucketExists（S3AFileSystem.java:332）
位于org.apache.hadoop.fs.s3a.S3AFileSystem.initialize（S3AFileSystem.java:275）
位于org.apache.hadoop.fs.FileSystem.createFileSystem（FileSystem.java:3288）
位于org.apache.hadoop.fs.FileSystem.access$200（FileSystem.java:123）
位于org.apache.hadoop.fs.FileSystem$Cache.getInternal（FileSystem.java:3337）
位于org.apache.hadoop.fs.FileSystem$Cache.get（FileSystem.java:3305）
位于org.apache.hadoop.fs.FileSystem.get（FileSystem.java:476）
位于org.apache.hadoop.fs.Path.getFileSystem（Path.java:361）
位于org.apache.spark.sql.execution.datasources.DataSource$$anonfun$org$apache$spark$sql$execution$datasources$datasources$$checkandglobpathif-needed$1.apply（DataSource.scala:547）
在org.apache.spark.sql.execution.datasources.DataSource$$anonfun$org$apache$spark$sql$execution$datasources$datasources$$checkandglobpathif needed$1.apply（DataSource.scala:545）
位于scala.collection.TraversableLike$$anonfun$flatMap$1.apply（TraversableLike.scala:241）
位于scala.collection.TraversableLike$$anonfun$flatMap$1.apply（TraversableLike.scala:241）
位于scala.collection.immutable.List.foreach（List.scala:392）
位于scala.collection.TraversableLike$class.flatMap（TraversableLike.scala:241）
位于scala.collection.immutable.List.flatMap（List.scala:355）
在org.apache.spark.sql.execution.datasources.DataSource.org$apache$spark$sql$execution$datasources$datasources$DataSource$$checkandglobpathif needed（DataSource.scala:545）
位于org.apache.spark.sql.execution.datasources.DataSource.resolveRelation（DataSource.scala:359）
位于org.apache.spark.sql.DataFrameReader.loadV1Source（DataFrameReader.scala:223）
位于org.apache.spark.sql.DataFrameReader.load（DataFrameReader.scala:211）
位于org.apache.spark.sql.DataFrameReader.csv（DataFrameReader.scala:615）
位于org.apache.spark.sql.DataFrameReader.csv（DataFrameReader.scala:467）
位于com.expediagroup.egdp.mydemoapp.task.S3TaskRunner.getS3Data（S3TaskRunner.scala:24）
位于com.expediagroup.egdp.mydemoapp.task.S3TaskRunner.run（S3TaskRunner.scala:17）
位于com.expediagroup.egdp.mydemoapp.util.TaskInitializer.initialize（TaskInitializer.scala:22）
在com.expediagroup.egdp.mydemoapp.main$.main上（main.scala:30）
在com.expediagroup.egdp.mydemoapp.main.main（main.scala）上
原因：com.amazonaws.AmazonClientException:BasicAWSCredentialsProvider环境VariableCredentialsProvider InstanceProfileCredentialsProvider:com.amazonaws.sdkClientxception:无法从服务端点加载凭据
位于org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials（AWSCredentialProviderList.java:151）
在com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext（AmazonHttpClient.java:1166）上
在com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers上（AmazonHttpClient.java:762）
在com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute（AmazonHttpClient.java:724）
在com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer（AmazonHttpClient.java:717）
位于com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute（AmazonHttpClient.java:699）
位于com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500（AmazonHttpClient.java:667）
在com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute（AmazonHttpClient.java:649）上
在com.amazonaws.http.AmazonHttpClient.execute上（AmazonHttpClient.java:513）
位于com.amazonaws.services.s3.AmazonS3Client.invoke（AmazonS3Client.java:4365）
在com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest上（AmazonS3Client.java:5126）
位于com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache（AmazonS3Client.java:5100）
位于com.amazonaws.services.s3.AmazonS3Client.invoke（AmazonS3Client.java:4399）
位于com.amazonaws.services.s3.AmazonS3Client.invoke（AmazonS3Client.java:4312）
位于com.amazonaws.services.s3.AmazonS3Client.headBucket（AmazonS3Client.java:1343）
位于com.amazonaws.services.s3.AmazonS3Client.doesBucketExist（AmazonS3Client.java:1283）
位于org.apache.hadoop.fs.s3a.S3AFileSystem.verifyBucketExists（S3AFileSystem.java:320）
... 25多
原因：com.amazonaws.SdkClientException:无法从服务端点加载凭据
位于com.amazonaws.auth.EC2CredentialsFetcher.handleError（EC2CredentialsFetcher.java:183）
访问com.amazonaws.auth.EC2CredentialsFetcher.fetchCredentials（EC2CredentialsFetcher.java:162）
访问com.amazonaws.auth.EC2CredentialsFetcher.getCredentials（EC2CredentialsFetcher.java:82）
在com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials（InstanceProfileCredentialsProvider.java:164）上
位于org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials（AWSCredentialProviderList.java:129）
... 41多
原因：java.net.SocketTimeoutException:连接超时
位于java.net.PlainSocketImpl.socketConnect（本机方法）
位于java.net.AbstractPlainSocketImpl.doConnect（AbstractPlainSocketImpl.java:350）
位于java.net.AbstractPlainSocketImpl.connectToAddress（AbstractPlainSocketImpl.java:206）
位于java.net.AbstractPlainSocketImpl.connect（AbstractPlainSocketImpl.java:188）
位于java.net.socksocketimpl.connect（socksocketimpl.java:392）
位于java.net.Socket.connect（Socket.java:589）
位于sun.net.NetworkClient.doConnect（NetworkClient.java:175）
位于sun.net.www.http.HttpClient.openServer（HttpClient.java:432）
位于sun.net.www.http.HttpCli