Amazon web services 无法从笔记本实例读取S3中的TFR记录
我尝试从Sage Maker笔记本实例中读取S3中的tfrecords,如下所示: 上述代码失败,出现错误:Amazon web services 无法从笔记本实例读取S3中的TFR记录,amazon-web-services,tensorflow,amazon-s3,amazon-sagemaker,Amazon Web Services,Tensorflow,Amazon S3,Amazon Sagemaker,我尝试从Sage Maker笔记本实例中读取S3中的tfrecords,如下所示: 上述代码失败,出现错误: --------------------------------------------------------------------------- NotFoundError Traceback (most recent call last) <ipython-input-7-770c0aef6d7b> in <
---------------------------------------------------------------------------
NotFoundError Traceback (most recent call last)
<ipython-input-7-770c0aef6d7b> in <module>()
1 from tensorflow.python.lib.io import file_io
----> 2 print(file_io.stat('s3://<my-bucket>/data/DEMO-mnist/train.tfrecords'))
~/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/lib/io/file_io.py in stat(filename)
551 with errors.raise_exception_on_not_ok_status() as status:
552 pywrap_tensorflow.Stat(compat.as_bytes(filename), file_statistics, status)
--> 553 return file_statistics
~/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py in __exit__(self, type_arg, value_arg, traceback_arg)
517 None, None,
518 compat.as_text(c_api.TF_Message(self.status.status)),
--> 519 c_api.TF_GetCode(self.status.status))
520 # Delete the underlying status object from memory otherwise it stays alive
521 # as there is a reference to status from this from the traceback due to
NotFoundError: Object s3://<my-bucket>/data/DEMO-mnist/train.tfrecords does not exist
但是,如果我在不使用SageMaker的情况下从常规EC2实例运行,同样的代码也可以正常工作
笔记本实例使用的IAM角色具有完全的S3访问权限。我在us-west-2中重现了这个问题 但是在我手动导出环境变量AWS_REGION='us-west-2'之后,它就工作了 此外,我还尝试不出口AWS_地区,并在us-east-1桶上进行了测试。它也起作用了
因此,由于某些原因,aws配置文件中的区域信息无法检索和使用。如果未使用环境变量AWS_REGION,它将始终为us-east-1,默认值。您是否检查了S3 bucket和notebook实例是否在同一区域?您还可以粘贴附加了执行角色的策略吗
---------------------------------------------------------------------------
NotFoundError Traceback (most recent call last)
<ipython-input-7-770c0aef6d7b> in <module>()
1 from tensorflow.python.lib.io import file_io
----> 2 print(file_io.stat('s3://<my-bucket>/data/DEMO-mnist/train.tfrecords'))
~/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/lib/io/file_io.py in stat(filename)
551 with errors.raise_exception_on_not_ok_status() as status:
552 pywrap_tensorflow.Stat(compat.as_bytes(filename), file_statistics, status)
--> 553 return file_statistics
~/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py in __exit__(self, type_arg, value_arg, traceback_arg)
517 None, None,
518 compat.as_text(c_api.TF_Message(self.status.status)),
--> 519 c_api.TF_GetCode(self.status.status))
520 # Delete the underlying status object from memory otherwise it stays alive
521 # as there is a reference to status from this from the traceback due to
NotFoundError: Object s3://<my-bucket>/data/DEMO-mnist/train.tfrecords does not exist