如何使用python从s3 bucket中的文件夹结构中读取文件

我是python新手，正在尝试了解如何使用python访问S3 bucket文件夹结构中的文件。我是否应该将bucket\u key=path/folder/file指定为这样的值？请帮忙

我主要是想从csv文件中获取行数。但我在读取文件时出错

import os
import sys
import string
import urllib
import urllib2
import boto
import boto.cloudformation
import boto.exception       
import boto.sns
import logging
from boto.s3.key import Key
from boto.s3.connection import S3Connection

#?pass this as a parameter?
bucket_name = "reporting"
bucket_key = "/compliance/testfile.csv"


def read_contents(bucket_key):
    # connect to the bucket
   conn = boto.connect_s3()
bucket = conn.get_bucket(bucket_name)
key = bucket_key  
# create a key to keep track 
k = Key(bucket)
k.key=key
testfile = k.get_contents_as_string()
return testfile


test = read_contents(bucket_key)
print test

使用安全操作实现对S3的高级访问 大家好，让我们定义一些从AWS S3访问和下载文件的功能。 首先，你必须连接到亚马逊

连接到S3 conf对象引用包含AWS凭据的字典。 一旦你做到了这一点，我们就可以专注于下载步骤

从S3下载一个文件 下载文件以文件名为参数，dirs3对应于文件的完整路径，您还可以设置输出路径、buckets3的名称，最后，您必须提供包含AWS凭据的字典。 因此，这个函数确定您的文件路径，连接到AmazonS3，获取想要的bucket，然后下载您的文件

安全下载错误处理 如果您想从S3安全下载文件，只需使用以下功能：

from boto.exception import S3ResponseError

def safe_download_from_s3(filename, output_path, buckets3, dirs3, conf):
  """
  :param filename - str: filename
  :param dirs3 - str: full path to file
  :param output_path - str: output path
  :param buckets3 - str: bucket name
  :param conf - dict: contains AWS credentials
  """

  print('Trying to download file from s3, filename={}, output_path={}, dirs3={}, buckets3={}'.format(
    filename, output_path, dirs3, buckets3))
  try:
      download_file_s3(filename, dirs3, output_path, buckets3, conf)
      print('File downloaded successfully')
  except S3ResponseError as err:
      print('An S3ResponseError occurred while downloading, err={}'.format(err))
  except TypeError as err:
      print('A TypeError occurred while downloading, err={}'.format(err))
  except NameError as err:
      print('A NameError occurred while downloading, err={}'.format(err))
  except:
      print('Unexpected error, exec_info={}'.format(sys.exc_info()[0]))

在本例中，conf是如下所示的字典：

conf = {'AWS_ACCESS_KEY_ID':'<your_aws_access_key_id>',
        'AWS_SECRET_ACCESS_KEY':'<your_aws_secret_access>'}

这就是我向你推荐的

---

我希望这有帮助。如果您有问题，欢迎使用。

是的，S3键是文件的完整路径。我尝试了bucket name/foldername/fileInFolder。但是我没有运气。访问文件时出错。显示代码。您使用的是什么版本的Boto/AWS SDK？我现在已经在帖子中找到了代码。错误消息是什么？

from boto.exception import S3ResponseError

def safe_download_from_s3(filename, output_path, buckets3, dirs3, conf):
  """
  :param filename - str: filename
  :param dirs3 - str: full path to file
  :param output_path - str: output path
  :param buckets3 - str: bucket name
  :param conf - dict: contains AWS credentials
  """

  print('Trying to download file from s3, filename={}, output_path={}, dirs3={}, buckets3={}'.format(
    filename, output_path, dirs3, buckets3))
  try:
      download_file_s3(filename, dirs3, output_path, buckets3, conf)
      print('File downloaded successfully')
  except S3ResponseError as err:
      print('An S3ResponseError occurred while downloading, err={}'.format(err))
  except TypeError as err:
      print('A TypeError occurred while downloading, err={}'.format(err))
  except NameError as err:
      print('A NameError occurred while downloading, err={}'.format(err))
  except:
      print('Unexpected error, exec_info={}'.format(sys.exc_info()[0]))

conf = {'AWS_ACCESS_KEY_ID':'<your_aws_access_key_id>',
        'AWS_SECRET_ACCESS_KEY':'<your_aws_secret_access>'}

export AWS_ACCESS_KEY_ID=<your_aws_access_key_id>
export AWS_SECRET_ACCESS_KEY=<your_aws_secret_access>

import os
conf=os.environ