如何从S3读取CSV文件_Csv_Go_Amazon S3_Aws Lambda

如何从S3读取CSV文件

csv go amazon-s3 aws-lambda

如何从S3读取CSV文件,csv,go,amazon-s3,aws-lambda,Csv,Go,Amazon S3,Aws Lambda,我有以下代码： package main import ( "encoding/csv" "fmt" "io/ioutil" "path" "github.com/aws/aws-lambda-go/events" "github.com/aws/aws-lambda-go/lambda" "github.com/aws/aws-sdk-go/aws" "github.com/aws/aws-sdk-go/aws/session

我有以下代码：

package main

import (
    "encoding/csv"
    "fmt"
    "io/ioutil"
    "path"

    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
    "github.com/aws/aws-sdk-go/service/s3/s3iface"
)

var (
    // TOKEN = os.Getenv("TOKEN")
    svc s3iface.S3API
)

func main() {    
    // Panics if there is an error in creating session
    svc = s3iface.S3API(s3.New(session.Must(session.NewSession())))

    lambda.Start(Handler)
}

func Handler(evt events.S3Event) error {

    for _, rec := range evt.Records {

        key := rec.S3.Object.Key

        dir, file := path.Split(key)
        // Download the file from S3
        obj, err := svc.GetObject(&s3.GetObjectInput{
            Bucket: aws.String(rec.S3.Bucket.Name),
            Key:    aws.String(key),
        })
        if err != nil {
            return fmt.Errorf("error in downloading %s from S3: %s\n", key, err)
        }

        body, err := ioutil.ReadAll(obj.Body)
        if err != nil {
            return fmt.Errorf("error in reading file %s: %s\n", key, err)
        }

        reader := csv.NewReader(body)
        record, err := reader.ReadAll()
        if err != nil {
            fmt.Println("Error", err)
        }

        for value := range record { // for i:=0; i<len(record)
            fmt.Println("", record[value])
        }
    }
    return nil
}

非常感谢您的建议，因为错误说明：

无法将正文类型[]字节用作io类型。要 csv.NewReader： []字节未实现io。读取器缺少读取方法

因为您正在将响应返回的[]字节传递给csv.NewReader

需要通过body实现io.Reader，才能将其作为参数传递给csv.NewReader。因为它以io.Reader作为参数。尝试将代码更改为：

reader := csv.NewReader(bytes.NewBuffer(body))
record, err := reader.ReadAll()
if err != nil {
    fmt.Println("Error", err)
}

也因为返回指向GetObjectOutput结构的指针

它本身实现了读卡器

type GetObjectOutput struct {
    ....
    // Object data.
    Body io.ReadCloser `type:"blob"`
    ....
}

因此，您可以将返回的正文直接传递给csv.NewReader

还有一件事，你可以选择下载管理器

s3manager包的下载程序提供并发下载来自S3的对象。下载程序将使用一个io.WriterAt。创建Downloader实例后，可以调用安全地从多个goroutine并发下载

Download在S3中下载一个对象，并使用并发GET请求将有效负载写入w

跨goroutine并发调用此方法是安全的

// The session the S3 Downloader will use
sess := session.Must(session.NewSession())

// Create a downloader with the session and default options
downloader := s3manager.NewDownloader(sess)

// Create a file to write the S3 Object contents to.
f, err := os.Create(filename)
if err != nil {
    return fmt.Errorf("failed to create file %q, %v", filename, err)
}

// Write the contents of S3 Object to the file
n, err := downloader.Download(f, &s3.GetObjectInput{
    Bucket: aws.String(myBucket),
    Key:    aws.String(myString),
})
if err != nil {
    return fmt.Errorf("failed to download file, %v", err)
}

正如错误所说：