Amazon dynamodb AWS数据管道S3 CSV到DynamoDB JSON错误

我试图用AWS数据管道插入S3目录中的几个csv，但是，我犯了这个错误

javax.security.auth.Subject.doAs（Subject.java:422）org.apache.hadoop.security.UserGroupInformation.doAs（UserGroupInformation.java:1844）org.apache.hadoop.mapred.YarnChild.main（YarnChild.java:169）由com.google.gson.stream.MalformedJSONReader.syntaxError第1行第10列的异常：预期“：”引起（JsonReader.java:1505）在com.google.gson.stream.JsonReader.doPeek（JsonReader.java:519）在com.google.gson.stream.JsonReader.peek（JsonReader.java:414）在com.google.gson.internal.bind.ReflectTypeAdapterFactory$Adapter.read（ReflectTypeAdapterFactory.java:157）在com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read（TypeAdapterRuntimeTypeWrapper.java:40）在com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read（MapTypeAdapterFactory.java:187）在com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read（MapTypeAdapterFactory.java:145）在com.google.gson.gson.gson.fromJson（gson.java:803）…线程“main”java.io中还有15个异常。 错误堆栈跟踪

---

amazonaws.datapipeline.taskrunner.TaskExecutionException:未能在amazonaws.datapipeline.activity.EmrActivity.runActivity（EmrActivity.java:67）在amazonaws.datapipeline.objects.AbstractActivity.run（AbstractActivity.java:16）在amazonaws.datapipeline.taskrunner.TaskPoller.ExecuterMemorRunner（TaskPoller.java:136）完成EMR转换在amazonaws.datapipeline.taskrunner.TaskPoller.executeTask（TaskPoller.java:105）在amazonaws.datapipeline.taskrunner.TaskPoller$1.run（TaskPoller.java:81）在private.com.amazonaws.services.datapipeline.poller.PollWorker.executeWork（PollWorker.java:76）在private.com.amazonaws.services.datapipeline.poller.PollWorker.run（PollWorker.java:53）上在java.lang.Thread.run（Thread.java:748）上，由以下原因引起：amazonaws.datapipeline.taskrunner.TaskExecutionException:at javax.security.auth.Subject.doAs（Subject.java:422）在org.apache.hadoop.security.UserGroupInformation.doAs（UserGroupInformation.java:1844）在org.apache.hadoop.mapred.YarnChild.main（YarnChild.java:169）上由以下原因引起：com.google.gson.stream.MalformedJsonException:在com.google.gson.stream.JsonReader.syntaxError（JsonReader.java:1505）在com.google.gson.stream.JsonReader.doPeek（JsonReader.java:519）在com.google.gson.stream.JsonReader.peek（JsonReader.java:414）的第1行第10列应为“：”com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read（ReflectiveTypeAdapterFactory.java:157）com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read（TypeAdapterRuntimeTypeWrapper.java:40）com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read（MapTypeAdapterFactory.java:187）com.google.gson.gson.internal.bind.MapTypeAdapterFactory$Adapter.read（MapTypeAdapterFactory.java:145）com.google.gson.gson.fromJson（gson.java:803）…线程“main”java.io.IOException:Job失败！org.apache.hadoop.mapred.JobClient.runjobclient（JobClient.java:873）org.apache.hadoop.dynamodb.tools.DynamoDBImport.run（DynamoDBImport.java:81）在org.apache.hadoop.util.ToolRunner.run（ToolRunner.java:76）在org.apache.hadoop.dynamodb.tools.DynamoDBImport.main（DynamoDBImport.java:43）在sun.reflect.NativeMethodAccessorImpl.invoke0（本机方法）在sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl.java:62）在amazonaws.datapipeline.cluster.EmrUtil.runSteps（EmrUtil.java:286）上的java.lang.reflect.Method.invoke（Method.java:498）上的sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:43）上在amazonaws.datapipeline.activity.EmrActivity.runActivity（EmrActivity.java:63）…还有7个

这解决了我的问题

AWS数据管道使用的格式

{"Name": {"S":"Amazon push"},"Category": {"S":"Amazon Web Services"}}
{"Name": {"S":"Amazon S3"},"Category": {"S":"Amazon Web Services"}}```

References:

https://calorious.wordpress.com/2016/03/18/episode-4-importing-json-into-dynamodb/

https://medium.com/@ashleywnj/appsync-s3-data-pipeline-dynamodb-854f99d70b41