Python KeyError:'_id'；严重（'；异常写入记录'；，键错误（'；#u id'；））Singer.io

我在从mongo tap到redshift执行ELT时出错。当我简单地将数据从MongoDB提取到CSV时，效果很好。但是当把输出输出到redshift时，我得到了这个id错误。不知道原因是什么。有人能指引我吗 更多细节。因为它是用python编写的，所以也需要python社区的帮助

我已通过说明设置了tap mongo。我正在运行此命令以导出

 ~/.virtualenvs/tap-mongodb/bin/tap-mongodb --config ~/Documents/tap/config.json --catalog ~/Documents/tap/catalog.json |  ~/.virtualenvs/target-redshift/bin/target-redshift -c ~/Documents/target/config-redshift.json

运行此命令会导致此错误

ERROR Exception writing records
Traceback (most recent call last):
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/postgres.py", line 300, in write_batch
    {'version': target_table_version})
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/sql_base.py", line 808, in write_batch_helper
    for table_batch in denest.to_table_batches(schema, key_properties, records):
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/denest.py", line 21, in to_table_batches
    key_properties)
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/denest.py", line 47, in _get_streamed_table_schemas
    key_prop_schemas[key] = schema['properties'][key]
KeyError: '_id'
CRITICAL ('Exception writing records', KeyError('_id'))
Traceback (most recent call last):
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/postgres.py", line 300, in write_batch
    {'version': target_table_version})
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/sql_base.py", line 808, in write_batch_helper
    for table_batch in denest.to_table_batches(schema, key_properties, records):
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/denest.py", line 21, in to_table_batches
    key_properties)
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/denest.py", line 47, in _get_streamed_table_schemas
    key_prop_schemas[key] = schema['properties'][key]
KeyError: '_id'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/macbookpro/.virtualenvs/target-redshift/bin/target-redshift", line 10, in <module>
    sys.exit(cli())
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_redshift/__init__.py", line 55, in cli
    main(args.config)
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_redshift/__init__.py", line 49, in main
    target_tools.main(redshift_target)
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/target_tools.py", line 28, in main
    stream_to_target(input_stream, target, config=config)
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/target_tools.py", line 77, in stream_to_target
    raise e
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/target_tools.py", line 64, in stream_to_target
    line
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/target_tools.py", line 152, in _line_handler
    state_tracker.flush_stream(line_data['stream'])
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/stream_tracker.py", line 41, in flush_stream
    self._write_batch_and_update_watermarks(stream)
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/stream_tracker.py", line 67, in _write_batch_and_update_watermarks
    self.target.write_batch(stream_buffer)
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_redshift/redshift.py", line 72, in write_batch
    return PostgresTarget.write_batch(self, nullable_stream_buffer)
  File "/Users/macbookpro/.virtualenvs/target-redshift/lib/python3.7/site-packages/target_postgres/postgres.py", line 309, in write_batch
    raise PostgresError(message, ex)
target_postgres.exceptions.PostgresError: ('Exception writing records', KeyError('_id'))

mongo catalog.json

{
 "streams": [
{
  "table_name": "users",
  "stream": "users",
  "metadata": [
    {
      "breadcrumb": [],
      "metadata": 
       {
       "selected": true,
       "replication-method": "FULL_TABLE",
        "table-key-properties": [
          "_id"
        ],
        "database-name": "efuse",
        "row-count": 434,
        "is-view": false,
        "valid-replication-keys": [
          "_id"
        ]
      }
    }
  ],
  "tap_stream_id": "efuse-users",
  "schema": {
    "type": "object",
    "properties": {
      "_id": {
        "type": "object"
      }
    }
  }
 }
]

mongo config.json

{
 "password": "root",
 "user": "stitch_root",
 "host": "localhost",
 "port": 27017,
 "database": "admin",
 "ssl": "false",
 "replica_set": "rs0"
}

---

谢谢

请发布您的代码。你已经试过了。仅仅根据一条错误信息给你提供有效的答案几乎是不可能的。@Christopher我有一条更新指令。我在mongodb用户中有一个表，我想导出到redshift。如果你还需要什么，请告诉我it@Christopher ? 有什么建议吗？请张贴你的密码。你已经试过了。仅仅根据一条错误信息给你提供有效的答案几乎是不可能的。@Christopher我有一条更新指令。我在mongodb用户中有一个表，我想导出到redshift。如果你还需要什么，请告诉我it@Christopher ? 有什么建议吗？

{
 "password": "root",
 "user": "stitch_root",
 "host": "localhost",
 "port": 27017,
 "database": "admin",
 "ssl": "false",
 "replica_set": "rs0"
}