Hadoop 使用Sqoop导出从基于S3的表到MySQL的Sqoop导出失败
我有一个基于s3的表Hadoop 使用Sqoop导出从基于S3的表到MySQL的Sqoop导出失败,hadoop,amazon-s3,hive,sqoop,amazon-emr,Hadoop,Amazon S3,Hive,Sqoop,Amazon Emr,我有一个基于s3的表 CREATE TABLE IF NOT EXISTS sales_by_location ( client_id STRING, client_name STRING, merchant_id STRING, merchant_name STRING, location_id STRING, location_name STRING, sales_date DATE, sales_hour
CREATE TABLE IF NOT EXISTS sales_by_location (
client_id STRING,
client_name STRING,
merchant_id STRING,
merchant_name STRING,
location_id STRING,
location_name STRING,
sales_date DATE,
sales_hour INT,
sales_hour_half INT,
total_sales INT
)
CLUSTERED BY (client_id) INTO 32 BUCKETS
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
STORED AS ORC
LOCATION 's3://reports-test/sales_by_location/'
TBLPROPERTIES(
'transactional'='true',
'orc.compress'='SNAPPY');
sqoop export --connect $MYSQL_REPORTS_DATABASE_JDBC_URL --username $MYSQL_DB_USER --password $MYSQL_DB_PASSWORD
--table sales_by_location --columns "clientId,clientName,merchantId,merchantName,locationId,locationName,salesDate,salesHour,salesHourHalf,totalSales"
--update-key clientId,clientName,merchantId,merchantName,locationId,locationName,salesDate,salesHour,salesHourHalf
--update-mode allowinsert --export-dir s3://reports-test/sales_by_location/ --input-fields-terminated-by ';' --batch
它说一些IO错误，但不清楚。一个简单的sqoop导出（不更新）是否有效？如果是，则sqoop进程正在工作，并且存在数据更新问题。你能检查一下键列吗？试试不同的组合。