Python 启动映射作业时遇到错误
启动地图作业时,我遇到此错误 下面是我如何启动map函数的 *处理程序代码:- 此代码读取设备健康模型的每一行,并为其准备CSV文件。 在完成对数据模型行的迭代之后,我们准备CSV文件并通过电子邮件发送给用户Python 启动映射作业时遇到错误,python,google-app-engine,mapreduce,Python,Google App Engine,Mapreduce,启动地图作业时,我遇到此错误 下面是我如何启动map函数的 *处理程序代码:- 此代码读取设备健康模型的每一行,并为其准备CSV文件。 在完成对数据模型行的迭代之后,我们准备CSV文件并通过电子邮件发送给用户 def process_health_logs(health_logs): ctx = context.get() params = ctx.mapreduce_spec.mapper.params to_date = params.get('to_date') from_
def process_health_logs(health_logs):
ctx = context.get()
params = ctx.mapreduce_spec.mapper.params
to_date = params.get('to_date')
from_date = params.get('from_date')
to_date = datetime.datetime.strptime(str(to_date),'%Y-%m-%d').date()
from_date = datetime.datetime.strptime(str(from_date),'%Y-%m-%d').date()
error_date = health_logs.error_date if health_logs.error_date else health_logs.added_date
date = error_date.date()
if date >= from_date and date <= to_date:
added_date = time_util.utc_to_local(error_date, health_logs.place.timezone)
device_id = health_logs.device_id
user = health_logs.user.email
place = health_logs.place.name
unique_device_id = health_logs.unique_device_id
error_type = health_logs.error_type
error_level = health_logs.error_level
error_description = health_logs.error_description
row_data=[
added_date,
device_id,
user,
place,
unique_device_id,
error_type,
error_level,
error_description
]
row_data = [gtools.str_or_empty(x).encode('utf-8').replace(',', ';') for x in row_data]
yield ','.join(row_data) + '\n'
def过程健康日志(健康日志):
ctx=context.get()
params=ctx.mapreduce\u spec.mapper.params
to_date=params.get('to_date')
from_date=params.get('from_date')
to_date=datetime.datetime.StrTime(str(to_date),“%Y-%m-%d”).date()
from_date=datetime.datetime.strtime(str(from_date),“%Y-%m-%d”).date()
error\u date=健康日志。error\u date if health\u logs.error\u date else health\u logs.added\u date
日期=错误\u日期。日期()
如果date>=from_date和date很可能您使用的是旧的cloudstorage模块,与mapreduce模块不兼容。请显示完整的回溯。它的存在是有原因的。我已经添加了完整的回溯,您可以将其从保留中删除吗。对此有任何看法吗?您可能还需要显示处理程序的代码。@Daniel Roseman请查看此内容。
control.start_map(name='Export Device Health Logs',
handler_spec='data_process.process_health_logs',
reader_spec='mapreduce.input_readers.DatastoreInputReader',
mapper_parameters={"input_reader":{'entity_kind': 'models.DeviceHealth','email':user.email,'to_date':to_date,'from_date':from_date},'output_writer': {'bucket_name': bucket_name,
'content_type': 'text/plain'}},
shard_count=1,
output_writer_spec='mapreduce.output_writers.GoogleCloudStorageConsistentOutputWriter',
mapreduce_parameters={'done_callback':'/tasks/mapreduce/done/export_health_data',
'done_callback_queue':'mapreduce-done'})
def process_health_logs(health_logs):
ctx = context.get()
params = ctx.mapreduce_spec.mapper.params
to_date = params.get('to_date')
from_date = params.get('from_date')
to_date = datetime.datetime.strptime(str(to_date),'%Y-%m-%d').date()
from_date = datetime.datetime.strptime(str(from_date),'%Y-%m-%d').date()
error_date = health_logs.error_date if health_logs.error_date else health_logs.added_date
date = error_date.date()
if date >= from_date and date <= to_date:
added_date = time_util.utc_to_local(error_date, health_logs.place.timezone)
device_id = health_logs.device_id
user = health_logs.user.email
place = health_logs.place.name
unique_device_id = health_logs.unique_device_id
error_type = health_logs.error_type
error_level = health_logs.error_level
error_description = health_logs.error_description
row_data=[
added_date,
device_id,
user,
place,
unique_device_id,
error_type,
error_level,
error_description
]
row_data = [gtools.str_or_empty(x).encode('utf-8').replace(',', ';') for x in row_data]
yield ','.join(row_data) + '\n'