Python 3.x CloudWatch将日志流记录到Lambda python
我在CloudWatch日志组中创建了一个订阅筛选器,并将其流式传输到lambda函数,但我的lambda函数中出现错误 代码:Python 3.x CloudWatch将日志流记录到Lambda python,python-3.x,aws-lambda,decode,amazon-cloudwatchlogs,Python 3.x,Aws Lambda,Decode,Amazon Cloudwatchlogs,我在CloudWatch日志组中创建了一个订阅筛选器,并将其流式传输到lambda函数,但我的lambda函数中出现错误 代码: import boto3 import binascii import json import base64 import zlib def stream_gzip_decompress(stream): dec = zlib.decompressobj(32 + zlib.MAX_WBITS) # offset 32 to skip the header
import boto3
import binascii
import json
import base64
import zlib
def stream_gzip_decompress(stream):
dec = zlib.decompressobj(32 + zlib.MAX_WBITS) # offset 32 to skip the header
foo=''
for chunk in stream:
rv = dec.decompress(chunk)
if rv:
foo += rv
return foo
def lambda_handler(event, context):
# Decode and decompress the AWS Log stream to extract json object
stream=json.dumps(event['awslogs']['data'])
f = base64.b64decode(stream)
payload=json.loads(stream_gzip_decompress(f.decode(f)))
print(payload)
错误:
import boto3
import binascii
import json
import base64
import zlib
def stream_gzip_decompress(stream):
dec = zlib.decompressobj(32 + zlib.MAX_WBITS) # offset 32 to skip the header
foo=''
for chunk in stream:
rv = dec.decompress(chunk)
if rv:
foo += rv
return foo
def lambda_handler(event, context):
# Decode and decompress the AWS Log stream to extract json object
stream=json.dumps(event['awslogs']['data'])
f = base64.b64decode(stream)
payload=json.loads(stream_gzip_decompress(f.decode(f)))
print(payload)
答复:
{
"errorMessage": "decode() argument 1 must be str, not bytes",
"errorType": "TypeError",
"stackTrace": [
[
"/var/task/lambda_function.py",
34,
"lambda_handler",
"payload=json.loads(stream_gzip_decompress(f.decode(f)))"
]
]
}
任何帮助或线索都将不胜感激!如果您有其他解决方案,请提出建议。我的要求是使用lambda处理来自CloudWatch的日志
提前谢谢 下面是我在处理发送给AWS Lambda的CloudWatch日志时通常遵循的大纲
import gzip
import json
from StringIO import StringIO
def lambda_handler(event, context):
cw_data = str(event['awslogs']['data'])
cw_logs = gzip.GzipFile(fileobj=StringIO(cw_data.decode('base64', 'strict'))).read()
log_events = json.loads(cw_logs)
for log_event in logevents['logEvents']:
# Process Logs
我看到您将发送到AWS Lambda的数据视为JSON对象。首先要对数据进行base64解码,然后解压缩数据。解码和解压缩后,您应该拥有带有日志信息的JSON对象 以防其他人正在寻求有关此主题的帮助 我采取了一种稍微不同的方法,但我确实看到了一个“awslog”键 这是一个我很成功的例子。 Python3.6lambda。 设置cloudwatch触发器以调用lambda
import gzip
import json
import base64
def lambda_handler(event, context):
print(f'Logging Event: {event}')
print(f"Awslog: {event['awslogs']}")
cw_data = event['awslogs']['data']
print(f'data: {cw_data}')
print(f'type: {type(cw_data)}')
compressed_payload = base64.b64decode(cw_data)
uncompressed_payload = gzip.decompress(compressed_payload)
payload = json.loads(uncompressed_payload)
log_events = payload['logEvents']
for log_event in log_events:
print(f'LogEvent: {log_event}')
下面是quasar转换为Python 3的答案
import gzip
import json
import base64
from io import BytesIO
cw_data = str(event['awslogs']['data'])
cw_logs = gzip.GzipFile(fileobj=BytesIO(base64.b64decode(cw_data, validate=True))).read()
log_events = json.loads(cw_logs)
for log_event in log_events['logEvents']:
# Process Logs
主要的变化是使用io.BytesIO和另一个base64解码函数来获取日志事件数据。感谢您的回答,但是我得到了keyerror,因为dict类型“event”没有键“awslogs”,格式应该是
{awslogs':{data':}
。我假设str(event['awslogs']['data']
是一个打字错误,意思是str(event['awslogs']['data'])
我更喜欢这个答案,因为它更简单、更清晰。谢谢不错。但是,如何从这些日志的来源获取函数名呢?您的失败是因为您试图json.dumps(event['awslogs']['data'])
其中data
是基本的64编码gzip压缩列表数据。只需将事件['awslogs']['data']
数据直接传递给解码,如P.Ryan的回答所示