Javascript 节点-创建没有数据的索引
我一直在开发一个AWS Lambda函数,每当S3 Bucket从Cloudfront接收日志时,它都会对S3 Bucket发送的通知做出反应。 我能够解压日志包,并用 但是,我仍然无法将它们发送到ElasticSearch端点。我主要与和进行tr,但它们都不起作用 这是我的密码:Javascript 节点-创建没有数据的索引,javascript,node.js,
elasticsearch,Javascript,Node.js,
elasticsearch,我一直在开发一个AWS Lambda函数,每当S3 Bucket从Cloudfront接收日志时,它都会对S3 Bucket发送的通知做出反应。 我能够解压日志包,并用 但是,我仍然无法将它们发送到ElasticSearch端点。我主要与和进行tr,但它们都不起作用 这是我的密码: exports.handler = function(event, context, callback) { var srcBucket = event.Records[0].s3.bucket.name;
exports.handler = function(event, context, callback) {
var srcBucket = event.Records[0].s3.bucket.name;
var srcKey = event.Records[0].s3.object.key;
async.waterfall([
function fetchLogFromS3(next){
console.log('Fetching compressed log from S3...');
s3.getObject({
Bucket: srcBucket,
Key: srcKey
},
next);
},
function uncompressLog(response, next){
console.log("Uncompressing log...");
zlib.gunzip(response.Body, next);
},
function publishNotifications(jsonBuffer, next) {
console.log('Filtering log...');
var json = jsonBuffer.toString();
console.log('CloudFront JSON from S3:', json);
var records;
CloudFrontParser.parse(json, { format: 'web' }, function (err, accesses) {
if(err){
console.log(err);
} else {
records = accesses;
}
});
// Here, how to send the parsed data?
console.log('CloudFront parsed:', records);
}
], function (err) {
if (err) {
console.error('Failed to send data: ', err);
} else {
console.log('Successfully send data.');
}
callback(null,"message");
});
};
有没有简单的方法将数据发送到ES比如:
var client = new elasticsearch.Client({
host: process.env.ES_HOST,
log: 'trace',
keepAlive: false
});
client.index({
index: 'cloudfront_index',
type: 'log',
body: records
}, function(err, resp, status) {
console.log(resp);
});
它可以工作,但不发送数据:
GET cloudfront_index/_search
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}
你快到了。您需要使用来实现您想要的:
var client = new elasticsearch.Client({
host: process.env.ES_HOST,
log: 'trace',
keepAlive: false
});
var bulk = [];
records.forEach(function(record) {
bulk.push({"index": {}})
bulk.push(record);
});
client.bulk({
index: 'cloudfront_index',
type: 'log',
body: bulk
}, function(err, resp, status) {
console.log(resp);
});
谢谢!工作完美:)太棒了,很高兴它有帮助!不要忘记在批量回调中调用
next
,尽管函数(err,resp,status){console.log(resp);next();})代码>?