Google cloud platform 我有一个从云存储加载.CSV文件的GCP函数,我需要让它跳过添加除.CSV文件以外的任何内容
我的函数由云存储事件触发,将文件加载到BigQuery表中,我的问题是我们收到了一些同名的.zip文件,函数也试图加载这些文件,这导致表出现一些问题。我需要使代码只处理.csv文件。以下是我目前掌握的代码:Google cloud platform 我有一个从云存储加载.CSV文件的GCP函数,我需要让它跳过添加除.CSV文件以外的任何内容,google-cloud-platform,google-cloud-storage,google-cloud-functions,Google Cloud Platform,Google Cloud Storage,Google Cloud Functions,我的函数由云存储事件触发,将文件加载到BigQuery表中,我的问题是我们收到了一些同名的.zip文件,函数也试图加载这些文件,这导致表出现一些问题。我需要使代码只处理.csv文件。以下是我目前掌握的代码: exports.ToBigQuery = (event, callback) => { const file = event.data; const context = event.context; const BigQuery = require('@google-cl
exports.ToBigQuery = (event, callback) => {
const file = event.data;
const context = event.context;
const BigQuery = require('@google-cloud/bigquery');
const Storage = require('@google-cloud/storage');
const projectId = "gas-ddr";
const datasetId = "gas_ddr";
const bucketName = file.bucket;
const filename = file.name;
const dashOffset = filename.indexOf('-');
const tableId = filename.substring(0, dashOffset);
console.log(`Load ${filename} into ${tableId}.`);
// Instantiates clients
const bigquery = new BigQuery({
projectId: projectId,
});
const storage = Storage({
projectId: projectId,
});
const metadata = {
allowJaggedRows: true,
skipLeadingRows: 1
};
let job;
// Loads data from a Google Cloud Storage file into the table
bigquery
.dataset(datasetId)
.table(tableId)
.load(storage.bucket(bucketName).file(filename),metadata)
.then(results => {
job = results[0];
console.log(`Job ${job.id} started.`);
// Wait for the job to finish
return job;
})
.then(metadata => {
// Check the job's status for errors
const errors = metadata.status.errors;
if (errors && errors.length > 0) {
throw errors;
}
})
.then(() => {
console.log(`Job ${job.id} completed.`);
})
.catch(err => {
console.error('ERROR:', err);
});
callback();
};
这只是一个与javascript相关的问题。您只需提取文件名的扩展名部分并相应地处理文件:
function getExtension(filename) {
var parts = filename.split('.');
return parts[parts.length - 1];
}
if (getExtension(filename) == "csv") {
// Loads data from a Google Cloud Storage file into the table
bigquery
.dataset(datasetId)
...
}
这只是一个与javascript相关的问题。您只需提取文件名的扩展名部分并相应地处理文件:
function getExtension(filename) {
var parts = filename.split('.');
return parts[parts.length - 1];
}
if (getExtension(filename) == "csv") {
// Loads data from a Google Cloud Storage file into the table
bigquery
.dataset(datasetId)
...
}