Python 使用大查询API将数据摄取到按时间分区的表中,但得到SyntaxError:输入意外结束
我正在尝试将一个CSV文件加载到一个按月份分区的Bigquery表中 代码返回以下错误: google.api_core.exceptions.BadRequest:400语法错误:预期输入结束,但得到:在[17:24] 语法错误似乎指的是冒号,它是我试图插入到表中的URL字符串的一部分: https**:*//www.example.com 考虑到它只是字符串的一部分,这会提示错误,这似乎很奇怪 我需要以某种方式逃离结肠吗?如果是,怎么做 我的代码是:Python 使用大查询API将数据摄取到按时间分区的表中,但得到SyntaxError:输入意外结束,python,google-api,google-bigquery,Python,Google Api,Google Bigquery,我正在尝试将一个CSV文件加载到一个按月份分区的Bigquery表中 代码返回以下错误: google.api_core.exceptions.BadRequest:400语法错误:预期输入结束,但得到:在[17:24] 语法错误似乎指的是冒号,它是我试图插入到表中的URL字符串的一部分: https**:*//www.example.com 考虑到它只是字符串的一部分,这会提示错误,这似乎很奇怪 我需要以某种方式逃离结肠吗?如果是,怎么做 我的代码是: import pandas as pd
import pandas as pd
import pandas_gbq
from google.oauth2 import service_account
from google.cloud import storage
from google.cloud import bigquery
from datetime import datetime
query =
'''
INSERT INTO
<<project id>>.<<Dataset>>.<<table>>(_PARTITIONTIME,
url,
title,
h1
)
SELECT {},{},{},{}
'''
now = datetime.now().strftime('%Y-%m-%d')
def run():
client = \
storage.Client.from_service_account_json('<<path to file>>'
)
bq_client = \
bigquery.Client.from_service_account_json('<<path to file>>'
)
bucket = client.bucket('<<bucket name>>')
blobs = bucket.list_blobs()
list_temp_raw = []
for file in blobs:
filename = file.name
temp = pd.read_csv('gs://<<bucket name>>/' + filename)
list_temp_raw.append(temp)
df = pd.concat(list_temp_raw)
df = df[cols]
for i in range(len(df.head())):
**load_query = query.format(
now,
df.loc[i, 'url'],
df.iloc[i, 'title'],
df.loc[i, 'h1']
)
query_job = bq_client.query(load_query)**
query_job.result()
run()
不确定。。。但是,向查询添加时间戳或等待作业完成可能会有所帮助:
import pandas as pd
import pandas_gbq
from google.oauth2 import service_account
from google.cloud import storage
from google.cloud import bigquery
from datetime import datetime
query =
'''
INSERT INTO
<<project id>>.<<Dataset>>.<<table>>(_PARTITIONTIME,
a,
b,
c,
d,
)
SELECT TIMESTAMP("{}"),{},{},{},{}
'''
now = datetime.now().strftime('%Y-%m-%d')
def run():
client = \
storage.Client.from_service_account_json('<<path to file>>'
)
bq_client = \
bigquery.Client.from_service_account_json('<<path to file>>'
)
bucket = client.bucket('<<bucket name>>')
blobs = bucket.list_blobs()
list_temp_raw = []
for file in blobs:
filename = file.name
temp = pd.read_csv('gs://<<bucket name>>/' + filename)
list_temp_raw.append(temp)
df = pd.concat(list_temp_raw)
df = df[cols]
for i in range(len(df.head())):
load_query = query.format(
now,
df.loc[i, 'a'],
df.iloc[i, 'b'],
df.loc[i, 'c'],
df.loc[i, 'd']
)
query_job = bq_client.query(load_query)
query_job.result() # Wait for the job to complete.
run()
尝试添加括号,如SELECT{}、{}、{}、{}。另外,最好问一个新问题,因为旧的答案现在已经不适用了。这让其他人感到困惑。