有没有一种方法可以使用Python将bigquery返回的结果转换为Json格式?
目前使用python从bigquery中拉取数据,得到结果后需要将其转换为JSON格式,如何将结果转换为JSON有没有一种方法可以使用Python将bigquery返回的结果转换为Json格式?,json,python-3.x,google-bigquery,Json,Python 3.x,Google Bigquery,目前使用python从bigquery中拉取数据,得到结果后需要将其转换为JSON格式,如何将结果转换为JSON query_job2 = client_bq.query(query) query_job2.result() rows = list(query_job2.result()) # Waits for query to finish response = dict() """ Creating a nested
query_job2 = client_bq.query(query)
query_job2.result()
rows = list(query_job2.result()) # Waits for query to finish
response = dict()
"""
Creating a nested dictionary with the tables as the keys and inside each respective table will hold cost as keys and
have a list of values
"""
for row in rows:
table = get_table_name(str(row.query))
start_time =int(row.start_time.timestamp())
end_time =int(row.end_time.timestamp())
if table in response:
if row.cost in response[table]:
response[table] = list(response[table])
response[table].append((str(row.creation_time),start_time,end_time , row.cost, str(row.query)))
else:
response[table] = {}
response[table] = (str(row.creation_time), start_time,end_time, row.cost, str(row.query))
以下是我正在使用的查询:
with data AS (
SELECT
creation_time,
total_bytes_processed,
query
FROM `project.region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT`
where creation_time > TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -60 SECOND) AND job_type = "QUERY"
Group BY creation_time, job_id, total_bytes_processed, query
ORDER BY total_bytes_processed DESC
)
select as value
array_agg(struct( creation_time,
regexp_extract(query, r'(?i)\sfrom\s+`?(?:[\w-]+\.)*([\w-]+\.[\w-]+)`?\s' ) as table,
(total_bytes_processed/1099511627776) * 5 as cost,
query) order by (total_bytes_processed/1099511627776) * 5 desc limit 1)[offset(0)]
from data
group by timestamp_trunc(creation_time, minute)
"""
问题可分为两部分:
从google.cloud导入bigquery
client=bigquery.client()
query_sql=“”从“表”中选择列“
df=client.query(query\u sql).to\u dataframe()
to_json
方法将其转换为json字符串:
df.to_json(orient='index')
如果您需要使用pandas操作内存中的数据,那么第一种方法更好,第二种方法将允许您以更大的规模序列化数据。这个问题的措辞可以更好。您正在询问如何将pandas.DataFrame格式化为json。您可以通过调用df.to_json(orient=..)来实现。它与BigQuery无关,因为您可以从CSV中等效读取数据。@gidutz真的很有趣,我想也许我可以直接从查询或其他内容中获取结果,或者将row.iterator对象转换为字典或其他内容