有没有一种方法可以使用Python将bigquery返回的结果转换为Json格式?

有没有一种方法可以使用Python将bigquery返回的结果转换为Json格式?,json,python-3.x,google-bigquery,Json,Python 3.x,Google Bigquery,目前使用python从bigquery中拉取数据,得到结果后需要将其转换为JSON格式,如何将结果转换为JSON query_job2 = client_bq.query(query) query_job2.result() rows = list(query_job2.result()) # Waits for query to finish response = dict() """ Creating a nested

目前使用python从bigquery中拉取数据,得到结果后需要将其转换为JSON格式,如何将结果转换为JSON

 query_job2 = client_bq.query(query)
    query_job2.result()
    rows = list(query_job2.result())  # Waits for query to finish
    response = dict()
    """
    Creating a nested dictionary with the tables as the keys and inside each respective table will hold cost as keys and
    have a list of values
    """
    for row in rows:
        table = get_table_name(str(row.query))
        start_time =int(row.start_time.timestamp())
        end_time =int(row.end_time.timestamp())

        if table in response:
            if row.cost in response[table]:
                response[table] = list(response[table])
                response[table].append((str(row.creation_time),start_time,end_time , row.cost, str(row.query)))
        else:
            response[table] = {}
            response[table] = (str(row.creation_time), start_time,end_time, row.cost, str(row.query))
以下是我正在使用的查询:

     with data AS (
       SELECT
       creation_time,
       total_bytes_processed,
       query
       FROM `project.region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT`
       where creation_time > TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -60 SECOND)  AND job_type = "QUERY"
       Group BY creation_time,  job_id, total_bytes_processed, query
       ORDER BY total_bytes_processed DESC
)
select as value
  array_agg(struct( creation_time,
       regexp_extract(query, r'(?i)\sfrom\s+`?(?:[\w-]+\.)*([\w-]+\.[\w-]+)`?\s' ) as table,
       (total_bytes_processed/1099511627776) * 5 as cost,
       query) order by (total_bytes_processed/1099511627776) * 5  desc limit 1)[offset(0)]
from data
group by timestamp_trunc(creation_time, minute)
    """

问题可分为两部分:

  • 如何将数据从Google BigQuery读取到Pandas。你可以找到答案

    从google.cloud导入bigquery
    client=bigquery.client()
    query_sql=“”从“表”中选择列“
    df=client.query(query\u sql).to\u dataframe()
    
  • 获取pandas对象中的数据后,可以使用
    to_json
    方法将其转换为json字符串:

    df.to_json(orient='index')
    
  • 或者,您可以直接从BigQuery将数据导出为JSON文件,如图所示,可以从UI导出,也可以使用python客户端导出


    如果您需要使用pandas操作内存中的数据,那么第一种方法更好,第二种方法将允许您以更大的规模序列化数据。

    这个问题的措辞可以更好。您正在询问如何将pandas.DataFrame格式化为json。您可以通过调用df.to_json(orient=..)来实现。它与BigQuery无关,因为您可以从CSV中等效读取数据。@gidutz真的很有趣,我想也许我可以直接从查询或其他内容中获取结果,或者将row.iterator对象转换为字典或其他内容