Google bigquery 如何对Bigquery查询的结果进行分页
根据文档,您可以通过定义特定的表对结果进行分页。但是在查询中添加分页怎么样?例如,如果我有以下查询:Google bigquery 如何对Bigquery查询的结果进行分页,google-bigquery,Google Bigquery,根据文档,您可以通过定义特定的表对结果进行分页。但是在查询中添加分页怎么样?例如,如果我有以下查询: client = bigquery.Client(location='US') job_config = bigquery.QueryJobConfig() job_config.query_parameters = params result = client.query(query, job_config=job_config) 如何对此查询进行分页以获得从
client = bigquery.Client(location='US')
job_config = bigquery.QueryJobConfig()
job_config.query_parameters = params
result = client.query(query, job_config=job_config)
如何对此查询进行分页以获得从10到20的行?您没有发布查询,但我猜您正在寻找您的意思是将限制和偏移量作为参数传递给查询吗
from google.cloud import bigquery
client = bigquery.Client(project="project")
query = """SELECT name FROM `dataset` LIMIT @limit OFFSET @offset"""
limit = 10
offset = 11
params=[
bigquery.ScalarQueryParameter('limit', 'INT64', limit), bigquery.ScalarQueryParameter('offset', 'INT64', offset)
]
job_config=bigquery.QueryJobConfig()
job_config.query_parameters=params
query_job=client.query(query,job_config=job_config)
for row in query_job:
print("{}".format(row.name))
您可以通过两种方式使用大查询作业来实现它
String query="big query here...";
int startIndex=10;
int maxResults=10;
//fetches rows numbered 10 to 20 from result set
resultCollection = getPaginatedResultCollection(query, startIndex,maxResults);
//NOTE: Do what you want to do with paged data result i.e. resultCollection
/**
* Polls the status of a BigQuery job, returns TableReference to results if
* "DONE"
*/
private static TableReference checkQueryResults(Bigquery bigquery, String projectId, JobReference jobId) throws IOException, InterruptedException {
// Variables to keep track of total query time
while (true) {
Job pollJob = bigquery.jobs().get(projectId, jobId.getJobId()).execute();
if (pollJob.getStatus().getState().equals("DONE")) {
return pollJob.getConfiguration().getQuery().getDestinationTable();
}
// Pause execution for one second before polling job status again,
// to
// reduce unnecessary calls to the BigQUery API and lower overall
// application bandwidth.
// Thread.sleep(1000);
}
}
/**
* @param bigquery
* @param completedJob
* @param startIndex
* @param maxResultsPerPage
* @return
* @throws Exception
*/
private static ResultCollection displayQueryResults(Bigquery bigquery, TableReference completedJob, int startIndex, Integer maxResultsPerPage) throws Exception {
maxResultsPerPage = (maxResultsPerPage==null)? 20:maxResultsPerPage;
JSONObject responseMap = new JSONObject();
List<JSONObject> resultArray = new ArrayList<JSONObject>();
TableDataList queryResult = null;
queryResult = bigquery.tabledata().list(completedJob.getProjectId(), completedJob.getDatasetId(), completedJob.getTableId())
.setMaxResults(new Long(maxResultsPerPage))
.setStartIndex(BigInteger.valueOf(startIndex))
.execute();
//Table table = bigquery.tables().get(completedJob.getProjectId(), completedJob.getDatasetId(), completedJob.getTableId()).execute();
//NOTE: Schema can be read from table.getSchema().getFields()
if (CollectionUtils.isNotEmpty(queryResult.getRows())) {
//NOTE: read result data from queryResult.getRows() and transform the way you want to get them modeled, say resultCollection, for now
}
return resultCollection;
}
String query=“此处大查询…”;
int startIndex=10;
int maxResults=10;
//从结果集中获取编号为10到20的行
resultCollection=getPaginatedResultCollection(查询、startIndex、maxResults);
//注意:对分页数据结果执行您想要执行的操作,即resultCollection
/**
*轮询BigQuery作业的状态,如果
*“完成”
*/
私有静态TableReference checkQueryResults(Bigquery Bigquery、字符串projectId、JobReference jobId)引发IOException、InterruptedException{
//用于跟踪总查询时间的变量
while(true){
Job pollJob=bigquery.jobs().get(projectId,jobId.getJobId()).execute();
if(pollJob.getStatus().getState().equals(“DONE”)){
返回pollJob.getConfiguration().getQuery().getDestinationTable();
}
//暂停执行一秒钟,然后再次轮询作业状态,
//到
//减少对BigQUery API的不必要调用,并降低总体
//应用程序带宽。
//睡眠(1000);
}
}
/**
*@param bigquery
*@param completedJob
*@param startIndex
*@param maxResultsPerPage
*@返回
*@抛出异常
*/
私有静态ResultCollection displayQueryResults(Bigquery Bigquery、TableReference completedJob、int startIndex、Integer maxResultsPerPage)引发异常{
maxResultsPerPage=(maxResultsPerPage==null)?20:maxResultsPerPage;
JSONObject responseMap=新的JSONObject();
List resultArray=new ArrayList();
TableDataList queryResult=null;
queryResult=bigquery.tabledata().list(completedJob.getProjectId(),completedJob.getDatasetId(),completedJob.getTableId())
.setMaxResults(新长(maxResultsPerPage))
.setStartIndex(BigInteger.valueOf(startIndex))
.execute();
//Table Table=bigquery.tables().get(completedJob.getProjectId(),completedJob.getDatasetId(),completedJob.getTableId()).execute();
//注意:可以从表.getSchema().getFields()中读取架构
if(CollectionUtils.isNotEmpty(queryResult.getRows()){
//注意:从queryResult.getRows()中读取结果数据,并转换您想要对其建模的方式,比如现在的resultCollection
}
返回结果集合;
}
oh。。好的,谢谢,但我想知道是否有某种参数或方法可以通过编程方式添加,比如start\u index=10,max\u results=10