Python &引用;私钥丢失或无效。应为服务和报价;使用凭据进行BigQuery调用时
我将遵循GCP AI平台的本教程。我尝试使用以下命令对BigQuery进行API调用:Python &引用;私钥丢失或无效。应为服务和报价;使用凭据进行BigQuery调用时,python,python-3.x,google-cloud-platform,google-bigquery,Python,Python 3.x,Google Cloud Platform,Google Bigquery,我将遵循GCP AI平台的本教程。我尝试使用以下命令对BigQuery进行API调用: def query_to_dataframe(query): import pandas as pd import pkgutil privatekey = pkgutil.get_data('trainer', 'privatekey.json') print(privatekey[:200]) return pd.read_gbq(query,
def query_to_dataframe(query):
import pandas as pd
import pkgutil
privatekey = pkgutil.get_data('trainer', 'privatekey.json')
print(privatekey[:200])
return pd.read_gbq(query,
project_id=PROJECT,
dialect='standard',
private_key=privatekey)
但出现以下错误:
Traceback (most recent call last):
[...]
TypeError: a bytes-like object is required, not 'str'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/root/.local/lib/python3.7/site-packages/trainer/task.py", line 66, in <module>
arguments['numTrees']
File "/root/.local/lib/python3.7/site-packages/trainer/model.py", line 119, in train_and_evaluate
train_df, eval_df = create_dataframes(frac)
File "/root/.local/lib/python3.7/site-packages/trainer/model.py", line 95, in create_dataframes
train_df = query_to_dataframe(train_query)
File "/root/.local/lib/python3.7/site-packages/trainer/model.py", line 82, in query_to_dataframe
private_key=privatekey)
File "/usr/local/lib/python3.7/dist-packages/pandas/io/gbq.py", line 149, in read_gbq
credentials=credentials, verbose=verbose, private_key=private_key)
File "/root/.local/lib/python3.7/site-packages/pandas_gbq/gbq.py", line 846, in read_gbq
dialect=dialect, auth_local_webserver=auth_local_webserver)
File "/root/.local/lib/python3.7/site-packages/pandas_gbq/gbq.py", line 184, in __init__
self.credentials = self.get_credentials()
File "/root/.local/lib/python3.7/site-packages/pandas_gbq/gbq.py", line 193, in get_credentials
return self.get_service_account_credentials()
File "/root/.local/lib/python3.7/site-packages/pandas_gbq/gbq.py", line 413, in get_service_account_credentials
"Private key is missing or invalid. It should be service "
pandas_gbq.gbq.InvalidPrivateKeyFormat: Private key is missing or invalid. It should be service account private key JSON (file path or string contents) with at least two keys: 'client_email' and 'private_key'. Can be obtained from: https://console.developers.google.com/permissions/serviceaccounts
我不确定问题是否是由于Python中的错误造成的,或者我将
privatekey.json
放置在何处。根据@rmestevs的建议,我更改了read\u gbq
的属性,以便将BigQuery访问密钥从private\u key
读取到凭据,如图所示,之后我就能够解决问题。然后,我将该值设置为privatekey.json
的绝对路径,如图所示。现在作业可以无误运行了
注意:我只在Python3+中遇到了这个问题,但在Python2.7中没有遇到。我不知道为什么。这可能是由于执行了read\u gbq
1)将字符串转换为字节:private\u key=privatekey.encode('utf-8'))
。2) 什么是privatekey.json?这是谷歌服务账户JSON密钥文件还是你创建的东西?我同意@JohnHanley的说法。要解决字节问题,需要将其编码为utf-8。为什么需要使用此私钥属性?也许你可以使用这些凭证attribute@JohnHanley是privatekey.json用于调用BigQuery API。@rmesteves根据您的建议,我将private\u keys
更改为credentials
属性,如图所示,然后构建privatekey.json的绝对路径。现在作业可以无误运行了。@mlo可以将其总结为一个答案,或者让我知道我是否可以这样做
/babyweight
- setup.py
- trainer
- __init__.py
- model.py
- privatekey.json
- task.py