Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/google-cloud-platform/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 使用';初始化群集时指定连接器版本;集群。创建';美国石油学会_Python_Google Cloud Platform_Google Cloud Dataproc - Fatal编程技术网

Python 使用';初始化群集时指定连接器版本;集群。创建';美国石油学会

Python 使用';初始化群集时指定连接器版本;集群。创建';美国石油学会,python,google-cloud-platform,google-cloud-dataproc,Python,Google Cloud Platform,Google Cloud Dataproc,我正在使用Python中的API在Dataproc中创建集群 { "projectId": "my-project-id", "clusterName": "example-cluster", "config": { "configBucket": "", "gceClusterConfig": { "subnetworkUri": "default", "zoneUri": "us-central1-b" }, "masterConfig": { "numInstances"

我正在使用Python中的API在Dataproc中创建集群

{
 "projectId": "my-project-id",
 "clusterName": "example-cluster",
 "config": {
"configBucket": "",
"gceClusterConfig": {
  "subnetworkUri": "default",
  "zoneUri": "us-central1-b"
},
"masterConfig": {
  "numInstances": 1,
  "machineTypeUri": "n1-standard-4",
  "diskConfig": {
    "bootDiskSizeGb": 500,
    "numLocalSsds": 0
  }
},
"workerConfig": {
  "numInstances": 2,
  "machineTypeUri": "n1-standard-4",
  "diskConfig": {
    "bootDiskSizeGb": 500,
    "numLocalSsds": 0
  }
},
"initializationActions": [
  {
    "executableFile": "gs://cloud-example-bucket/my-init-action.sh"
   }
  ]
 }
 }
在gcloud shell中,连接器版本和可执行文件指定为:

gcloud dataproc clusters create <CLUSTER_NAME> \
    --initialization-actions gs://dataproc-initialization- 
    actions/connectors/connectors.sh \
    --metadata 'gcs-connector-version=1.7.0' \
    --metadata 'bigquery-connector-version=0.11.0'
gcloud dataproc集群创建\
--初始化操作gs://dataproc初始化-
操作/连接器/连接器.sh\
--元数据“gcs连接器版本=1.7.0”\
--元数据“bigquery连接器版本=0.11.0”
如何将连接器版本(元数据下)传递给API

在不传递版本的情况下运行我的脚本会出现以下错误:

错误:未指定任何连接器版本'
错误:未指定任何连接器版本

+退出1

可以在config/gceClusterConfig下指定元数据字段,如下所示:

'config': {
        'gceClusterConfig': {
             "metadata": {
                 "bigquery-connector-version": "0.12.1",
                 "gcs-connector-version": "1.8.1"
                         }
                       }
          }