Google cloud platform 调整节点配置代理和调度程序
今天,我尝试更改支持cloud composer环境的集群的节点类型,并切换到Ubuntu映像,而不是COS。为此,我向GKE集群添加了第二个节点池,然后删除了第一个节点池,并迁移了所有工作负载 这会在sqlproxy日志中产生以下错误:Google cloud platform 调整节点配置代理和调度程序,google-cloud-platform,google-cloud-composer,Google Cloud Platform,Google Cloud Composer,今天,我尝试更改支持cloud composer环境的集群的节点类型,并切换到Ubuntu映像,而不是COS。为此,我向GKE集群添加了第二个节点池,然后删除了第一个节点池,并迁移了所有工作负载 这会在sqlproxy日志中产生以下错误: couldn't connect to "XXXXX:europe-west1:XXXXX": ensure that the Cloud SQL API is enabled for your project (https://console.cloud.g
couldn't connect to "XXXXX:europe-west1:XXXXX": ensure that the Cloud SQL API is enabled for your project (https://console.cloud.google.com/flows/enableapi?apiid=sqladmin). Error during createEphemeral for XXXXX:europe-west1:XXXXX: googleapi: Error 403: Insufficient Permission, insufficientPermissions
调度程序无法完全启动,并发出以下堆栈跟踪:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 27, in <module>
args.func(args)
File "/usr/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 826, in scheduler
job.run()
File "/usr/local/lib/python2.7/site-packages/airflow/jobs.py", line 192, in run
session.commit()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 943, in commit
self.transaction.commit()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 467, in commit
self._prepare_impl()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 447, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2254, in flush
self._flush(objects)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2380, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2344, in _flush
flush_context.execute()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 391, in execute
rec.execute(self)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 556, in execute
uow
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 156, in save_obj
base_mapper, states, uowtransaction
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 286, in _organize_states_for_save(states):
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1252, in _connections_for_states
connection = uowtransaction.transaction.connection(base_mapper)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 294, in connection
return self._connection_for_bind(bind, execution_options)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 398, in _connection_for_bind
conn = self._parent._connection_for_bind(bind, execution_options)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 409, in _connection_for_bind
conn = bind.contextual_connect()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2123, in contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2162, in _wrap_pool_connect
e, dialect, self)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception_noconnection
exc_info
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2158, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 403, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 788, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 532, in checkout
rec = pool._do_get()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1193, in _do_get
self._dec_overflow()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1190, in _do_get
return self._create_connection()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 350, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 477, in __init__
self.__connect(first_connect_check=True)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 671, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 106, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 410, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python2.7/site-packages/MySQLdb/__init__.py", line 86, in Connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 204, in __init__
super(Connection, self).__init__(*args, **kwargs2)
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0") (Background on this error at: http://sqlalche.me/e/e3q8)
回溯(最近一次呼叫最后一次):
文件“/usr/local/bin/afflow”,第27行,在
args.func(args)
调度器中的文件“/usr/local/lib/python2.7/site packages/afflow/bin/cli.py”,第826行
job.run()
文件“/usr/local/lib/python2.7/site packages/afflow/jobs.py”,第192行,正在运行
session.commit()
提交中的文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/session.py”,第943行
self.transaction.commit()
提交中的文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/session.py”,第467行
self.\u prepare\u impl()
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/session.py”,第447行,在
self.session.flush()
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/session.py”,第2254行,刷新
自冲洗(对象)
文件“/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py”,第2380行,在
事务.rollback(\u capture\u exception=True)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/util/langhelpers.py”,第66行,在__
兼容性(exc_类型、exc_值、exc_tb)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/session.py”,第2344行,在\u flush中
flush_context.execute()
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/unitofwork.py”,第391行,在execute中
rec.execute(self)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/unitofwork.py”,执行中的第556行
uow
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/persistence.py”,第156行,在save_obj中
基本映射器、州、uowtransaction
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/persistence.py”,第286行,在“组织”状态中保存(状态):
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/persistence.py”,第1252行,在“连接”中表示状态
connection=uowtransaction.transaction.connection(基本映射器)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/session.py”,第294行,连接
为绑定返回self.\u连接(绑定、执行选项)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/session.py”,第398行,用于绑定的连接
conn=self.\u parent.\u连接\u用于绑定(绑定、执行\u选项)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/orm/session.py”,第409行,在用于绑定的连接中
conn=bind.context_connect()
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/engine/base.py”,第2123行,在上下文连接中
self.\u wrap\u pool\u connect(self.pool.connect,无),
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/engine/base.py”,第2162行,在“包装”池中
e、 方言(自我)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/engine/base.py”,第1476行,在\u handle\u dbapi\u exception\u noconnection中
exc_信息
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/util/compat.py”,第203行,从原因中提升
重新释放(类型(异常),异常,tb=exc\U tb,原因=原因)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/engine/base.py”,第2158行,在“包装”池中
返回fn()
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/pool.py”,第403行,在connect中
返回\u连接\u签出(自我)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/pool.py”,第788行,在_checkout中
fairy=\u ConnectionRecord.checkout(池)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/pool.py”,第532行,在签出中
rec=pool.\u do\u get()
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/pool.py”,第1193行,在
self._dec_overflow()
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/util/langhelpers.py”,第66行,在__
兼容性(exc_类型、exc_值、exc_tb)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/pool.py”,第1190行,在
返回self.\u创建\u连接()
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/pool.py”,第350行,在创建连接中
返回连接记录(自身)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/pool.py”,第477行,在__
自我连接(第一次连接检查=真)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/pool.py”,第671行,在_connect中
连接=池。\u调用\u创建者(自身)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/engine/strategies.py”,第106行,在connect中
返回方言。连接(*cargs,**cparams)
文件“/usr/local/lib/python2.7/site packages/sqlalchemy/engine/default.py”,第410行,在connect中
返回self.dbapi.connect(*cargs,**cparams)
文件“/usr/local/lib/python2.7/site-packages/MySQLdb/_-init___.py”,第86行,在Connect中
返回连接(*args,**kwargs)
文件“/usr/local/lib/python2.7/site packages/MySQLdb/connections.py”,第204行,在__
超级(连接,自).\uuuu初始化(*args,**kwargs2)
sqlalchemy.exc.OperationalError:(_mysql_exceptions.OperationalError)(2013,“在读取初始通信数据包时与mysql服务器失去连接”,系统错误:0”)(此错误的背景信息位于:http://sqlalche.me/e/e3q8)
与支持SQL数据库的连接现在似乎已断开。它仍然是相同的集群,但节点不同。是否需要更新其他配置?SQLProxy依赖于用于创建Composer环境的服务帐户的凭据。如果不更改任何设置,则这应该是计算引擎的默认服务帐户 您应该验证新节点池和