Airflow 在minikube上使用舵图时气流Web服务器未启动

Airflow 在minikube上使用舵图时气流Web服务器未启动,airflow,gunicorn,kubernetes-helm,minikube,Airflow,Gunicorn,Kubernetes Helm,Minikube,我正在尝试使用minikube和helm chart stable/Airflow在本地运行气流(在部署前进行测试)。但由于gunicorn问题,airflow Web服务器无法启动 舵:v2.14.3 Kubernetes:v1.15.2 Minikube:v1.3.1 舵图图像:普克尔/多克气流 以下是步骤: minikube启动 helm安装--名称空间“airflow”--名称“airflow”stable/airflow 日志包括: Thu Sep 12 07:29:54 UTC 20

我正在尝试使用minikube和helm chart stable/Airflow在本地运行气流(在部署前进行测试)。但由于gunicorn问题,airflow Web服务器无法启动

舵:v2.14.3

Kubernetes:v1.15.2

Minikube:v1.3.1

舵图图像:普克尔/多克气流

以下是步骤:

minikube启动

helm安装--名称空间“airflow”--名称“airflow”stable/airflow

日志包括:

Thu Sep 12 07:29:54 UTC 2019 - waiting for Postgres... 1/20
Thu Sep 12 07:30:00 UTC 2019 - waiting for Postgres... 2/20
waiting 60s...
executing webserver...
[2019-09-12 07:31:05,745] {{settings.py:213}} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=1
/usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
[2019-09-12 07:31:06,030] {{__init__.py:51}} INFO - Using executor CeleryExecutor
  ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
[2019-09-12 07:31:06,585] {{dagbag.py:90}} INFO - Filling up the DagBag from /usr/local/airflow/dags
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
Logfiles: - -
=================================================================            
[2019-09-12 07:31:07,676] {{settings.py:213}} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=21
/usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
[2019-09-12 07:31:07 +0000] [21] [INFO] Starting gunicorn 19.9.0
[2019-09-12 07:31:07 +0000] [21] [INFO] Listening at: http://0.0.0.0:8080 (21)
[2019-09-12 07:31:07 +0000] [21] [INFO] Using worker: sync
[2019-09-12 07:31:07 +0000] [25] [INFO] Booting worker with pid: 25
[2019-09-12 07:31:07 +0000] [26] [INFO] Booting worker with pid: 26
[2019-09-12 07:31:07 +0000] [27] [INFO] Booting worker with pid: 27
[2019-09-12 07:31:07 +0000] [28] [INFO] Booting worker with pid: 28
[2019-09-12 07:31:08,444] {{__init__.py:51}} INFO - Using executor CeleryExecutor
[2019-09-12 07:31:08,446] {{__init__.py:51}} INFO - Using executor CeleryExecutor
[2019-09-12 07:31:08,545] {{__init__.py:51}} INFO - Using executor CeleryExecutor
[2019-09-12 07:31:08,669] {{__init__.py:51}} INFO - Using executor CeleryExecutor
[2019-09-12 07:31:10,047] {{dagbag.py:90}} INFO - Filling up the DagBag from /usr/local/airflow/dags
[2019-09-12 07:31:20,932] {{cli.py:825}} ERROR - [0 / 0] some workers seem to have died and gunicorndid not restart them as expected
[2019-09-12 07:31:22,095] {{dagbag.py:90}} INFO - Filling up the DagBag from /usr/local/airflow/dags
[2019-09-12 07:31:22 +0000] [25] [INFO] Parent changed, shutting down: <Worker 25>
[2019-09-12 07:31:22 +0000] [25] [INFO] Worker exiting (pid: 25)
[2019-09-12 07:31:32 +0000] [28] [INFO] Parent changed, shutting down: <Worker 28>
[2019-09-12 07:31:32 +0000] [28] [INFO] Worker exiting (pid: 28)
[2019-09-12 07:31:33,289] {{dagbag.py:90}} INFO - Filling up the DagBag from /usr/local/airflow/dags
[2019-09-12 07:31:33,324] {{dagbag.py:90}} INFO - Filling up the DagBag from /usr/local/airflow/dags
[2019-09-12 07:31:35 +0000] [26] [INFO] Parent changed, shutting down: <Worker 26>
[2019-09-12 07:31:35 +0000] [26] [INFO] Worker exiting (pid: 26)
[2019-09-12 07:31:35 +0000] [27] [INFO] Parent changed, shutting down: <Worker 27>
[2019-09-12 07:31:35 +0000] [27] [INFO] Worker exiting (pid: 27)
[2019-09-12 07:33:32,017] {{cli.py:832}} ERROR - No response from gunicorn master within 120 seconds
[2019-09-12 07:33:32,018] {{cli.py:833}} ERROR - Shutting down webserver

2019年9月12日星期四07:29:54 UTC-等待博士后。。。1/20
2019年9月12日星期四07:30:00 UTC-等待博士后。。。2/20
等待60年代。。。
正在执行Web服务器。。。
[2019-09-12 07:31:05745]{{settings.py:213}}INFO-settings.configure_orm():使用池设置。池大小=5,最大溢出=10,池回收=1800,pid=1
/usr/local/lib/python3.7/site-packages/psycopg2/__-init__;.py:144:UserWarning:psycopg2-wheel-packages将从2.8版重命名;为了保持从二进制文件安装,请改用“pip安装psycopg2二进制文件”。有关详细信息,请参阅:。
""")
[2019-09-12 07:31:06030]{{{{uuuuu init_uuuuuu.py:51}}信息-使用executor CeleryExecutor
____________       _____________
____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
_/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
[2019-09-12 07:31:06585]{{dagbag.py:90}}信息-从/usr/local/aiffort/dags填充dagbag
使用以下各项运行Gunicorn服务器:
工人:4人
主持人:0.0.0.0:8080
超时:120
日志文件:--
=================================================================            
[2019-09-12 07:31:07676]{{settings.py:213}}INFO-settings.configure_orm():使用池设置。池大小=5,最大溢出=10,池回收=1800,pid=21
/usr/local/lib/python3.7/site packages/psycopg2/__init__u;.py:144:UserWarning:psycopg2.8版将重命名psycopg2控制盘软件包;为了保持从二进制文件安装,请改用“pip安装psycopg2二进制文件”。有关详细信息,请参阅:。
""")
[2019-09-12 07:31:07+0000][21][INFO]开始gunicorn 19.9.0
[2019-09-12 07:31:07+0000][21][INFO]收听地点:http://0.0.0.0:8080 (21)
[2019-09-12 07:31:07+0000][21][INFO]使用辅助程序:同步
[2019-09-12 07:31:07+0000][25][INFO]带pid的引导工人:25
[2019-09-12 07:31:07+0000][26][INFO]带pid的引导工人:26
[2019-09-12 07:31:07+0000][27][INFO]带pid的引导工人:27
[2019-09-12 07:31:07+0000][28][INFO]带pid的引导工人:28
[2019-09-12 07:31:08444]{{{{uuuuu init_uuuuu.py:51}}信息-使用executor CeleryExecutor
[2019-09-12 07:31:08446]{{{{uuuuu init_uuuuu.py:51}}信息-使用executor CeleryExecutor
[2019-09-12 07:31:08545]{{{{uuuuu init_uuuuuu.py:51}}信息-使用executor CeleryExecutor
[2019-09-12 07:31:08669]{{{{uuuuu init_uuuuuu.py:51}}信息-使用executor CeleryExecutor
[2019-09-12 07:31:10047]{{dagbag.py:90}}信息-从/usr/local/aiffair/dags填充dagbag
[2019-09-12 07:31:20932]{{cli.py:825}}错误-[0/0]一些工人似乎已经死亡,Gunicorn没有按预期重新启动他们
[2019-09-12 07:31:22095]{{dagbag.py:90}}信息-从/usr/local/aiffair/dags填充dagbag
[2019-09-12 07:31:22+0000][25][INFO]母公司已更改,正在关闭:
[2019-09-12 07:31:22+0000][25][INFO]工人退出(pid:25)
[2019-09-12 07:31:32+0000][28][INFO]父级已更改,正在关闭:
[2019-09-12 07:31:32+0000][28][INFO]工人退出(pid:28)
[2019-09-12 07:31:33289]{{dagbag.py:90}}信息-从/usr/local/aiffort/dags填充dagbag
[2019-09-12 07:31:33324]{{dagbag.py:90}}信息-从/usr/local/aiffort/dags填充dagbag
[2019-09-12 07:31:35+0000][26][INFO]母公司已更改,正在关闭:
[2019-09-1207:31:35+0000][26][INFO]工人退出(pid:26)
[2019-09-12 07:31:35+0000][27][INFO]母公司已更改,正在关闭:
[2019-09-12 07:31:35+0000][27][INFO]工人退出(pid:27)
[2019-09-12 07:33:32017]{{cli.py:832}错误-gunicorn主机在120秒内没有响应
[2019-09-12 07:33:32018]{{cli.py:833}错误-关闭Web服务器

我可以使用docker compose在本地运行docker映像,没有问题,但使用helm时运气不佳,它会失败并不断重新启动。

结果表明,问题在于minikube配置没有使postgres的pod可用,使用它工作的postgres实例的ip编辑pod部署