Kubernetes 气流服务尚未开始

Kubernetes 气流服务尚未开始,kubernetes,webserver,airflow,postgres-9.6,Kubernetes,Webserver,Airflow,Postgres 9.6,我们正在尝试部署airflow(1.9.0)版本,该版本使用Postgres作为其DB,并使用Redis实现Kubernetes中吊舱之间的连接 同样的设置在暂存环境中运行良好,在prod环境中失败,经过调查,我了解到image 9.6的Postgres基本版本最近发生了更改,这会导致任何问题吗 附件是postgress和Web服务器的日志(气流) 尝试使用不同的姿势图像,但仍然相同 Web服务器日志 Collecting botocore Downloading https://files

我们正在尝试部署airflow(1.9.0)版本,该版本使用Postgres作为其DB,并使用Redis实现Kubernetes中吊舱之间的连接

同样的设置在暂存环境中运行良好,在prod环境中失败,经过调查,我了解到image 9.6的Postgres基本版本最近发生了更改,这会导致任何问题吗

附件是postgress和Web服务器的日志(气流)

尝试使用不同的姿势图像,但仍然相同

Web服务器日志

Collecting botocore
  Downloading https://files.pythonhosted.org/packages/a1/b0/7a8794d914b95ef3335a5a4ba20595b46081dbd1e29f13812eceacf091ca/botocore-1.12.215-py2.py3-none-any.whl (5.7MB)
Collecting docutils<0.16,>=0.10 (from botocore)
  Downloading https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl (547kB)
Collecting jmespath<1.0.0,>=0.7.1 (from botocore)
  Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Requirement already satisfied: urllib3<1.26,>=1.20; python_version >= "3.4" in /usr/lib/python3/dist-packages (from botocore) (1.22)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= "2.7" in /usr/local/lib/python3.6/dist-packages (from botocore) (2.8.0)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore) (1.11.0)
Installing collected packages: docutils, jmespath, botocore
Successfully installed botocore-1.12.215 docutils-0.15.2 jmespath-0.9.4
Multi-tenant details not configured in this instance - Exiting
Cluster "abc" set.
User "abc@airflow.com" set.
Context "abc" created.
Switched to context "cedp".
[2019-08-26 14:01:03,391] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-08-26 14:01:03,415] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 17, in <module>
    from airflow.bin.cli import CLIFactory
  File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 47, in <module>
    from airflow import jobs, settings
  File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 64, in <module>
    class BaseJob(Base, LoggingMixin):
  File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 96, in BaseJob
    executor=executors.GetDefaultExecutor(),
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 42, in GetDefaultExecutor
    DEFAULT_EXECUTOR = _get_executor(executor_name)
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 60, in _get_executor
    from airflow.executors.celery_executor import CeleryExecutor
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/celery_executor.py", line 18, in <module>
    from celery import Celery
  File "/usr/local/lib/python2.7/dist-packages/celery/local.py", line 509, in __getattr__
    module = __import__(self._object_origins[name], None, None, [name])
  File "/usr/local/lib/python2.7/dist-packages/celery/app/__init__.py", line 5, in <module>
    from celery import _state
  File "/usr/local/lib/python2.7/dist-packages/celery/_state.py", line 17, in <module>
    from celery.utils.threads import LocalStack
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/__init__.py", line 9, in <module>
    from .nodenames import worker_direct, nodename, nodesplit
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/nodenames.py", line 9, in <module>
    from kombu.entity import Exchange, Queue
  File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 9, in <module>
    from .serialization import prepare_accept_content
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 456, in <module>
    for ep, args in entrypoints('kombu.serializers'):  # pragma: no cover
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/compat.py", line 89, in entrypoints
    for ep in importlib_metadata.entry_points().get(namespace, [])
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 456, in entry_points
    ordered = sorted(eps, key=by_group)
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 454, in <genexpr>
    dist.entry_points for dist in distributions())
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 364, in <genexpr>
    cls._search_path(path, pattern)
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 373, in _switch_path
    return pathlib.Path(path)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 1256, in __new__
    self = cls._from_parts(args, init=False)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 898, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 891, in _parse_args
    return cls._flavour.parse_parts(parts)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 250, in parse_parts
    parsed.append(intern(x))
TypeError: can't intern subclass of string
[2019-08-26 14:01:04,253] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-08-26 14:01:04,277] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Traceback (most recent call last):
  File "/usr/local/airflow/set_auth.py", line 16, in <module>
    session.commit()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 927, in commit
    self.transaction.commit()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 467, in commit
    self._prepare_impl()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 447, in _prepare_impl
    self.session.flush()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2209, in flush
    self._flush(objects)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2329, in _flush
    transaction.rollback(_capture_exception=True)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2293, in _flush
    flush_context.execute()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
    rec.execute(self)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
    uow
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
    mapper, table, insert)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
    execute(statement, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "users" does not exist
LINE 1: INSERT INTO users (username, email, password) VALUES ('admin...
                    ^
 [SQL: 'INSERT INTO users (username, email, password) VALUES (%(username)s, %(email)s, %(password)s) RETURNING users.id'] [parameters: {'username': 'admin', 'password': '$2b$12$F.8CTth9cL5G9f.pd180Duz/nC8S5KwTctwf/jG1Y/QB8PZagkTa.', 'email': 'abc@airflow.com'}]
[2019-08-26 14:01:15,229] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-08-26 14:01:15,253] {{driver.py:124}} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 17, in <module>
    from airflow.bin.cli import CLIFactory
  File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 47, in <module>
    from airflow import jobs, settings
  File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 64, in <module>
    class BaseJob(Base, LoggingMixin):
  File "/usr/local/lib/python2.7/dist-packages/airflow/jobs.py", line 96, in BaseJob
    executor=executors.GetDefaultExecutor(),
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 42, in GetDefaultExecutor
    DEFAULT_EXECUTOR = _get_executor(executor_name)
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py", line 60, in _get_executor
    from airflow.executors.celery_executor import CeleryExecutor
  File "/usr/local/lib/python2.7/dist-packages/airflow/executors/celery_executor.py", line 18, in <module>
    from celery import Celery
  File "/usr/local/lib/python2.7/dist-packages/celery/local.py", line 509, in __getattr__
    module = __import__(self._object_origins[name], None, None, [name])
  File "/usr/local/lib/python2.7/dist-packages/celery/app/__init__.py", line 5, in <module>
    from celery import _state
  File "/usr/local/lib/python2.7/dist-packages/celery/_state.py", line 17, in <module>
    from celery.utils.threads import LocalStack
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/__init__.py", line 9, in <module>
    from .nodenames import worker_direct, nodename, nodesplit
  File "/usr/local/lib/python2.7/dist-packages/celery/utils/nodenames.py", line 9, in <module>
    from kombu.entity import Exchange, Queue
  File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 9, in <module>
    from .serialization import prepare_accept_content
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 456, in <module>
    for ep, args in entrypoints('kombu.serializers'):  # pragma: no cover
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/compat.py", line 89, in entrypoints
    for ep in importlib_metadata.entry_points().get(namespace, [])
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 456, in entry_points
    ordered = sorted(eps, key=by_group)
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 454, in <genexpr>
    dist.entry_points for dist in distributions())
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 364, in <genexpr>
    cls._search_path(path, pattern)
  File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py", line 373, in _switch_path
    return pathlib.Path(path)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 1256, in __new__
    self = cls._from_parts(args, init=False)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 898, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 891, in _parse_args
    return cls._flavour.parse_parts(parts)
  File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 250, in parse_parts
    parsed.append(intern(x))
TypeError: can't intern subclass of string



Postgress LOg

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok


WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start....LOG:  database system was shut down at 2019-08-26 13:42:41 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
 done
server started
CREATE DATABASE


/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*

LOG:  received fast shutdown request
waiting for server to shut down...LOG:  aborting any active transactions
.LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

LOG:  database system was shut down at 2019-08-26 13:42:43 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  relation "users" does not exist at character 13
STATEMENT:  INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$Vmkgo0OBgjLmylPMi3yrCOhVIWhWAgrEpCCojRZw0weeP..3nneg.') RETURNING users.id
LOG:  incomplete startup packet
LOG:  incomplete startup packet
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  relation "users" does not exist at character 13
STATEMENT:  INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$y2DtC8uEM5coowQZP3GZsOIw/QFkqKZqvV4TcOkCSJ0wM.QbiwbA2') RETURNING users.id
LOG:  incomplete startup packet
LOG:  incomplete startup packet
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  relation "users" does not exist at character 13
STATEMENT:  INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$0gj.4OfVy5y.xVt2FpVny.mRfCD/1wYnAbdMA22Xj4aI54tATo4Nu') RETURNING users.id
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  relation "users" does not exist at character 13
STATEMENT:  INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$KXKdhuhdt5rmehEPxuX1He8uwE2fvgMcWoS4rg4oGzL5xWfn8Cgd6') RETURNING users.id
LOG:  incomplete startup packet
LOG:  incomplete startup packet
LOG:  incomplete startup packet
ERROR:  relation "users" does not exist at character 13
STATEMENT:  INSERT INTO users (username, email, password) VALUES ('admin', 'abc@airflow.com', '$2b$12$QVtj0DHd6uLOnlIlwbE3kezYDzP.Y8m/Ln9H9of77pEKCihOiLhnq') RETURNING users.id
LOG:  incomplete startup packet
收集botocore
正在下载https://files.pythonhosted.org/packages/a1/b0/7a8794d914b95ef3335a5a4ba20595b46081dbd1e29f13812eceacf091ca/botocore-1.12.215-py2.py3-none-any.whl (5.7MB)
收集文档=0.10(从botocore)
正在下载https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl (547kB)
正在收集jmespath=0.7.1(从botocore)
正在下载https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
已经满足的要求:urllib3=1.20;python_version>=/usr/lib/python3/dist包中的“3.4”(来自botocore)(1.22)
已经满足的需求:python dateutil=2.1;python_version>=/usr/local/lib/python3.6/dist-packages中的“2.7”(来自botocore)(2.8.0)
已满足要求:六个>=1.5 in/usr/lib/python3/dist包(来自python dateutil=2.1;python_版本>=“2.7”->botocore)(1.11.0)
安装收集的软件包:docutils、jmespath、botocore
已成功安装botocore-1.12.215 docutils-0.15.2 jmespath-0.9.4
此实例中未配置多租户详细信息-正在退出
集群“abc”集合。
用户“abc@airflow.com”“好的。
上下文“abc”已创建。
切换到上下文“cedp”。
[2019-08-26 14:01:03391]{{driver.py:124}}INFO-从/usr/lib/python2.7/lib2to3/grammar.txt生成语法表
[2019-08-26 14:01:03415]{{driver.py:124}}INFO-从/usr/lib/python2.7/lib2to3/patternegrammar.txt生成语法表
回溯(最近一次呼叫最后一次):
文件“/usr/local/bin/afflow”,第17行,在
从afflow.bin.cli导入CLIFactory
文件“/usr/local/lib/python2.7/dist-packages/afflow/bin/cli.py”,第47行,在
从导入作业、设置
文件“/usr/local/lib/python2.7/dist-packages/afflow/jobs.py”,第64行,在
类BaseJob(Base,LoggingMixin):
BaseJob中的第96行文件“/usr/local/lib/python2.7/dist-packages/afflow/jobs.py”
executor=executors.GetDefaultExecutor(),
文件“/usr/local/lib/python2.7/dist包/气流/executors/\u init\uuuy”,第42行,GetDefaultExecutor
默认执行者=\u获取执行者(执行者名称)
文件“/usr/local/lib/python2.7/dist packages/afflow/executors/__init__.py”,第60行,in_get_executor
从airflow.executors.celery\u executor导入芹菜executor
文件“/usr/local/lib/python2.7/dist packages/afflow/executors/celery_executor.py”,第18行,在
从芹菜进口芹菜
文件“/usr/local/lib/python2.7/dist packages/芹菜/local.py”,第509行,在__
模块=\uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu导入(self.\u对象\u来源[名称],无,无,[名称])
文件“/usr/local/lib/python2.7/dist-packages/芹菜/app/_-init__.py”,第5行,在
来自西芹进口州
文件“/usr/local/lib/python2.7/dist packages/芹菜/_state.py”,第17行,在
从celery.utils.threads导入LocalStack
文件“/usr/local/lib/python2.7/dist-packages/cellery/utils/__init__.py”,第9行,在
from.nodenames导入worker\u direct、nodename、nodesplit
文件“/usr/local/lib/python2.7/dist packages/cellery/utils/nodenames.py”,第9行,在
从kombu.entity导入交换,队列
文件“/usr/local/lib/python2.7/dist-packages/kombu/entity.py”,第9行,在
from.serialization导入准备\u接受\u内容
文件“/usr/local/lib/python2.7/dist-packages/kombu/serialization.py”,第456行,在
对于ep,入口点中的参数('kombu.serializers'):#pragma:无封面
文件“/usr/local/lib/python2.7/dist-packages/kombu/utils/compat.py”,第89行,入口点
对于importlib_metadata.entry_points().get(命名空间,[])中的ep
文件“/usr/local/lib/python2.7/dist packages/importlib_metadata/_init__.py”,第456行,入口点
已排序=已排序(eps,键=按组)
文件“/usr/local/lib/python2.7/dist packages/importlib_metadata/_init__.py”,第454行,在
dist.entry_分发中dist的点()
文件“/usr/local/lib/python2.7/dist packages/importlib_metadata/_init__.py”,第364行,在
cls.\u搜索\u路径(路径、模式)
文件“/usr/local/lib/python2.7/dist packages/importlib_metadata/_init__.py”,第373行,在开关路径中
返回pathlib.Path(路径)
文件“/usr/local/lib/python2.7/dist-packages/pathlib2/_-init__uuuuuu.py”,第1256行,在新的__
self=cls.\u来自零件(args,init=False)
文件“/usr/local/lib/python2.7/dist-packages/pathlib2/_-init___;.py”,第898行,位于from_u-parts中
drv,root,parts=self.\u parse\u args(args)
文件“/usr/local/lib/python2.7/dist-packages/pathlib2/_-init___;.py”,第891行,在解析参数中
返回cls.\u flavor.parse\u部分(parts)
文件“/usr/local/lib/python2.7/dist-packages/pathlib2/_-init__uuu.py”,第250行,在parse\u部分中
解析。追加(实习生(x))
TypeError:无法插入字符串的子类
[2019-08-26 14:01:04253]{{driver.py:124}}INFO-从/usr/lib/python2.7/lib2to3/grammar.txt生成语法表
[2019-08-26 14:01:04277]{{driver.py:124}}INFO-从/usr/lib/python2.7/lib2to3/patternegrammar.txt生成语法表
回溯(最近一次呼叫最后一次):
文件“/usr/local/afflow/set_auth.py”,第16行,在
session.commit()
提交中的文件“/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py”,第927行
self.transaction.commit()
提交中的文件“/usr/local/lib/python2.7/dist packages/sqlalchemy/orm/session.py”,第467行
self.\u prepare\u impl()
文件“/usr/local/lib/python2.7/dist-packages/sqlalchemy/or