Airflow 气流计划程序错误:将线程最大值设置为1

Airflow 气流计划程序错误:将线程最大值设置为1,airflow,Airflow,我安装了airflow 1.10,可以毫无错误地运行airflow Web服务器,但当我运行“airflow scheduler”时,出现以下错误: {jobs.py:583}错误-使用sqlite时不能使用多个线程。将最大线程数设置为1 以下是此错误: sqlite3.OperationalError:没有这样的列:task\u instance.executor\u config 以下是完整的日志: [2018-12-28 22:59:04,556] {jobs.py:583} ERROR

我安装了airflow 1.10,可以毫无错误地运行airflow Web服务器,但当我运行“airflow scheduler”时,出现以下错误:

{jobs.py:583}错误-使用sqlite时不能使用多个线程。将最大线程数设置为1

以下是此错误:

sqlite3.OperationalError:没有这样的列:task\u instance.executor\u config

以下是完整的日志:

[2018-12-28 22:59:04,556] {jobs.py:583} ERROR - Cannot use more than 1 thread when using sqlite. Setting max_threads to 1

[2018-12-28 22:59:04,562] {jobs.py:1545} INFO - Starting the scheduler

[2018-12-28 22:59:04,563] {jobs.py:1559} INFO - Processing files using up to 1 processes at a time

[2018-12-28 22:59:04,563] {jobs.py:1560} INFO - Running execute loop for -1 seconds

[2018-12-28 22:59:04,563] {jobs.py:1561} INFO - Processing each file at most -1 times

[2018-12-28 22:59:04,563] {jobs.py:1564} INFO - Process each file at most once every 0 seconds

[2018-12-28 22:59:04,563] {jobs.py:1568} INFO - Checking for new files in /Users/mtahe0806/airflow/dags every 300 seconds

[2018-12-28 22:59:04,563] {jobs.py:1571} INFO - Searching for files in /Users/mtahe0806/airflow/dags

[2018-12-28 22:59:04,588] {jobs.py:1573} INFO - There are 6 files in /Users/mtahe0806/airflow/dags

[2018-12-28 22:59:04,588] {jobs.py:1635} INFO - Resetting orphaned tasks for active dag runs

[2018-12-28 22:59:04,590] {jobs.py:1590} INFO - Exited execute loop
Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: no such column: task_instance.executor_config

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/anaconda3/bin/airflow", line 32, in <module>
    args.func(args)
  File "/anaconda3/lib/python3.6/site-packages/airflow/utils/cli.py", line 74, in wrapper
    return f(*args, **kwargs)
  File "/anaconda3/lib/python3.6/site-packages/airflow/bin/cli.py", line 928, in scheduler
    job.run()
  File "/anaconda3/lib/python3.6/site-packages/airflow/jobs.py", line 203, in run
    self._execute()
  File "/anaconda3/lib/python3.6/site-packages/airflow/jobs.py", line 1588, in _execute
    self._execute_helper(processor_manager)
  File "/anaconda3/lib/python3.6/site-packages/airflow/jobs.py", line 1636, in _execute_helper
    self.reset_state_for_orphaned_tasks()
  File "/anaconda3/lib/python3.6/site-packages/airflow/utils/db.py", line 74, in wrapper
    return func(*args, **kwargs)
  File "/anaconda3/lib/python3.6/site-packages/airflow/jobs.py", line 256, in reset_state_for_orphaned_tasks
    TI.state.in_(resettable_states))).all()
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2703, in all
    return list(self)
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__
    return self._execute_and_instances(context)
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2878, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such column: task_instance.executor_config [SQL: 'SELECT task_instance.try_number AS task_instance_try_number, task_instance.task_id AS task_instance_task_id, task_instance.dag_id AS task_instance_dag_id, task_instance.execution_date AS task_instance_execution_date, task_instance.start_date AS task_instance_start_date, task_instance.end_date AS task_instance_end_date, task_instance.duration AS task_instance_duration, task_instance.state AS task_instance_state, task_instance.max_tries AS task_instance_max_tries, task_instance.hostname AS task_instance_hostname, task_instance.unixname AS task_instance_unixname, task_instance.job_id AS task_instance_job_id, task_instance.pool AS task_instance_pool, task_instance.queue AS task_instance_queue, task_instance.priority_weight AS task_instance_priority_weight, task_instance.operator AS task_instance_operator, task_instance.queued_dttm AS task_instance_queued_dttm, task_instance.pid AS task_instance_pid, task_instance.executor_config AS task_instance_executor_config \nFROM task_instance JOIN dag_run ON task_instance.dag_id = dag_run.dag_id AND task_instance.execution_date = dag_run.execution_date \nWHERE dag_run.state = ? AND dag_run.run_id NOT LIKE ? AND task_instance.state IN (?, ?)'] [parameters: ('running', 'backfill_%', 'scheduled', 'queued')]
[2018-12-28 22:59:04556]{jobs.py:583}错误-使用sqlite时不能使用多个线程。将最大线程数设置为1
[2018-12-28 22:59:04562]{jobs.py:1545}信息-启动调度程序
[2018-12-28 22:59:04563]{jobs.py:1559}信息-一次最多使用一个进程处理文件
[2018-12-28 22:59:04563]{jobs.py:1560}INFO-运行执行循环-1秒
[2018-12-28 22:59:04563]{jobs.py:1561}信息-最多处理一次每个文件
[2018-12-28 22:59:04563]{jobs.py:1564}INFO-每0秒最多处理一次每个文件
[2018-12-28 22:59:04563]{jobs.py:1568}INFO-每300秒检查一次/Users/mtahe0806/afflow/dags中的新文件
[2018-12-28 22:59:04563]{jobs.py:1571}INFO-在/Users/mtahe0806/aiffort/dags中搜索文件
[2018-12-28 22:59:04588]{jobs.py:1573}INFO-在/Users/mtahe0806/aiffort/dags中有6个文件
[2018-12-28 22:59:04588]{jobs.py:1635}INFO-为活动dag运行重置孤立任务
[2018-12-28 22:59:04590]{jobs.py:1590}信息-已退出执行循环
回溯(最近一次呼叫最后一次):
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/engine/base.py”,第1182行,在执行上下文中
(上下文)
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/engine/default.py”,第470行,在do_execute中
cursor.execute(语句、参数)
sqlite3.OperationalError:没有这样的列:task\u instance.executor\u config
上述异常是以下异常的直接原因:
回溯(最近一次呼叫最后一次):
文件“/anaconda3/bin/afflow”,第32行,在
args.func(args)
文件“/anaconda3/lib/python3.6/site packages/afflow/utils/cli.py”,第74行,在包装器中
返回f(*args,**kwargs)
调度器中的文件“/anaconda3/lib/python3.6/site packages/afflow/bin/cli.py”,第928行
job.run()
文件“/anaconda3/lib/python3.6/site packages/afflow/jobs.py”,第203行,运行中
self._execute()
文件“/anaconda3/lib/python3.6/site packages/afflow/jobs.py”,第1588行,在
self.\u execute\u helper(处理器\u管理器)
文件“/anaconda3/lib/python3.6/site packages/afflow/jobs.py”,第1636行,在“执行”助手中
self.reset\u state\u用于\u孤立的\u任务()
文件“/anaconda3/lib/python3.6/site packages/afflow/utils/db.py”,第74行,在包装器中
返回函数(*args,**kwargs)
文件“/anaconda3/lib/python3.6/site packages/afflow/jobs.py”,第256行,对于孤立的任务,处于重置状态
TI.state.in_u(可重置_状态)).all()
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/orm/query.py”,第2703行,共
返回列表(自我)
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/orm/query.py”,第2855行,在__
返回self.\u执行\u和\u实例(上下文)
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/orm/query.py”,第2878行,在执行和实例中
结果=conn.execute(querycontext.statement,self.\u参数)
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/engine/base.py”,执行中的第945行
返回方法(自身、多线程、参数)
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/sql/elements.py”,第263行,在连接上执行
返回连接。_execute_clauseelement(self、multiparams、params)
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/engine/base.py”,第1053行,在执行元素中
编译的sql,提取的参数
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/engine/base.py”,第1189行,在执行上下文中
(上下文)
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/engine/base.py”,第1402行,在_handle_dbapi_exception中
exc_信息
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/util/compat.py”,第203行,从原因中提升
重新释放(类型(异常),异常,tb=exc\U tb,原因=原因)
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/util/compat.py”,第186行,在reraise中
通过_回溯(tb)提升值
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/engine/base.py”,第1182行,在执行上下文中
(上下文)
文件“/anaconda3/lib/python3.6/site packages/sqlalchemy/engine/default.py”,第470行,在do_execute中
cursor.execute(语句、参数)
sqlalchemy.exc.OperationalError:(sqlite3.OperationalError)没有这样的列:task\u instance.executor\u config[SQL:'选择task\u instance.try\u number作为task\u instance\u number,task\u instance.task\u id作为task\u instance\u task\u id,task\u instance.dag\u id作为task\u instance\u dag\u id,task\u instance.execution\u date作为task\u instance\u execution\u日期,task\u instance.start\u日期作为task\u instance\u开始日期,task\u instance.end\u日期作为task\u实例结束日期,task\u instance.Duran作为任务\实例\持续时间,任务\实例.state作为任务\实例\状态,任务\实例.max\尝试作为任务\实例\ max\尝试,任务\实例.hostname作为任务\实例\主机名,任务\实例.unixname作为任务\实例\ unixname,任务\实例.job\ id作为任务\实例\作业\ id,任务\实例.pool作为任务\实例\池,任务\实例.queue作为任务\实例_队列,task_instance.priority_weight AS task_instance_priority_weight,task_instance.operator AS task_instance_operator,task_instance.queued_dttm AS task_instance_排队_dttm,task_instance.pid AS task_instance_pid,task_instance.executor_config AS task_instance_executor_config\n从task_instance加入dag_在task_instance上运行。dag_id=dag_运行。dag_id和task_instance.execution_date=dag_run.execution_date\n其中dag_run.state=?和dag_run.run_id与(?,)中的task_instance.state不一样[parameters:('running','backfill_%','scheduled','queued')]
检查<