Amazon web services 气流1.8.1&；芹菜4.0.2 AWS SQS broker_url类型错误

我正在尝试在AWS的EC2上配置气流服务器。我已经将数据库设置为指向RDS中的MySQL实例。我想使用AWS SQS而不是RabbitMQ来设置它，尽管这是我遇到的难题

我已经为CeleryExecutor配置好了

broker_url的My afflow.cfg条目的格式如下

broker_url = sqs://[AWS KEY]:[AWS SECRET]@[SQS URL]

我已经试过了

broker_url = sqs://[AWS KEY]:[AWS SECRET]@

我得到以下回应

 [ec2-user@ip-10-0-0-196 airflow]$ airflow worker
[2017-06-27 00:03:38,611] {__init__.py:57} INFO - Using executor CeleryExecutor

 -------------- celery@ip-10-0-0-196 v4.0.2 (latentcall)
---- **** -----
--- * ***  * -- Linux-4.9.27-14.33.amzn1.x86_64-x86_64-with-glibc2.2.5 2017-06-27 00:03:38
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         airflow.executors.celery_executor:0x7fd07c867f50
- ** ---------- .> transport:   sqs://[AWS KEY]:**@localhost//
- ** ---------- .> results:     mysql://[RDS DB USERNAME]:**@[RDS URL]:3306/airflowdb
- *** --- * --- .> concurrency: 16 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
            .> default          exchange=default(direct) key=default


[2017-06-27 00:03:39,270] {__init__.py:57} INFO - Using executor CeleryExecutor
Starting flask
[2017-06-27 00:03:39,545] {_internal.py:87} INFO -  * Running on http://0.0.0.0:8793/ (Press CTRL+C to quit)
[2017-06-27 00:03:40,671: CRITICAL/MainProcess] Unrecoverable error: TypeError(u'Cannot mix str and non-str arguments
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/celery/worker/worker.py", line 203, in start
self.blueprint.start(self)
File "/usr/local/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/local/lib/python2.7/site-packages/celery/bootsteps.py", line 370, in start
return self.obj.start()
File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
blueprint.start(self)
File "/usr/local/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 594, in start
c.loop(*c.loop_args())
File "/usr/local/lib/python2.7/site-packages/celery/worker/loops.py", line 88, in asynloop
next(loop)
File "/usr/local/lib/python2.7/site-packages/kombu/async/hub.py", line 282, in create_loop
item()
File "/usr/local/lib/python2.7/site-packages/vine/promises.py", line 139, in __call__
return self.throw()
File "/usr/local/lib/python2.7/site-packages/vine/promises.py", line 136, in __call__
retval = fun(*final_args, **final_kwargs)
File "/usr/local/lib/python2.7/site-packages/kombu/transport/SQS.py", line 290, in _schedule_queue
queue, callback=promise(self._loop1, (queue,)),
File "/usr/local/lib/python2.7/site-packages/kombu/transport/SQS.py", line 306, in _get_bulk_async
return self._get_async(queue, maxcount, callback=callback)
File "/usr/local/lib/python2.7/site-packages/kombu/transport/SQS.py", line 316, in _get_async
callback=transform(self._on_messages_ready, callback, q, queue),
File "/usr/local/lib/python2.7/site-packages/kombu/transport/SQS.py", line 336, in _get_from_sqs
callback=callback,
File "/usr/local/lib/python2.7/site-packages/kombu/async/aws/sqs/connection.py", line 82, in receive_message
queue.id, callback=callback,
File "/usr/local/lib/python2.7/site-packages/kombu/async/aws/connection.py", line 239, in get_list
self._on_list_ready, callback, parent or self, markers,
File "/usr/local/lib/python2.7/site-packages/kombu/async/aws/connection.py", line 232, in make_request
return self._mexe(request, callback=callback)
File "/usr/local/lib/python2.7/site-packages/kombu/async/aws/connection.py", line 198, in _mexe
conn.getresponse(callback=callback)
File "/usr/local/lib/python2.7/site-packages/kombu/async/aws/connection.py", line 127, in getresponse
request = self.getrequest()
File "/usr/local/lib/python2.7/site-packages/kombu/async/aws/connection.py", line 120, in getrequest
url = urlunsplit((scheme, host, self.path, '', ''))
File "/usr/local/lib/python2.7/site-packages/future/backports/urllib/parse.py", line 399, in urlunsplit
_coerce_args(*components))
File "/usr/local/lib/python2.7/site-packages/future/backports/urllib/parse.py", line 115, in _coerce_args
raise TypeError("Cannot mix str and non-str arguments")

TypeError: Cannot mix str and non-str arguments

密钥和机密都不包含任何特殊字符

是否有人使用SQS成功设置了气流，他们可以共享他们的Airflow.cfg

谢谢

---

Anant

你让它工作了吗？我正在挑选一个经纪人，但不确定我是否信任SQS。试试这个链接：这似乎是一个类似的问题。