Python 3.x 当pip需求中包含雪花依赖项时,docker构建失败

Python 3.x 当pip需求中包含雪花依赖项时,docker构建失败,python-3.x,docker,dockerfile,snowflake-cloud-data-platform,Python 3.x,Docker,Dockerfile,Snowflake Cloud Data Platform,我正在尝试用python应用程序构建docker映像。但是,当我将“雪花连接器python”添加到pip requirements.txt文件时,docker构建失败了 requirements.txt:- boto3 pycryptodome snowflake-connector-python Dockerfile:- FROM python:alpine3.6 RUN python --version COPY . /sample-app WORKDIR /sample-app RU

我正在尝试用python应用程序构建docker映像。但是,当我将“雪花连接器python”添加到pip requirements.txt文件时,docker构建失败了

requirements.txt:-

boto3
pycryptodome
snowflake-connector-python
Dockerfile:-

FROM python:alpine3.6
RUN python --version

COPY . /sample-app
WORKDIR /sample-app

RUN python -m pip install --upgrade pip
RUN pip install --upgrade setuptools
RUN pip --version
RUN pip install -r requirements.txt

CMD [ "python", "runner.py" ]
CLI docker生成:-

docker build -t sample-app:1.0 .
错误:-

Step 8/9 : RUN pip install -r requirements.txt
 ---> Running in 13ab5873e0b5
Collecting boto3 (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/92/37/bcd816851f8a04de7d2ba9df5893acc4c6680fb3ab4442895490724143fb/boto3-1.7.67-py2.py3-none-any.whl (128kB)
Collecting pycryptodome (from -r requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/96/15/a2a2aa89f57a17075579fcd296a0fcff98640776666a4371d33f7f613bb9/pycryptodome-3.6.4.tar.gz (7.1MB)
Collecting snowflake-connector-python (from -r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/66/df/e44ec4d55fce341c9756f5f6012049b85f6a679fc3a423eefd7f66527a46/snowflake_connector_python-1.6.5-py2.py3-none-any.whl (142kB)
Collecting jmespath<1.0.0,>=0.7.1 (from boto3->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl
Collecting botocore<1.11.0,>=1.10.67 (from boto3->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/d6/81/59f34f9965277be8e480efea09b949e406b1afbfcf29a87c6ae6c2b52121/botocore-1.10.67-py2.py3-none-any.whl (4.4MB)
Collecting s3transfer<0.2.0,>=0.1.10 (from boto3->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl (59kB)
Collecting pyjwt (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/93/d1/3378cc8184a6524dc92993090ee8b4c03847c567e298305d6cf86987e005/PyJWT-1.6.4-py2.py3-none-any.whl
Collecting pyasn1<0.5.0,>=0.2.1 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl (72kB)
Collecting six (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting pyOpenSSL<18.0.0,>=16.2.0 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/79/db/7c0cfe4aa8341a5fab4638952520d8db6ab85ff84505e12c00ea311c3516/pyOpenSSL-17.5.0-py2.py3-none-any.whl (53kB)
Collecting cryptography<2.3,>=1.8.2 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/ec/b2/faa78c1ab928d2b2c634c8b41ff1181f0abdd9adf9193211bd606ffa57e2/cryptography-2.2.2.tar.gz (443kB)
Collecting azure-storage<=0.36.0,>0.34.2 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/76/26/482c033f8f3a48d16cf75aad91c3e1256856719f4117fabb4696d33aa5da/azure_storage-0.36.0-py2.py3-none-any.whl (190kB)
Collecting azure-common<2 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/91/66/6636001c0b6be5423f4bbaffe5febbf9eda39bd1941eb39a73504d8f6f71/azure_common-1.1.14-py2.py3-none-any.whl
Collecting ijson (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/7f/e9/8508c5f4987ba238a2b169e582c1f70a47272b22a2f1fb06b9318201bb9e/ijson-2.3-py2.py3-none-any.whl
Collecting cffi>=1.9 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/e7/a7/4cd50e57cc6f436f1cc3a7e8fa700ff9b8b4d471620629074913e3735fb2/cffi-1.11.5.tar.gz (438kB)
    Complete output from command python setup.py egg_info:

        No working compiler found, or bogus compiler options passed to
        the compiler from Python's standard "distutils" module.  See
        the error messages above.  Likely, the problem is not related
        to CFFI but generic to the setup.py of any Python package that
        tries to compile C code.  (Hints: on OS/X 10.8, for errors about
        -mno-fused-madd see http://stackoverflow.com/questions/22313407/
        Otherwise, see https://wiki.python.org/moin/CompLangPython or
        the IRC channel #python on irc.freenode.net.)

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-7dudc372/cffi/
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1
步骤8/9:运行pip安装-r requirements.txt
--->在13ab5873e0b5中运行
收集boto3(来自-r requirements.txt(第1行))
正在下载https://files.pythonhosted.org/packages/92/37/bcd816851f8a04de7d2ba9df5893acc4c6680fb3ab4442895490724143fb/boto3-1.7.67-py2.py3-none-any.whl (128kB)
收集pycryptodome(来自-r requirements.txt(第2行))
正在下载https://files.pythonhosted.org/packages/96/15/a2a2aa89f57a17075579fcd296a0fcff98640776666a4371d33f7f613bb9/pycryptodome-3.6.4.tar.gz (7.1MB)
收集雪花连接器python(来自-r requirements.txt(第3行))
正在下载https://files.pythonhosted.org/packages/66/df/e44ec4d55fce341c9756f5f6012049b85f6a679fc3a423eefd7f66527a46/snowflake_connector_python-1.6.5-py2.py3-none-any.whl (142kB)
收集jmespath=0.7.1(来自boto3->-r requirements.txt(第1行))
正在下载https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl
收集botocore=1.10.67(来自boto3->-r requirements.txt(第1行))
正在下载https://files.pythonhosted.org/packages/d6/81/59f34f9965277be8e480efea09b949e406b1afbfcf29a87c6ae6c2b52121/botocore-1.10.67-py2.py3-none-any.whl (4.4MB)
收集s3transfer=0.1.10(来自boto3->-r requirements.txt(第1行))
正在下载https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl (59kB)
收集pyjwt(来自雪花连接器python->-r requirements.txt(第3行))
正在下载https://files.pythonhosted.org/packages/93/d1/3378cc8184a6524dc92993090ee8b4c03847c567e298305d6cf86987e005/PyJWT-1.6.4-py2.py3-none-any.whl
收集pyasn1=0.2.1(来自雪花连接器python->-r requirements.txt(第3行))
正在下载https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl (72kB)
收集六个(来自雪花连接器python->-r requirements.txt(第3行))
正在下载https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
收集pyOpenSSL=16.2.0(来自雪花连接器python->-r requirements.txt(第3行))
正在下载https://files.pythonhosted.org/packages/79/db/7c0cfe4aa8341a5fab4638952520d8db6ab85ff84505e12c00ea311c3516/pyOpenSSL-17.5.0-py2.py3-none-any.whl (53kB)
收集加密=1.8.2(来自雪花连接器python->-r requirements.txt(第3行))
正在下载https://files.pythonhosted.org/packages/ec/b2/faa78c1ab928d2b2c634c8b41ff1181f0abdd9adf9193211bd606ffa57e2/cryptography-2.2.2.tar.gz (443kB)
收集azure-storage0.34.2(来自雪花连接器python->-r requirements.txt(第3行))
正在下载https://files.pythonhosted.org/packages/76/26/482c033f8f3a48d16cf75aad91c3e1256856719f4117fabb4696d33aa5da/azure_storage-0.36.0-py2.py3-none-any.whl (190kB)
收集azure-common-r requirements.txt(第3行))
正在下载https://files.pythonhosted.org/packages/91/66/6636001c0b6be5423f4bbaffe5febbf9eda39bd1941eb39a73504d8f6f71/azure_common-1.1.14-py2.py3-none-any.whl
收集ijson(来自雪花连接器python->-r requirements.txt(第3行))
正在下载https://files.pythonhosted.org/packages/7f/e9/8508c5f4987ba238a2b169e582c1f70a47272b22a2f1fb06b9318201bb9e/ijson-2.3-py2.py3-none-any.whl
收集cffi>=1.9(来自雪花连接器python->-r requirements.txt(第3行))
正在下载https://files.pythonhosted.org/packages/e7/a7/4cd50e57cc6f436f1cc3a7e8fa700ff9b8b4d471620629074913e3735fb2/cffi-1.11.5.tar.gz (438kB)
从命令python setup.py egg_info完成输出:
找不到可工作的编译器,或将伪编译器选项传递给
来自Python标准“distutils”模块的编译器。看见
上面的错误消息。这个问题很可能与此无关
对于CFFI,但对于任何
尝试编译C代码。(提示:在OS/X 10.8上,有关
-mno熔接马德西http://stackoverflow.com/questions/22313407/
否则,请参见https://wiki.python.org/moin/CompLangPython 或
IRC频道(IRC.freenode.net上的python)
----------------------------------------
命令“python setup.py egg_info”失败,错误代码为1,位于/tmp/pip-install-7dudc372/cffi中/
命令'/bin/sh-cpip install-r requirements.txt'返回一个非零代码:1
可能是什么问题?我是不是遗漏了什么


将编译库/包添加到您的alpine映像中,在上面的Dockerfile中添加下面的
RUN
语句,然后在
pip安装之前

WORKDIR /sample-app
RUN apk add --no-cache build-base libffi-dev openssl-dev ncurses-dev
RUN python -m pip install --upgrade pip

这基本上是安装gcc、g++&make来编译包,在您的情况下也需要其他包。我猜它们已经被删除了,或者没有与原始的
python:alpine3.6
image一起提供。

阿尔卑斯linux非常小,它们必须删除所有内容,所以您必须自己安装它。有些pip软件包需要在您的计算机上编译/构建,但alpine缺少这些依赖项,因此请安装它们,然后执行pip安装,然后删除它们,因为不再需要它们,您将减小映像大小

雪花还说您必须安装
libffi-dev
openssl-dev

FROM python:alpine3.6

COPY . /sample-app
WORKDIR /sample-app

RUN apk add --update --no-cache --virtual build-deps gcc python3-dev musl-dev libc-dev linux-headers libxslt-dev libxml2-dev \
&& apk add libffi-dev openssl-dev \
&& pip install --upgrade pip setuptools \
&& pip install -r requirements.txt \
&& apk del build-deps

CMD [ "python", "runner.py" ]
我发现,在为AWS lambda构建python部署包时,在linux之上使用会有所帮助,但我不确定这是否有帮助

我需要像Mazel Tov建议的那样
yum-y安装gcc-libffi-devel-openssl-devel
,以获得剩余的资源