Docker映像构建中的Python statsmodels需求问题

我正试图为AWS批处理作业构建Docker映像，但一个新的需求（Python包

statsmodels

）正在破坏我的构建过程

requirements.txt

行：

pandas
numpy
statsmodels

完成

Dockerfile

：

FROM python:3

COPY requirements.txt /home/
COPY .env /home/
COPY src home/src
COPY process.py /home/

WORKDIR /home/
ENV TZ=America/New_York
RUN /usr/local/bin/python -m pip install --upgrade pip
RUN pip install -r ./requirements.txt

ENV PYTHONPATH='.' 

错误日志摘录：

#12 271.7 INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
#12 288.4 INFO: pip is looking at multiple versions of us to determine which version is compatible with other requirements. This could take a while.
#12 288.4 Collecting us
#12 288.4   Downloading us-2.0.1.tar.gz (13 kB)
#12 288.8   Downloading us-2.0.0.tar.gz (13 kB)
#12 289.3   Downloading us-1.0.0.tar.gz (13 kB)
#12 289.8   Downloading us-0.9.1.tar.gz (13 kB)
#12 290.2   Downloading us-0.9.0.tar.gz (13 kB)
#12 290.6   Downloading us-0.8.0.tar.gz (12 kB)
#12 291.2   Downloading us-0.7.1.tar.gz (12 kB)
#12 291.5 INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
#12 291.5 INFO: pip is looking at multiple versions of us to determine which version is compatible with other requirements. This could take a while.
#12 291.6   Downloading us-0.7.tar.gz (11 kB)
#12 291.9   Downloading us-0.6.tar.gz (9.9 kB)
#12 292.3   Downloading us-0.5.tar.gz (9.3 kB)
#12 292.7   Downloading us-0.4.tar.gz (9.3 kB)
#12 293.0   Downloading us-0.3.tar.gz (7.5 kB)
#12 293.4 INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
#12 293.4 INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
#12 293.4   Downloading us-0.2.tar.gz (6.9 kB)
#12 294.0   Downloading us-0.1.tar.gz (6.3 kB)
#12 294.4 INFO: pip is looking at multiple versions of statsmodels to determine which version is compatible with other requirements. This could take a while.
#12 294.9 Collecting statsmodels
#12 294.9   Downloading statsmodels-0.12.0.tar.gz (17.5 MB)
#12 297.5   Installing build dependencies: started
#12 309.5   Installing build dependencies: finished with status 'done'
#12 309.5   Getting requirements to build wheel: started
#12 392.2   Getting requirements to build wheel: still running...
#12 400.0   Getting requirements to build wheel: finished with status 'done'
#12 400.1     Preparing wheel metadata: started
#12 401.3     Preparing wheel metadata: finished with status 'done'
#12 401.9   Downloading statsmodels-0.11.1.tar.gz (15.4 MB)
#12 404.2   Installing build dependencies: started
#12 415.7   Installing build dependencies: finished with status 'done'
#12 415.7   Getting requirements to build wheel: started
#12 480.8   Getting requirements to build wheel: still running...
#12 481.5   Getting requirements to build wheel: finished with status 'done'
#12 481.5     Preparing wheel metadata: started
#12 482.6     Preparing wheel metadata: finished with status 'done'
#12 483.1   Downloading statsmodels-0.11.0.tar.gz (15.4 MB)
#12 485.5   Installing build dependencies: started
#12 488.6   Installing build dependencies: finished with status 'error'
#12 488.6   ERROR: Command errored out with exit status 1:
#12 488.6    command: /usr/local/bin/python /usr/local/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-_0xxe4fy/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'cython>=0.29.14' 'numpy==1.14.5; python_version=='"'"'3.5'"'"'' 'numpy==1.14.5; python_version=='"'"'3.6'"'"'' 'numpy==1.14.5; python_version>='"'"'3.7'"'"'' 'numpy==1.17.5; python_version>='"'"'3.8'"'"'' 'scipy>=1.0'
#12 488.6        cwd: None
#12 488.6   Complete output (13 lines):
#12 488.6   Ignoring numpy: markers 'python_version == "3.5"' don't match your environment
#12 488.6   Ignoring numpy: markers 'python_version == "3.6"' don't match your environment
#12 488.6   ERROR: Cannot install numpy==1.14.5 and numpy==1.17.5 because these package versions have conflicting dependencies.
#12 488.6   
#12 488.6   The conflict is caused by:
#12 488.6       The user requested numpy==1.14.5
#12 488.6       The user requested numpy==1.17.5
#12 488.6   
#12 488.6   To fix this you could try to:
#12 488.6   1. loosen the range of package versions you've specified
#12 488.6   2. remove package versions to allow pip attempt to solve the dependency conflict
#12 488.6   
#12 488.6   ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
#12 488.6   ----------------------------------------
#12 488.6 ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-_0xxe4fy/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'cython>=0.29.14' 'numpy==1.14.5; python_version=='"'"'3.5'"'"'' 'numpy==1.14.5; python_version=='"'"'3.6'"'"'' 'numpy==1.14.5; python_version>='"'"'3.7'"'"'' 'numpy==1.17.5; python_version>='"'"'3.8'"'"'' 'scipy>=1.0' Check the logs for full command output.
------
failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c pip install -r ./requirements.txt]: runc did not terminate sucessfully

更新进程的错误日志：

[+] Building 0.7s (7/12)                                                        
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 979B                                       0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/library/debian:10-slim          0.6s
 => [internal] load build context                                          0.0s
 => => transferring context: 15.91kB                                       0.0s
 => CANCELED [1/8] FROM docker.io/library/debian:10-slim@sha256:062bbd9a1  0.0s
 => => resolve docker.io/library/debian:10-slim@sha256:062bbd9a1a58c9c5b8  0.0s
 => CACHED [2/8] RUN apt-get update     && apt-get install --no-install-r  0.0s
 => ERROR [3/8] COPY .env /root/.env  # this may need to go into /root/.e  0.0s
------
 > [3/8] COPY .env /root/.env  # this may need to go into /root/.env (or a custom user):
------
failed to solve with frontend dockerfile.v0: failed to build LLB: failed to compute cache key: "/#" not found: not found

---

从Dockerfile中还不完全清楚为什么（我很惊讶标签

3

不足以选择最新版本的

3.9.1-buster

），但是

pip

正试图解决多个Python版本的依赖关系（意味着在基本容器中安装了多个）

#12 488.6错误：命令出错，退出状态为1:/usr/local/bin/python/usr/local/lib/python3.9/site-packages/pip安装--忽略已安装--无用户--前缀/tmp/pip build env-_0xxe4fy/overlay--无警告脚本位置--无二进制：无：-仅二进制：无：-i--setuptools wheel'cython>=0.29.14''numpy==1.14.5；python_version=='''''3.5'''''numpy==1.14.5；python_version=='''''3.6'''''numpy==1.14.5；python_version>='''''3.7'''''numpy==1.17.5；python_version>='''''''3.8''''''''''scipy>=1.0'检查日志以获取完整的命令输出

指定次要版本可能会清除此问题

如果您没有特定的版本需要，（Debian 10和Python 3.8，目前实际上是3.8.9，尽管这可能会随着该版本的错误修复而改变）可能是一个实际的选择，并为您解决这个问题

from python:3.8-slim-buster
...

---

如果这不能修复错误，那么很可能您的

3

标记很好，相反，您的

.env

极有可能有问题（可能是您试图保留的Python虚拟环境）

使用新的

运行

块，而不是

复制主机系统的内容，在容器上下文中重新生成venv


在评论之后，我认为值得尝试一个不同的Dockerfile，它比我使用的要多得多，至少有以下更改


直接以Debian 10为基础（您可能会发现另一个容器更高效/更小/随便什么，但这很可能是开箱即用的）
自己安装Python3
不要使用
/home/
作为基础

如果这行得通的话


使用
/home
可能有问题（而不是
/home/someuser/…
）
上游的
python
base容器可能有一些奇怪的地方（我个人从未使用过它，不过这也会让人惊讶）

如果这不起作用


您可能没有在您认为自己所在的目录中工作（例如，在中复制当前目录、另一个外观类似的目录等）
您的
src
可能有一些奇怪的地方
statsmodels
可能已损坏（版本为0.12，因此这并非完全不合理）
您好，谢谢您的回复。在尝试使用
3.8-slim-buster
运行时，我遇到了类似的错误，但现在
modulenofounderror:没有名为'numpy'的模块
。在本例中，My
.env
仅是两行指定Airtable API参数（Airtable_BASE=xxx…），无漏洞。神秘-尝试删除
PYTHONPATH
并简单调用
python3
，而不是任何对
/usr/local
Hmm的引用，在删除
PYTHONPATH
行并调用
运行python3-m pip安装-升级pip
后，我返回到初始post中记录的错误。。。还有其他想法吗？@OJT我用另一个Dockerfile和一些想法做了一个大编辑，这可能对你有用。我已经更新了我的Dockerfile内容和问题中的错误日志。一般来说，我对这些东西不是特别有经验；这里的具体区别是什么，以及它如何影响在AWS Batch EC2实例环境中运行（例如，使用
/app/
与实例
/home/
目录，正如我们在其他不需要
statsmodels
的停靠进程中成功使用的那样）？
from python:3.8-slim-buster
...

FROM debian:10-slim

# install Python, pip, and misc build needs
RUN apt-get update \
    && apt-get install --no-install-recommends -y \
        build-essential \
        python3-dev \
        python3-pip \
        python3-setuptools \
    && apt-get clean

# bring in program logic
COPY .env /app/  # this may need to go into /root/.env (or a custom user)
COPY src /app/
COPY process.py /app/
COPY requirements.txt /app/

# install Python dependencies
RUN pip3 install --no-cache-dir -r /app/requirements.txt

# set env vars
ENV TZ=America/New_York

# running process logic
WORKDIR /app
ENTRYPOINT ["python3", "process.py"]