Python 使用Docker compose启动Jupyter笔记本时,在Docker容器内激活conda env
我有以下Python 使用Docker compose启动Jupyter笔记本时,在Docker容器内激活conda env,python,docker,docker-compose,conda,Python,Docker,Docker Compose,Conda,我有以下Dockerfile FROM continuumio/miniconda3:4.5.11 # create a new user (defaults to 'al-khawarizmi') USER root ARG username=al-khawarizmi RUN useradd --create-home --home-dir /home/${username} ${username} ENV HOME /home/${username} # switch to newly
Dockerfile
FROM continuumio/miniconda3:4.5.11
# create a new user (defaults to 'al-khawarizmi')
USER root
ARG username=al-khawarizmi
RUN useradd --create-home --home-dir /home/${username} ${username}
ENV HOME /home/${username}
# switch to newly created user to avoid running container as root
USER ${username}
WORKDIR $HOME
# build and activate the specified conda environment from a file (defaults to 'environment.yml')
ARG environment=environment.yml
COPY ${environment} .
RUN conda env create --file ${environment} && \
echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
echo "conda activate $(head -1 ${environment} | cut -d' ' -f2)" >> ~/.bashrc
name: nessie-py
channels:
- conda-forge
- defaults
dependencies:
- python=3.6
- "notebook=5.7.*"
- "matplotlib=3.0.*"
- "numpy=1.15.*"
- "pandas=0.23.*"
Dockerfile允许用户将conda环境文件指定为生成参数。下面是一个典型的environment.yml文件
FROM continuumio/miniconda3:4.5.11
# create a new user (defaults to 'al-khawarizmi')
USER root
ARG username=al-khawarizmi
RUN useradd --create-home --home-dir /home/${username} ${username}
ENV HOME /home/${username}
# switch to newly created user to avoid running container as root
USER ${username}
WORKDIR $HOME
# build and activate the specified conda environment from a file (defaults to 'environment.yml')
ARG environment=environment.yml
COPY ${environment} .
RUN conda env create --file ${environment} && \
echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
echo "conda activate $(head -1 ${environment} | cut -d' ' -f2)" >> ~/.bashrc
name: nessie-py
channels:
- conda-forge
- defaults
dependencies:
- python=3.6
- "notebook=5.7.*"
- "matplotlib=3.0.*"
- "numpy=1.15.*"
- "pandas=0.23.*"
用户可以标准方式运行图像,conda环境将自动激活。运行
$ docker run -it image_name:image_tag
在激活conda环境的Docker容器中生成bash提示符
(environment_name)$
现在,我想使用docker compose
在容器中启动一个Jupyter笔记本服务器(使用conda环境文件构建,该文件将Jupyter指定为依赖项)
当我使用以下docker compose.yml
version: "3.7"
services:
notebook-server:
build:
context: ./
ports:
- "8888:8888"
volumes:
- ./:/home/al-khawarizmi
command: jupyter notebook --no-browser ip=0.0.0.0
我得到以下错误
$ docker-compose up
Creating network "nessie-py_default" with the default driver
Creating nessie-py_notebook-server_1 ... done
Attaching to nessie-py_notebook-server_1
notebook-server_1 | [FATAL tini (7)] exec jupyter failed: No such file or directory
nessie-py_notebook-server_1 exited with code 127
我怀疑这个错误意味着康达环境没有被激活。然后我尝试将tty:true
和stdin\u open:true
添加到docker compose.yml
中,认为这应该在运行命令之前调用和交互bash提示符。这导致了与上述相同的错误
我还尝试定义一个start notebook.sh
脚本,在运行笔记本之前显式激活conda环境
#!/bin/bash
set -e
# activate the environment and start the notebook
conda activate nessie-py
jupyter notebook --no-browser ip=0.0.0.0
导致不同的错误
$ docker-compose up
Creating network "nessie-py_default" with the default driver
Creating nessie-py_notebook-server_1 ... done
Attaching to nessie-py_notebook-server_1
notebook-server_1 |
notebook-server_1 | CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
notebook-server_1 | If your shell is Bash or a Bourne variant, enable conda for the current user with
notebook-server_1 |
notebook-server_1 | $ echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc
notebook-server_1 |
notebook-server_1 | or, for all users, enable conda with
notebook-server_1 |
notebook-server_1 | $ sudo ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh
notebook-server_1 |
notebook-server_1 | The options above will permanently enable the 'conda' command, but they do NOT
notebook-server_1 | put conda's base (root) environment on PATH. To do so, run
notebook-server_1 |
notebook-server_1 | $ conda activate
notebook-server_1 |
notebook-server_1 | in your terminal, or to put the base environment on PATH permanently, run
notebook-server_1 |
notebook-server_1 | $ echo "conda activate" >> ~/.bashrc
notebook-server_1 |
notebook-server_1 | Previous to conda 4.4, the recommended way to activate conda was to modify PATH in
notebook-server_1 | your ~/.bashrc file. You should manually remove the line that looks like
notebook-server_1 |
notebook-server_1 | export PATH="/opt/conda/bin:$PATH"
notebook-server_1 |
notebook-server_1 | ^^^ The above line should NO LONGER be in your ~/.bashrc file! ^^^
notebook-server_1 |
notebook-server_1 |
nessie-py_notebook-server_1 exited with code 1
此错误表明在运行脚本之前,bash
未在源代码~/.bashrc
在激活conda环境之前,我尝试显式地寻找/opt/conda/etc/profile.d/conda.sh
#!/bin/bash
set -e
# activate the environment and start the notebook
. /opt/conda/etc/profile.d/conda.sh
conda activate nessie-py
jupyter notebook --no-browser ip=0.0.0.0
这会导致不同的错误
$ docker-compose up
Creating network "nessie-py_default" with the default driver
Creating nessie-py_notebook-server_1 ... done
Attaching to nessie-py_notebook-server_1
notebook-server_1 | Could not find conda environment: nessie-py
notebook-server_1 | You can list all discoverable environments with `conda info --envs`.
notebook-server_1 |
nessie-py_notebook-server_1 exited with code 1
我可以通过运行来检查容器中可发现哪些conda环境
$ docker run -it nessie-py conda info --envs
也就是说环境确实存在
$ docker run -it nessie-py_notebook-server conda info --envs
# conda environments:
#
nessie-py /home/al-khawarizmi/.conda/envs/nessie-py
base * /opt/conda
我现在没有主意了。这应该是可能的。是一个具有docker compose.yml
文件的项目示例,该文件是一个Dockerfile,用于指定conda环境并启动Jupyter笔记本服务器
我需要的额外复杂性包括向Dockerfile添加一个非root用户,并创建一个新的conda环境,而不是更新默认的base
conda环境。发生的情况是:
#!/bin/bash
set -e
# activate the environment and start the notebook
. /opt/conda/etc/profile.d/conda.sh
conda activate nessie-py
jupyter notebook --no-browser ip=0.0.0.0
在docker compose.yml
中,您在ip=0.0.0
中有一个输入错误,应该是--ip=0.0.0
将主机文件夹绑定到容器将覆盖.bashrc
。一个简单的改变是装入一个子目录
您需要以交互模式(-i
)运行bash,以便正确读取.bashrc
例如,docker compose.yml
中反映的这些点上的更改:
version: "3.7"
services:
notebook-server:
build:
context: ./
ports:
- "8888:8888"
volumes:
- ./:/home/al-khawarizmi/hosthome
command: bash -ic 'jupyter notebook --no-browser --ip=0.0.0.0'
你能发布你的environment.yml
的内容吗?添加了一个存根environment.yml
文件。在我看来,错误来自ip=0.0.0
参数,它应该是--ip=0.0.0
。@fernandezcuesta感谢你抓到了输入错误!但这不会影响我收到的任何错误消息。关键问题似乎在于激活conda环境。您是如何发现将主机文件夹绑定到容器中的操作将覆盖。bashrc
?如中所述