Dockerfile生成过程中实例化Julia环境时出现问题

全部,

我在为开发设计的Docker容器中使用Julia。不同的开发人员在具有不同体系结构的各种主机上使用相同的Dockerfile。Julia安装在Docker映像中，但保存源的git Repo是在运行时装入的卷。它工作得很好，只是每次启动容器时，我们都会重新启动Julia环境。因此，我试图在构建时将实例化的环境烘焙到Docker映像中

在将Julia安装到Docker映像中之后，我将项目repo临时复制到映像中的一个目录中，该目录与运行时安装它们的位置相同。这是Dockerfile的一个片段

# syntax = edrevo/dockerfile-plus

ARG PROGRAM_NAME=wrong_program
ARG DOCKER_UBUNTU_VERSION=wrong_version
FROM ubuntu:${DOCKER_UBUNTU_VERSION}
SHELL ["/bin/bash", "-c"]

RUN apt-get update \
  && apt-get install -y -qq --no-install-recommends \
    libglvnd0 \
    libgl1 \
    libglx0 \
    libegl1 \
    libxext6 \
    libx11-6 \
    glmark2 \
    mesa-utils \
  && rm -rf /var/lib/apt/lists/*

# Env vars for the nvidia-container-runtime.
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES graphics,utility,compute

ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
    apt-get install -qqy --no-install-recommends \
    sudo \
    apt-utils \
    autotools-dev \
    build-essential \ 
    ca-certificates \
    g++ \
    git \
    iputils-ping \
    libssl-dev \
    nano \
    openssl \
    python-dev \
    unzip \
    vim \
    sed \
    x11-apps 

RUN update-ca-certificates

# Change to non-root privilege
# https://dev.to/emmanuelnk/using-sudo-without-password-prompt-as-non-root-docker-user-52bg
ARG USERNAME=wrongUser
ARG USERID=1000
ARG GID=101
ARG GROUPNAME=dev

RUN addgroup --gid ${GID} ${GROUPNAME}  \
    && adduser --uid ${USERID} --disabled-password --home /home/${USERNAME} --shell /bin/bash --gecos '' ${USERNAME} \
    && adduser  ${USERNAME} sudo \
    && adduser  ${USERNAME} dev  \
    && grep -qxF '%sudo ALL=(ALL) NOPASSWD:ALL' /etc/sudoers || echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers 

USER ${USERNAME}
RUN grep -qxF 'export LS_COLORS="$LS_COLORS:ow=1;34:tw=1;34:"' ~/.bashrc || echo 'export LS_COLORS="$LS_COLORS:ow=1;34:tw=1;34:"' >> ~/.bashrc

ARG TEMPDIR=${THIRDPARTYDIR}/temp 
RUN echo "THIRDPARTYDIR = ${THIRDPARTYDIR}" && \
    echo "TEMPDIR = ${TEMPDIR}"

#--------------------------------------
#  Install julia
#--------------------------------------
ARG JULIA_VERSION
ARG JULIA_DOWNLOAD_FILE
RUN echo "JULIA_VERSION = ${JULIA_VERSION}" && \
    echo "JULIA_DOWNLOAD_FILE = ${JULIA_DOWNLOAD_FILE}"

COPY --chown=${USERNAME}:${USERNAME} downloads/${JULIA_DOWNLOAD_FILE} ${THIRDPARTYDIR}/
RUN chown ${USERNAME}:${USERNAME} ${THIRDPARTYDIR}/${JULIA_DOWNLOAD_FILE} && \
    chmod 744 ${THIRDPARTYDIR}/${JULIA_DOWNLOAD_FILE} 
WORKDIR ${THIRDPARTYDIR}
RUN tar xf ${JULIA_DOWNLOAD_FILE} && \
    rm -rf ${JULIA_DOWNLOAD_FILE}

WORKDIR ${THIRDPARTYDIR}
RUN sudo ln -s ${THIRDPARTYDIR}/julia-${JULIA_VERSION}/bin/julia /usr/bin/julia && \ 
    sudo apt-get install -y wget
ENV CMAKE_PREFIX_PATH=${THIRDPARTYDIR}/julia-${JULIA_VERSION}:${CMAKE_PREFIX_PATH}

# instantiate all Julia Packages
RUN echo "intiantiating Julia Packages"
ARG WORKSPACEDIR
RUN mkdir -p ${WORKSPACEDIR}/src/
COPY --chown=${USERNAME}:${USERNAME} JuliaProjectManifests/ ${WORKSPACEDIR}/src/

RUN echo "WORKSPACEDIR = ${WORKSPACEDIR}" 

#RUN sudo chown -R ${USERNAME}:${GROUPNAME} /tmp
WORKDIR ${WORKSPACEDIR}/src/
COPY --chown=${USERNAME}:${USERNAME} ./julia_instantiate.sh ${WORKSPACEDIR}/src/
RUN chmod +x ./julia_instantiate.sh 
RUN source ./julia_instantiate.sh 
#RUN sudo -E -s \
#   && source ./julia_instantiate.sh \
#   && exit

WORKDIR ${THIRDPARTYDIR}
RUN rm -rf temp
RUN sudo ldconfig

WORKDIR ${THIRDPARTYDIR}/..

CMD ["bash"]

#!/bin/bash

cd ${WORKSPACEDIR}/src/;
folders=$(ls -d */) 
echo ${folders} 
for m in ${folders[@]}; do
        cd "${WORKSPACEDIR}/src/${m}"
        echo "instantiating: ${WORKSPACEDIR}/src/${m}"
        julia --project=. -e "using Pkg; Pkg.instantiate(); using $(echo "${m}" | tr -d /); exit();"
done

为了帮助澄清更多。。。ARG环境变量在启动生成时在命令行上定义。该脚本看起来像：

DOCKER_BUILDKIT=1 docker build                                                            \
    --no-cache=false                                                          \
    --network=host   \
    --rm                                                                \
    --build-arg PROGRAM_NAME=${PROGRAM_NAME}                                    \
    --build-arg WORKSPACEDIR=${WORKSPACEDIR}                                    \
    --build-arg USERNAME=${USER}                                    \
    --build-arg USERID=${UID}                                    \
    --build-arg THIRDPARTYDIR=${THIRDPARTYDIR}                                    \
    --build-arg DOCKER_UBUNTU_VERSION=${DOCKER_UBUNTU_VERSION}                                    \
    --build-arg JULIA_VERSION=${JULIA_VERSION}                    \
    --build-arg JULIA_DOWNLOAD_FILE=${JULIA_DOWNLOAD_FILE}            \
    -f ${CDDS}/${DOCKER_BUILD_FOLDER}/${DOCKER_FILE_NAME} \
    -t ${DOCKER_IMAGE} \
    ${DOCKER_BUILD_CONTEXT} \
    2>&1 | tee ${CDDS}/${DOCKER_BUILD_FOLDER}/${DOCKER_BUILD_LOG}

当我尝试在docker构建期间实例化项目时，我得到以下错误：

#115 sha256:0c589043bbee81f6e25ae6da0177acfea129c478ba518ca117a39838dc68d6f1
#115 0.148 iKinQP/
#115 0.924  Installing known registries into `~/.julia`
#115 1.032 ERROR: SystemError: mktemp: No such file or directory
#115 1.703 Stacktrace:
#115 1.935  [1] systemerror(::Symbol, ::Int32; extrainfo::Nothing) at ./error.jl:168
#115 2.272  [2] #systemerror#48 at ./error.jl:167 [inlined]
#115 2.279  [3] systemerror at ./error.jl:167 [inlined]
#115 2.279  [4] #mktemp#18 at ./file.jl:589 [inlined]
#115 2.279  [5] mktemp at ./file.jl:587 [inlined] (repeats 2 times)
#115 2.298  [6] probe_platform_engines!(; verbose::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/PlatformEngines.jl:280
#115 2.299  [7] probe_platform_engines! at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/PlatformEngines.jl:181 [inlined]
#115 2.299  [8] pkg_server_registry_urls() at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:944
#115 2.349  [9] clone_default_registries(::Pkg.Types.Context; only_if_empty::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:868
#115 2.350  [10] clone_default_registries at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:862 [inlined]
#115 2.350  [11] find_registered!(::Pkg.Types.Context, ::Array{String,1}, ::Array{Base.UUID,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:1239
#115 2.355  [12] find_registered! at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:1200 [inlined]
#115 2.355  [13] check_registered(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Operations.jl:1054
#115 2.356  [14] instantiate(::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Pkg.BinaryPlatforms.Linux, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/API.jl:833
#115 2.403  [15] instantiate(::Pkg.Types.Context) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/API.jl:795
#115 2.404  [16] #instantiate#169 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/API.jl:791 [inlined]
#115 2.404  [17] instantiate() at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/API.jl:791
#115 2.404  [18] top-level scope at none:1
#115 DONE 2.5s

下面是julia_instantiate.sh由Dockerfile运行

# syntax = edrevo/dockerfile-plus

ARG PROGRAM_NAME=wrong_program
ARG DOCKER_UBUNTU_VERSION=wrong_version
FROM ubuntu:${DOCKER_UBUNTU_VERSION}
SHELL ["/bin/bash", "-c"]

RUN apt-get update \
  && apt-get install -y -qq --no-install-recommends \
    libglvnd0 \
    libgl1 \
    libglx0 \
    libegl1 \
    libxext6 \
    libx11-6 \
    glmark2 \
    mesa-utils \
  && rm -rf /var/lib/apt/lists/*

# Env vars for the nvidia-container-runtime.
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES graphics,utility,compute

ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
    apt-get install -qqy --no-install-recommends \
    sudo \
    apt-utils \
    autotools-dev \
    build-essential \ 
    ca-certificates \
    g++ \
    git \
    iputils-ping \
    libssl-dev \
    nano \
    openssl \
    python-dev \
    unzip \
    vim \
    sed \
    x11-apps 

RUN update-ca-certificates

# Change to non-root privilege
# https://dev.to/emmanuelnk/using-sudo-without-password-prompt-as-non-root-docker-user-52bg
ARG USERNAME=wrongUser
ARG USERID=1000
ARG GID=101
ARG GROUPNAME=dev

RUN addgroup --gid ${GID} ${GROUPNAME}  \
    && adduser --uid ${USERID} --disabled-password --home /home/${USERNAME} --shell /bin/bash --gecos '' ${USERNAME} \
    && adduser  ${USERNAME} sudo \
    && adduser  ${USERNAME} dev  \
    && grep -qxF '%sudo ALL=(ALL) NOPASSWD:ALL' /etc/sudoers || echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers 

USER ${USERNAME}
RUN grep -qxF 'export LS_COLORS="$LS_COLORS:ow=1;34:tw=1;34:"' ~/.bashrc || echo 'export LS_COLORS="$LS_COLORS:ow=1;34:tw=1;34:"' >> ~/.bashrc

ARG TEMPDIR=${THIRDPARTYDIR}/temp 
RUN echo "THIRDPARTYDIR = ${THIRDPARTYDIR}" && \
    echo "TEMPDIR = ${TEMPDIR}"

#--------------------------------------
#  Install julia
#--------------------------------------
ARG JULIA_VERSION
ARG JULIA_DOWNLOAD_FILE
RUN echo "JULIA_VERSION = ${JULIA_VERSION}" && \
    echo "JULIA_DOWNLOAD_FILE = ${JULIA_DOWNLOAD_FILE}"

COPY --chown=${USERNAME}:${USERNAME} downloads/${JULIA_DOWNLOAD_FILE} ${THIRDPARTYDIR}/
RUN chown ${USERNAME}:${USERNAME} ${THIRDPARTYDIR}/${JULIA_DOWNLOAD_FILE} && \
    chmod 744 ${THIRDPARTYDIR}/${JULIA_DOWNLOAD_FILE} 
WORKDIR ${THIRDPARTYDIR}
RUN tar xf ${JULIA_DOWNLOAD_FILE} && \
    rm -rf ${JULIA_DOWNLOAD_FILE}

WORKDIR ${THIRDPARTYDIR}
RUN sudo ln -s ${THIRDPARTYDIR}/julia-${JULIA_VERSION}/bin/julia /usr/bin/julia && \ 
    sudo apt-get install -y wget
ENV CMAKE_PREFIX_PATH=${THIRDPARTYDIR}/julia-${JULIA_VERSION}:${CMAKE_PREFIX_PATH}

# instantiate all Julia Packages
RUN echo "intiantiating Julia Packages"
ARG WORKSPACEDIR
RUN mkdir -p ${WORKSPACEDIR}/src/
COPY --chown=${USERNAME}:${USERNAME} JuliaProjectManifests/ ${WORKSPACEDIR}/src/

RUN echo "WORKSPACEDIR = ${WORKSPACEDIR}" 

#RUN sudo chown -R ${USERNAME}:${GROUPNAME} /tmp
WORKDIR ${WORKSPACEDIR}/src/
COPY --chown=${USERNAME}:${USERNAME} ./julia_instantiate.sh ${WORKSPACEDIR}/src/
RUN chmod +x ./julia_instantiate.sh 
RUN source ./julia_instantiate.sh 
#RUN sudo -E -s \
#   && source ./julia_instantiate.sh \
#   && exit

WORKDIR ${THIRDPARTYDIR}
RUN rm -rf temp
RUN sudo ldconfig

WORKDIR ${THIRDPARTYDIR}/..

CMD ["bash"]

#!/bin/bash

cd ${WORKSPACEDIR}/src/;
folders=$(ls -d */) 
echo ${folders} 
for m in ${folders[@]}; do
        cd "${WORKSPACEDIR}/src/${m}"
        echo "instantiating: ${WORKSPACEDIR}/src/${m}"
        julia --project=. -e "using Pkg; Pkg.instantiate(); using $(echo "${m}" | tr -d /); exit();"
done

• Docker镜像操作系统：Ubuntu 18.04
• 主机操作系统：WSL2 Ubuntu 20.04
• Docker版本：20.10.6，构建370c289
• Julia版本：1.5.3

朱莉娅的问题似乎有关联，但它并没有给我一个解决办法

如何解决这个问题

编辑： 另一点是，如果我将julia_instantiate.sh复制到docker映像中，但在构建过程中不执行它，则会在映像中获得一个包含脚本和julia包源文件的映像。如果我以交互方式运行这个容器并手动运行julia_instantiate.sh，它运行得很好。我运行容器并作为烘焙到docker图像中的同一用户附加。因此，只有当docker build运行脚本时，流程才会失败

---

注意：我已经将*julia_instantiate.sh更新到我一直在测试的最小值。我还添加了一些关于Dockerfile的附加信息。

什么是基本Dockerfile？您是在使用julia docker的官方图像，还是在dockerfile中自己安装julia？Julia应该自动创建~/.Julia目录（如果不存在）。如果目录已经存在，可能是权限问题。整个dockerfile看起来怎么样？你使用的是root用户还是其他用户？基本版本是Ubuntu 18.04。我正在手动安装julia。忽略~/.julia的创建（我在调试过程中就把它放进去了）。自从发布后，我已经删除了它，文件夹创建得很好。尝试实例化时，我不是root。在Dockerfile之前，我创建了一个用户，并在安装所有内容时切换到该用户。正如我提到的，我已经使用这个Docker映像好几个月了，所有的应用程序都工作得很好。这只是在docker构建过程中尝试实例化我的Julia环境的最新步骤失败。请确定，是该用户的~home目录，而不是root目录吗？julia是否安装在该用户下？该用户是否具有写入/tmp的权限？如果您使用

sudo

，实例化是否有效？如果您使用root用户，它能工作吗？感谢您发布整个dockerfile，如果在那之前无法解决，我将在下周尝试查看。@Matej，谢谢您的帮助。是，~指向用户的主目录，而不是根目录。Julia安装在用户下，而不是根目录下。在以$USERNAME运行时，我在

运行source./julia_instantiate.sh

之前添加了

RUN sudo chown-R${USERNAME}:${GROUPNAME}/tmp

，这并没有什么区别。然后，我删除了该命令并使用了以下命令，

运行sudo-E-s&&source./julia_instantiate.sh&&exit

相同的错误。。。所以，即使以root用户身份运行脚本，并且可以访问所有环境，也并没有改变任何事情。顺便说一句：这不是整个Dockerfile，而是最相关的部分。。。