使用特定的Miniconda Python和NumPy版本进行CircleCI测试
我正在从事一个使用CircleCI持续集成平台的项目。我使用Python作为主语言,Miniconda作为平台。我想在CircleCI上使用Miniconda测试多个Python和NumPy版本 我尝试使用不同的Python映像,但在安装最新的Miniconda版本时,它只使用Python3.7。你能告诉我如何使用多个版本吗 下面是使用特定的Miniconda Python和NumPy版本进行CircleCI测试,python,circleci,circleci-2.0,Python,Circleci,Circleci 2.0,我正在从事一个使用CircleCI持续集成平台的项目。我使用Python作为主语言,Miniconda作为平台。我想在CircleCI上使用Miniconda测试多个Python和NumPy版本 我尝试使用不同的Python映像,但在安装最新的Miniconda版本时,它只使用Python3.7。你能告诉我如何使用多个版本吗 下面是config.yml: version: 2.0 workflows: version: 2 test: jobs: - py3.6-np
config.yml
:
version: 2.0
workflows:
version: 2
test:
jobs:
- py3.6-np1.15
- py3.5-np1.15
- py3.6-np1.14
- py3.5-np1.14
- py3.7-np1.15
- py3.5-np1.16
- py3.6-np1.16
- py3.7-np1.16
jobs:
py3.6-np1.15: &test-template
docker:
- image: circleci/python:3.6.8
environment:
NUMPY_VERSION: 1.15.2
CYTHON_VERSION: 0.29.2
working_directory: ~/repo
steps:
- checkout
- run:
name: Install System Dependencies
command: sudo apt-get update && sudo apt-get install -y libmpich12 libmpich-dev build-essential
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ .Environment.CIRCLE_JOB }}-{{ checksum "setup.py" }}
- run:
name: install anaconda
command: |
wget https://repo.continuum.io/miniconda/Miniconda3-4.7.10-Linux-x86_64.sh -O ~/miniconda.sh
chmod +x ~/miniconda.sh && ~/miniconda.sh -b
export PATH=$HOME/miniconda3/bin:$PATH
conda update --quiet --yes conda
- run:
name: Install numpy, cython, mdtraj
command: |
export PATH=$HOME/miniconda3/bin:$PATH
conda install --quiet --yes numpy==$NUMPY_VERSION cython==$CYTHON_VERSION
conda install --quiet --yes -c conda-forge mdtraj
# - run:
# name: Upgrade pip
# command: |
# python3 -m venv venv
# . venv/bin/activate
# pip install pip==18.0
# - run:
# name: Install numpy and cython
# command: |
# python3 -m venv venv
# . venv/bin/activate
# pip install --progress-bar off numpy==$NUMPY_VERSION cython==$CYTHON_VERSION
- run:
name: Install and build
command: |
export PATH=$HOME/miniconda3/bin:$PATH
pip install --progress-bar off .[dev]
python setup.py build_ext --inplace
python setup.py install
py3.5-np1.15:
<<: *test-template
docker:
- image: circleci/python:3.5.7
environment:
NUMPY_VERSION: 1.14.2
CYTHON_VERSION: 0.29.2
py3.6-np1.14:
<<: *test-template
environment:
NUMPY_VERSION: 1.14.2
CYTHON_VERSION: 0.29.2
py3.5-np1.14:
<<: *test-template
docker:
- image: circleci/python:3.5.7
environment:
NUMPY_VERSION: 1.14.2
CYTHON_VERSION: 0.29.2
py3.7-np1.15:
<<: *test-template
docker:
- image: circleci/python:3.7.3
py3.5-np1.16:
<<: *test-template
docker:
- image: circleci/python:3.5.7
environment:
NUMPY_VERSION: 1.16.5
CYTHON_VERSION: 0.29.2
py3.6-np1.16:
<<: *test-template
environment:
NUMPY_VERSION: 1.16.5
CYTHON_VERSION: 0.29.2
py3.7-np1.16:
<<: *test-template
docker:
- image: circleci/python:3.7.3
environment:
NUMPY_VERSION: 1.16.5
CYTHON_VERSION: 0.29.2
版本:2.0
工作流程:
版本:2
测试:
工作:
-py3.6-np1.15
-py3.5-np1.15
-py3.6-np1.14
-py3.5-np1.14
-py3.7-np1.15
-py3.5-np1.16
-py3.6-np1.16
-py3.7-np1.16
工作:
py3.6-np1.15:&测试模板
码头工人:
-图:circleci/python:3.6.8
环境:
NUMPY_版本:1.15.2
CYTHON_版本:0.29.2
工作目录:~/repo
步骤:
-结帐
-运行:
名称:安装系统依赖项
命令:sudo apt get update&&sudo apt get install-y libmpich12 libmpich dev build-sential
#下载和缓存依赖项
-还原U缓存:
钥匙:
-v1依赖项-{{.Environment.CIRCLE_JOB}}-{{{checksum“setup.py”}
-运行:
名称:安装anaconda
命令:|
wgethttps://repo.continuum.io/miniconda/Miniconda3-4.7.10-Linux-x86_64.sh -O~/miniconda.sh
chmod+x~/miniconda.sh&~/miniconda.sh-b
导出路径=$HOME/miniconda3/bin:$PATH
康达更新-安静-是康达
-运行:
名称:安装numpy、cython、mdtraj
命令:|
导出路径=$HOME/miniconda3/bin:$PATH
conda安装--安静--是numpy==$numpy\u版本cython==$cython\u版本
conda安装--安静--是-c conda forge mdtraj
#-运行:
#名称:升级pip
#命令:|
#蟒蛇3-m静脉
# . venv/bin/激活
#pip安装pip==18.0
#-运行:
#名称:安装numpy和cython
#命令:|
#蟒蛇3-m静脉
# . venv/bin/激活
#pip安装--进程条关闭numpy==$numpy\u版本cython==$cython\u版本
-运行:
名称:安装和生成
命令:|
导出路径=$HOME/miniconda3/bin:$PATH
pip安装--关闭进度条。[dev]
python setup.py build_ext--inplace
python setup.py安装
py3.5-np1.15:
下面是一个关于如何将CircleCI与Miniconda以及特定Python和NumPy版本一起使用的最简单配置示例,从空的ubuntu:bionic
图像开始
version: 2
jobs:
build:
docker:
- image: ubuntu:bionic
environment:
PYTHON_VERSION: 3.5.5
NUMPY_VERSION: 1.14.2
steps:
- checkout
- run:
name: Setup Miniconda
command: |
apt update
apt install -y wget
cd $HOME
wget "https://repo.anaconda.com/miniconda/Miniconda3-4.7.10-Linux-x86_64.sh" -O miniconda.sh
printf '%s' "8a324adcc9eaf1c09e22a992bb6234d91a94146840ee6b11c114ecadafc68121 miniconda.sh" | sha256sum -c
bash miniconda.sh -b -p $HOME/miniconda
- run:
name: Setup environment and run tests
command: |
export PATH="$HOME/miniconda/bin:$PATH"
conda update -y conda
conda create -n myenv python=$PYTHON_VERSION -c conda-forge
source activate myenv
conda install -y numpy=$NUMPY_VERSION
python --version
python -c "import numpy; print(numpy.__version__)"
我认为在从internet下载Miniconda安装脚本Miniconda3-4.7.10-Linux-x86_64.sh
后验证校验和是一种很好的做法
您可以更改环境变量PYTHON\u VERSION
和NUMPY\u VERSION
以获得其他版本
除了“真正的”测试之外,目前我们只是要验证我们所需的Python和NumPy版本是否与Python--version
和Python-c“import NumPy;print(NumPy.\uuu version\uuuu)”一起使用
。对于上面的示例,在日志末尾,您应该找到:
Python 3.5.5
1.14.2
根据您选择的版本,可能会出现以下错误:
- 如果收到
PackagesNotFoundError
,则需要确保所选频道具有您要查找的软件包版本。(如上例中选择了conda forge
。)
unsatifiableerror
,则表明您选择了不兼容的软件包版本以下是多个版本的配置示例:
version: 2
workflows:
version: 2
test:
jobs:
- python_3.5
- python_3.6
- python_3.7
template: &template
docker:
- image: ubuntu:bionic
steps:
- checkout
- run:
name: Setup Miniconda
command: |
apt update
apt install -y wget
cd $HOME
wget "https://repo.anaconda.com/miniconda/Miniconda3-4.7.10-Linux-x86_64.sh" -O miniconda.sh
printf '%s' "8a324adcc9eaf1c09e22a992bb6234d91a94146840ee6b11c114ecadafc68121 miniconda.sh" | sha256sum -c
bash miniconda.sh -b -p $HOME/miniconda
- run:
name: Setup environment and run tests
command: |
export PATH="$HOME/miniconda/bin:$PATH"
conda update -y conda
conda create -n myenv python=$PYTHON_VERSION
source activate myenv
conda install -y pip numpy=$NUMPY_VERSION
python --version
pip --version
python -c "import numpy; print(numpy.__version__)"
jobs:
python_3.5:
<<: *template
environment:
PYTHON_VERSION: 3.5
NUMPY_VERSION: 1.14.2
python_3.6:
<<: *template
environment:
PYTHON_VERSION: 3.6
NUMPY_VERSION: 1.15.2
python_3.7:
<<: *template
environment:
PYTHON_VERSION: 3.7
NUMPY_VERSION: 1.16.5
版本:2
工作流程:
版本:2
测试:
工作:
-python_3.5
-python_3.6
-python_3.7
模板:&模板
码头工人:
-图片:ubuntu:bionic
步骤:
-结帐
-运行:
名称:安装程序Miniconda
命令:|
apt更新
apt安装-y wget
cd$HOME
wget“https://repo.anaconda.com/miniconda/Miniconda3-4.7.10-Linux-x86_64.sh“-O miniconda.sh
打印文件“%s”8A324ADCC9EAF1C09E22A992BB6234D91A94146840EE6B1C114ECADAFC68121 miniconda.sh“| sha256sum-c
bash miniconda.sh-b-p$HOME/miniconda
-运行:
名称:安装环境并运行测试
命令:|
导出路径=“$HOME/miniconda/bin:$PATH”
康达更新-y康达
conda create-n myenv python=$python\u版本
源代码激活myenv
conda安装-y pip numpy=$numpy\u版本
python——版本
pip——版本
python-c“导入numpy;打印(numpy.\uuuu版本)”
工作:
python_3.5:
尝试了,这就是收集包元数据(current_repodata.json)的结果:…正在工作。。。已完成解决环境:…正在工作。。。初始冻结解算失败。用灵活的解决方法重试。正在收集包元数据(repodata.json):…正在工作。。。已完成解决环境:…正在工作。。。初始冻结解算失败。用灵活的解决方法重试。PackagesNotFoundError:以下软件包无法从当前频道获得:-//repo.continuum.io/miniconda/miniconda3-4.7.10-linux-x86_64.shI遇到了一个奇怪的问题。前两个测试没有运行,因为它安装了3.7版本。即使删除了shell:/bin/bash
也可以检查这个和这个,这没关系。结果仍然如此。我也看不到echo$PYTHON\u版本的输出。怎么可能在两份工作中都是空的,而在第三份工作中又是空的呢?最后一部分是有效的。请不要忘记将最后一段添加到当前帖子中,我将标记为答案并添加非常感谢。最后一个问题是如何让缓存操作(保存缓存和恢复缓存)发挥作用?因为没有它们运行测试太长了。他
version: 2.0
workflows:
version: 2
test:
jobs:
- py3.6-np1.15
- py3.5-np1.15
- py3.6-np1.14
- py3.5-np1.14
- py3.7-np1.15
- py3.6-np1.16
- py3.7-np1.16
test-template: &test-template
docker:
- image: ubuntu:bionic
steps:
- checkout
- run:
name: Install System Dependencies
command: apt-get update && apt-get install -y libmpich12 libmpich-dev build-essential
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ .Environment.CIRCLE_JOB }}-{{ checksum "setup.py" }}
- run:
name: install anaconda
command: |
apt update
apt install -y wget
cd $HOME
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh
chmod +x ~/miniconda.sh && bash ~/miniconda.sh -b -p $HOME/miniconda
export PATH=$HOME/miniconda/bin:$PATH
- run:
name: Install numpy, cython, mdtraj
command: |
export PATH="$HOME/miniconda/bin:$PATH"
conda update --yes conda
echo $PYTHON_VERSION
conda create -n myenv python=$PYTHON_VERSION -c conda-forge
source activate myenv
conda install --yes pip
conda install --yes -c conda-forge numpy=$NUMPY_VERSION cython=$CYTHON_VERSION
conda install --yes -c conda-forge nose mdtraj
python --version
python -c "import numpy; print(numpy.__version__)"
- run:
name: Install and build package
command: |
export PATH=$HOME/miniconda/bin:$PATH
source activate myenv
pip install --progress-bar off .[dev]
python setup.py build_ext --inplace
python setup.py install
- save_cache:
paths:
- ~/miniconda
key: v1-dependencies-{{ checksum "setup.py" }}
- run:
name: Run non-MPI tests
command: |
export PATH=$HOME/miniconda/bin:$PATH
source activate myenv
nosetests -a '!mpi' package
- run:
name: Run MPI tests
command: |
export PATH=$HOME/miniconda/bin:$PATH
source activate myenv
OMP_NUM_THREADS=1 mpiexec -n 2 nosetests -a mpi package
- store_artifacts:
path: test-reports
destination: test-reports
jobs:
py3.6-np1.15:
<<: *test-template
environment:
NUMPY_VERSION: 1.14.2
CYTHON_VERSION: 0.26.1
PYTHON_VERSION: 3.6
py3.5-np1.15:
<<: *test-template
environment:
NUMPY_VERSION: 1.14.2
CYTHON_VERSION: 0.26.1
PYTHON_VERSION: 3.5
py3.6-np1.14:
<<: *test-template
environment:
NUMPY_VERSION: 1.14.2
CYTHON_VERSION: 0.26.1
PYTHON_VERSION: 3.6
py3.5-np1.14:
<<: *test-template
environment:
NUMPY_VERSION: 1.14.2
CYTHON_VERSION: 0.26.1
PYTHON_VERSION: 3.5
py3.7-np1.15:
<<: *test-template
environment:
NUMPY_VERSION: 1.15.2
CYTHON_VERSION: 0.26.1
PYTHON_VERSION: 3.7.1
py3.6-np1.16:
<<: *test-template
environment:
NUMPY_VERSION: 1.16.5
CYTHON_VERSION: 0.26.1
PYTHON_VERSION: 3.6
py3.7-np1.16:
<<: *test-template
environment:
NUMPY_VERSION: 1.16.5
CYTHON_VERSION: 0.29.2
PYTHON_VERSION: 3.7.1