使用Pytest、Selenium Grid和Docker创建测试基础架构

使用Pytest、Selenium Grid和Docker创建测试基础架构,selenium,docker,pytest,dockerfile,docker-compose,Selenium,Docker,Pytest,Dockerfile,Docker Compose,基于此,我成功地创建了可伸缩的selenium网格。 然后我想使用Pytest在网格中运行我的测试套件(用Python编写) 我是Docker的新手,试图找到将测试过程迁移到MicroService体系结构的最佳方法。基本上,我想让开发人员能够在他们的PC上本地设置完整的测试基础设施 因此,我有4个docker文件和1个docker-compose.yml 基本dockerfile: FROM ubuntu ENV SEL_VERSION 2.44.0 # Update repos for

基于此,我成功地创建了可伸缩的selenium网格。 然后我想使用Pytest在网格中运行我的测试套件(用Python编写)

我是Docker的新手,试图找到将测试过程迁移到MicroService体系结构的最佳方法。基本上,我想让开发人员能够在他们的PC上本地设置完整的测试基础设施

因此,我有4个docker文件和1个docker-compose.yml

基本dockerfile:

FROM ubuntu

ENV SEL_VERSION 2.44.0

# Update repos for Java
RUN apt-get update -qqy \
&& apt-get -qqy --no-install-recommends install \
software-properties-common \
&& rm -rf /var/lib/apt/lists/*
RUN add-apt-repository -y ppa:webupd8team/java

RUN echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections
RUN echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections

# Install Java7
RUN apt-get update -qqy \
&& apt-get -qqy --no-install-recommends install \
oracle-java7-installer \
&& rm -rf /var/lib/apt/lists/*

# Download Selenium Server
RUN wget http://selenium-release.storage.googleapis.com/${SEL_VERSION%.*}/selenium-server-standalone-${SEL_VERSION}.jar
中心dockerfile:

FROM org/grid:base

EXPOSE 4444

# Add and set permissions to the script that will launch the hub
ADD register-hub.sh /var/register-hub.sh
RUN chmod 755 /var/register-hub.sh

# start a shell and run the script
#WORKDIR /
CMD ["/bin/bash", "/var/register-hub.sh"]
FROM org/grid:base

# set the FF version to use
ENV FIREFOX_MINOR 34.0.5

# Update and install what's needed
RUN apt-get update -qqy \
&& apt-get -qqy --no-install-recommends install \
firefox \
xvfb \
bzip2 \
&& rm -rf /var/lib/apt/lists/*

# setup FF
RUN [ -e /usr/bin/firefox ] && rm /usr/bin/firefox
ADD https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/${FIREFOX_MINOR}/linux-x86_64/en-US/firefox-${FIREFOX_MINOR}.tar.bz2 /tmp/
RUN apt-get install -q -y libdbus-glib-1-2
RUN tar -xvjf /tmp/firefox-${FIREFOX_MINOR}.tar.bz2 -C /opt/
RUN chmod -R +x /opt/firefox/
RUN ln -s /opt/firefox/firefox /usr/bin/firefox

# add and set permissions for the bash script to register the node
ADD register-node.sh /var/register-node.sh
RUN chmod 755 /var/register-node.sh

# start a shell and run the script
CMD ["/bin/bash", "/var/register-node.sh"]
# Starting from base image
FROM ubuntu

# Set the Github personal token [to clone the QA code]
#todo------secret------
ENV GH_TOKEN some_token_some_token_some_token_

# Install Python & pip
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get install -y python python-pip python-dev && pip install --upgrade pip

# Install GIT
RUN apt-get update -y && apt-get install git -y

# [in the / folder] Create the folder and add the whole project from the repo to the container
RUN git clone https://$GH_TOKEN:x-oauth-basic@github.com/org/org_QA.git /org_QA_folder

# Install dependencies via pip
WORKDIR /org_QA_folder
RUN pip install -r dependencies.txt

#
CMD /bin/bash
节点dockerfile:

FROM org/grid:base

EXPOSE 4444

# Add and set permissions to the script that will launch the hub
ADD register-hub.sh /var/register-hub.sh
RUN chmod 755 /var/register-hub.sh

# start a shell and run the script
#WORKDIR /
CMD ["/bin/bash", "/var/register-hub.sh"]
FROM org/grid:base

# set the FF version to use
ENV FIREFOX_MINOR 34.0.5

# Update and install what's needed
RUN apt-get update -qqy \
&& apt-get -qqy --no-install-recommends install \
firefox \
xvfb \
bzip2 \
&& rm -rf /var/lib/apt/lists/*

# setup FF
RUN [ -e /usr/bin/firefox ] && rm /usr/bin/firefox
ADD https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/${FIREFOX_MINOR}/linux-x86_64/en-US/firefox-${FIREFOX_MINOR}.tar.bz2 /tmp/
RUN apt-get install -q -y libdbus-glib-1-2
RUN tar -xvjf /tmp/firefox-${FIREFOX_MINOR}.tar.bz2 -C /opt/
RUN chmod -R +x /opt/firefox/
RUN ln -s /opt/firefox/firefox /usr/bin/firefox

# add and set permissions for the bash script to register the node
ADD register-node.sh /var/register-node.sh
RUN chmod 755 /var/register-node.sh

# start a shell and run the script
CMD ["/bin/bash", "/var/register-node.sh"]
# Starting from base image
FROM ubuntu

# Set the Github personal token [to clone the QA code]
#todo------secret------
ENV GH_TOKEN some_token_some_token_some_token_

# Install Python & pip
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get install -y python python-pip python-dev && pip install --upgrade pip

# Install GIT
RUN apt-get update -y && apt-get install git -y

# [in the / folder] Create the folder and add the whole project from the repo to the container
RUN git clone https://$GH_TOKEN:x-oauth-basic@github.com/org/org_QA.git /org_QA_folder

# Install dependencies via pip
WORKDIR /org_QA_folder
RUN pip install -r dependencies.txt

#
CMD /bin/bash
和PYTEST dockerfile:

FROM org/grid:base

EXPOSE 4444

# Add and set permissions to the script that will launch the hub
ADD register-hub.sh /var/register-hub.sh
RUN chmod 755 /var/register-hub.sh

# start a shell and run the script
#WORKDIR /
CMD ["/bin/bash", "/var/register-hub.sh"]
FROM org/grid:base

# set the FF version to use
ENV FIREFOX_MINOR 34.0.5

# Update and install what's needed
RUN apt-get update -qqy \
&& apt-get -qqy --no-install-recommends install \
firefox \
xvfb \
bzip2 \
&& rm -rf /var/lib/apt/lists/*

# setup FF
RUN [ -e /usr/bin/firefox ] && rm /usr/bin/firefox
ADD https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/${FIREFOX_MINOR}/linux-x86_64/en-US/firefox-${FIREFOX_MINOR}.tar.bz2 /tmp/
RUN apt-get install -q -y libdbus-glib-1-2
RUN tar -xvjf /tmp/firefox-${FIREFOX_MINOR}.tar.bz2 -C /opt/
RUN chmod -R +x /opt/firefox/
RUN ln -s /opt/firefox/firefox /usr/bin/firefox

# add and set permissions for the bash script to register the node
ADD register-node.sh /var/register-node.sh
RUN chmod 755 /var/register-node.sh

# start a shell and run the script
CMD ["/bin/bash", "/var/register-node.sh"]
# Starting from base image
FROM ubuntu

# Set the Github personal token [to clone the QA code]
#todo------secret------
ENV GH_TOKEN some_token_some_token_some_token_

# Install Python & pip
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get install -y python python-pip python-dev && pip install --upgrade pip

# Install GIT
RUN apt-get update -y && apt-get install git -y

# [in the / folder] Create the folder and add the whole project from the repo to the container
RUN git clone https://$GH_TOKEN:x-oauth-basic@github.com/org/org_QA.git /org_QA_folder

# Install dependencies via pip
WORKDIR /org_QA_folder
RUN pip install -r dependencies.txt

#
CMD /bin/bash
和docker compose文件:

hub:
  image: org/grid:hub # image must already exist
  ports:
    - "4444:4444" # HOST:CONTAINER
  external_links: # link to a container created outside of this YAML file
    - pytest
  volumes_from:
    - pytest # must be created first

firefox:
  image: org/grid:nodeff # image must already exist
  links:
    - hub
  expose:
    - "5555" # grid console open to public
所以…我不明白我应该如何运行“py.test/org_QA_folder/testcase.py”这样的东西,并在网格上运行[本质上是节点]

我首先使用
docker-run-dit-v/org\u QA\u文件夹——名称为pytest-schoox/grid:py
和其他带有
docker-compose-d
的服务

我尝试了多种方法:

  • 不要使用Pytest容器,而是在集线器中克隆代码。但在本例中,HUB容器运行网格的HUB[每个容器指令一个CMD][编辑:我选择了Docker建议的微服务体系结构]

  • 我让集线器和节点容器运行它们指定的服务,并创建一个单独的容器[pytest]从那里发送测试。但是测试无法运行,因为此容器没有xvfb。[编辑:我刚刚安装了xvfb并工作]

  • 我尝试使用pytest容器作为集线器的卷,但仍然存在从已经运行服务的集线器容器运行测试的问题。[编辑:出于开发目的,我从pytest容器装载代码,并使用其控制台在其他容器上测试内容。之后,我将使用pytest容器将代码分开]

  • 我应该在其他容器中安装该节点容器拥有的所有包吗?理想情况下,我希望使用pytest容器控制台[至少开发基础设施]来运行将发送到hub并运行到多个节点的测试。[编辑:我没有混合使用hub和节点,我只是在pytest容器上安装了xvfb]

    你建议怎么做?我遗漏了什么吗

    编辑[迈向解决方案的步骤]:

    我首先启动pytest容器[
    docker run-dit-v/org/org\u QA_文件夹——名称为pytest org/grid:py
    ],然后运行
    docker compose up-d
    启动网格

    在撰写文件中,我使用
    external\u链接
    volumes\u from
    进行开发。安装完成后,我不会在任何容器中工作

    我更改了Pytest dockerfile并添加了以下内容:

    # Install xvfb [THAT WAS NEEDED!!!!!]
    RUN apt-get install -y xvfb
    

    我在pytest容器中运行了我的一个测试,作为概念证明,它运行到了我已经运行的网格(在EC2实例上),并且运行得很好。它仍然需要发送到停靠的网格。

    我想你就快到了

    我会在
    docker compose.yml
    中添加
    pytest
    服务,并从
    hub
    图像中删除
    volumes\u和
    external\u链接
    (这样看起来更像你链接的博客文章中的示例)


    pytest
    服务将运行python代码,但它应该连接到hub来运行selenium。我相信在
    org\u QA\u文件夹的某个地方有一个配置试图在本地启动selenium。它需要重新配置以使用
    hub
    服务器。

    我最终使它这样工作:

    从我的代码中,我使用以下内容:

    class Fixtures_docker(unittest.TestCase):
    
        def setUp(self, hub_ip="http://HUB_CONTAINER_IP", hub_port=4444):
            print "--Fixtures_docker in config.py trying to run on--: %s:%s"%(hub_ip, hub_port)
    
            if platform.system() == 'Linux':
                from pyvirtualdisplay import Display
    
                self.display = Display(visible=0, size=(1024, 768))
                self.display.start()
    
            if browser == "Firefox": 
                self.wd = webdriver.Remote(
                    command_executor=hub_ip + ':' + str(hub_port) + '/wd/hub',
                        desired_capabilities={
                            #FIREFOX
                            "browserName"   : "firefox",
                            # "maxInstances"  : 10,
                            "seleniumProtocol": "WebDriver",
                            "platform"      : "ANY",
                            "node"          : hub_port
                    })
    
            # for debugging purposes from local machine
            elif browser == "Firefox" and os == "Windows":
                self.wd = webdriver.Remote(
                    command_executor=hub_ip + ':' + str(hub_port) + '/wd/hub',
                        desired_capabilities={
                            # FIREFOX -> WINDOWS
                            # "firefox_binary":"C:\\firefox3\Firefox3.exe", # path to find another version of ff
                            "browserName"   : "firefox",
                            #"version"      : "3.0b3", # trying to run a specific ff version
                            "platform"      : "WINDOWS",
                            "node"          : hub_port,
                            "maxSession"    : 5
                    })
    
            self.wd.implicitly_wait(10)
    
        def tearDown(self):
            self.wd.quit()
            if platform.system() == 'Linux':
                self.display.stop()
    
    在docker-compose.yml中:

    hub:
      image: org/grid:hub # image must already exist
      ports:
        - "8080:4444" # HOST:CONTAINER
      external_links: # link to a container created outside of this YAML file
        - pytest
      volumes_from:
        - pytest # must be created first
      expose:
        - "4444" # intercontainer communication [other containers with selenium server]
    
    firefox:
      image: org/grid:nodeff # image must already exist
      links:
        - hub
      expose:
        - "5555" # intercontainer communication [node registers to hub]
    

    在设置整个管道之前,我暂时使用[external links/volumes_from]格式[在其中。我在ec2实例上已经使用网格的IP/端口对其进行了测试,效果很好。我知道我必须对这个Docker网格做同样的事情……但我感到困惑的是,来自两个容器的服务如何相互通信。我应该使用哪些IP/端口?我应该将webdriver[RemoteDriver]指向容器[172.17.0.3:port]或主机[ec2_uupublic_IP:8080]?都不使用服务名称作为主机名(例如:
    hub:4444
    ,或
    firefox:5555
    )。这些值由docker compose和docker提供。因此您从pytest切换到unittest?为什么?pytest根本不起作用?