Docker can';t使用另一个python3容器中带有psycopg2的postgres容器
我正在尝试使用Docker can';t使用另一个python3容器中带有psycopg2的postgres容器,docker,docker-compose,psycopg2,python-3.8,fastapi,Docker,Docker Compose,Psycopg2,Python 3.8,Fastapi,我正在尝试使用docker compose制作一个多容器docker应用程序 以下是我试图实现的目标:我有一个python3应用程序,它从API调用(fastAPI,使用gunicorn服务器)中获取一个数字列表作为输入,并将数字传递给一个函数(实际上是一个ML模型),该函数返回一个数字,然后作为该API调用的结果发送回(当然是json)。那部分工作非常好。当我引入一个postgres容器来将收到的输入存储到一个postgres表中时,问题就开始了,我还需要添加一个部分,在这个部分中,我还应该从
docker compose
制作一个多容器docker应用程序
以下是我试图实现的目标:我有一个python3应用程序,它从API调用(fastAPI
,使用gunicorn服务器)中获取一个数字列表作为输入,并将数字传递给一个函数(实际上是一个ML模型),该函数返回一个数字,然后作为该API调用的结果发送回(当然是json)。那部分工作非常好。当我引入一个postgres容器来将收到的输入存储到一个postgres表中时,问题就开始了,我还需要添加一个部分,在这个部分中,我还应该从本地pgadmin4应用程序访问此postgres数据库的数据
以下是我到目前为止所做的工作:我正在使用“docker compose.yml”文件来设置这两个容器,如下所示:
version: '3.8'
services:
postgres:
image: postgres:12.4
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres_password
- POSTGRES_DATABASE=postgres
docker_fastapi:
# use the Dockerfile in the current directory.
build: .
ports:
# 3000 is what I send API calls to
- "3000:3000"
# this is postgres's port
- "5432:5432"
environment:
# these are the environment variables that I am using inside psycop2 to make connection.
- POSTGRES_HOST=postgres
- POSTGRES_PORT=5432
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres_password
- POSTGRES_DATABASE=postgres
他
下面是我如何在psycopg2
中使用这些环境变量:
import os
from psycopg2 import connect
# making database connection using environement variables.
connection = connect(host=os.environ['POSTGRES_HOST'], port=os.environ['POSTGRES_PORT'],
user=os.environ['POSTGRES_USER'], password=os.environ['POSTGRES_PASSWORD'],
database=os.environ['POSTGRES_DATABASE']
)
以下是Dockerfile:
FROM tiangolo/uvicorn-gunicorn:python3.8-slim
# slim = debian-based. Not using alpine because it has poor python3 support.
LABEL maintainer="Sebastian Ramirez <tiangolo@gmail.com>"
RUN apt-get update
RUN apt-get install -y libpq-dev gcc
# copy and install from requirements.txt file
COPY requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r /app/requirements.txt
# remove all the dependency files to reduce the final image size
RUN apt-get autoremove -y gcc
# copying all the code files to the container's file system
COPY ./api /app/api
WORKDIR /app/api
EXPOSE 3000
ENTRYPOINT ["uvicorn"]
CMD ["api.main:app", "--host", "0.0.0.0", "--port", "3000"]
以下是我发送API调用的方式:
curl -X POST "http://0.0.0.0:3000/predict" -H "accept: application/json" -H "Content-Type: application/json" -d "{\"input_data\":[[
1.354e+01, 1.436e+01, 8.746e+01, 5.663e+02, 9.779e-02, 8.129e-02,
6.664e-02, 4.781e-02, 1.885e-01, 5.766e-02, 2.699e-01, 7.886e-01,
2.058e+00, 2.356e+01, 8.462e-03, 1.460e-02, 2.387e-02, 1.315e-02,
1.980e-02, 2.300e-03, 1.511e+01, 1.926e+01, 9.970e+01, 7.112e+02,
1.440e-01, 1.773e-01, 2.390e-01, 1.288e-01, 2.977e-01, 7.259e-02]]}"
当我使用AWS RDS的postgres实例的凭据(不使用第二个postgres容器)构建它,并直接在
psycopg2.connect()
中指定凭据(不使用环境变量),而直接使用上面显示的docker文件编写和构建docker compose时,这与预期一样有效;因此,我将接收到的数据插入postgres的代码大概是好的。当我引入第二个容器时,问题就开始了。是什么导致了这些错误以及如何修复这些错误?您必须添加网络网络和依赖于标志。试试这个:
version: '3.8'
services:
postgres:
image: postgres:12.4
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres_password
- POSTGRES_DB=postgres
networks:
- default
docker_fastapi:
# use the Dockerfile in the current directory.
build: .
ports:
# 3000 is what I send API calls to
- "3000:3000"
# this is postgres's port
# no need for this
# - "5432:5432"
networks:
- default
depends_on:
- postgres
# no need for this
# environment:
# these are the environment variables that I am using inside psycop2 to make connection.
# - POSTGRES_HOST=postgres
# - POSTGRES_PORT=5432
# - POSTGRES_USER=postgres
# - POSTGRES_PASSWORD=postgres_password
# - POSTGRES_DATABASE=postgres
这个问题源于“docker compose.yml”文件中的大量
postgres
在我的帮助下,我意识到;如果有人需要的话,这就是有效的方法
version: '3.8'
services:
postgres_instance:
image: postgres:12.4
# to expose postgres to local machine and monitor it in something like pgadmin
ports:
- "5432:5432"
restart: unless-stopped
# to persist data, if containers are stoopped and resumed.
volumes:
- ./postgres-data:/var/lib/postgresql/data
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres_password
- POSTGRES_DB=postgres
networks:
- default
docker_fastapi:
# using Dockerfile in current directory
build: .
# port I send API calls to
ports:
- "3000:3000"
restart: always
depends_on:
- postgres_instance
networks:
- default
# these environment variables must be specified here, to be able to use from .py file inside this container.
environment:
- POSTGRES_HOST=postgres_instance
- POSTGRES_PORT=5432
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres_password
- POSTGRES_DB=postgres
可以看出,将服务名称从
postgres
更改为postgres\u instance
修复了所有问题。大概是因为,postgres
实际上指的是数据库的托管地,但标记与用户名和数据库名相同。这是错误:root@naveen-hp:/home/naveen/Videos/ML模型使用fastapi和Docker#Docker组合错误:服务“postgres”依赖于未定义的服务“db”
出现了另一个问题,因为您建议我删除环境变量POSTGRES\u HOST
和POSTGRES\u PORT
。那么,我如何在psycopg2.connect()
中指定它们呢?最好是在代码中处理它们,或者我以前就是这样做的,但保留它们没有问题。看一看是,但您必须在.env中定义这些变量,看一看是,但使用python dotenv从.env文件加载变量,而不是在docker文件或配置文件中显式写入它们。env文件使得在不同环境之间加载环境变量更加容易和灵活。在.env文件中定义这些变量之后,您可以使用getenv或其他函数来读取它。你可以看看这里,类似的问题。
version: '3.8'
services:
postgres_instance:
image: postgres:12.4
# to expose postgres to local machine and monitor it in something like pgadmin
ports:
- "5432:5432"
restart: unless-stopped
# to persist data, if containers are stoopped and resumed.
volumes:
- ./postgres-data:/var/lib/postgresql/data
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres_password
- POSTGRES_DB=postgres
networks:
- default
docker_fastapi:
# using Dockerfile in current directory
build: .
# port I send API calls to
ports:
- "3000:3000"
restart: always
depends_on:
- postgres_instance
networks:
- default
# these environment variables must be specified here, to be able to use from .py file inside this container.
environment:
- POSTGRES_HOST=postgres_instance
- POSTGRES_PORT=5432
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres_password
- POSTGRES_DB=postgres