Python 2.7 使用Python 2.7和Apache Beam进行Cloudbuild
我使用ApacheBeam在Python 2.7上创建了一个管道,该管道在Google数据流上运行。当我从笔记本电脑本地部署它时,这个管道工作得很好。我现在希望通过CloudBuild部署它。这是我的cloudbuild.yaml文件:Python 2.7 使用Python 2.7和Apache Beam进行Cloudbuild,python-2.7,apache-beam,google-cloud-build,Python 2.7,Apache Beam,Google Cloud Build,我使用ApacheBeam在Python 2.7上创建了一个管道,该管道在Google数据流上运行。当我从笔记本电脑本地部署它时,这个管道工作得很好。我现在希望通过CloudBuild部署它。这是我的cloudbuild.yaml文件: steps: - name: "docker.io/library/python:2.7" args: ["pip", "install", "-t", "/workspace/lib", "-r", "requirements.txt"] -
steps:
- name: "docker.io/library/python:2.7"
args: ["pip", "install", "-t", "/workspace/lib", "-r", "requirements.txt"]
- name: "docker.io/library/python:2.7"
args: ["python2", "tests.py"]
env: ["PYTHONPATH=/workspace/lib"]
当触发CloudBuild时,它成功地安装了所有需求,但当它尝试在tests.py文件中导入apache_beam时,我收到以下错误:
File "tests.py", line 3, in <module>
import apache_beam as beam
File "/workspace/lib/apache_beam/__init__.py", line 97, in <module>
from apache_beam import coders
File "/workspace/lib/apache_beam/coders/__init__.py", line 19, in <module>
from apache_beam.coders.coders import *
File "/workspace/lib/apache_beam/coders/coders.py", line 29, in <module>
import google.protobuf.wrappers_pb2
ImportError: No module named google.protobuf.wrappers_pb2
注意:requirements.txt中列出了所有必要的需求,因为我可以在本地部署管道。我只需要另一个名为
tests.py的文件就可以重现您的问题,其中有一行:
将apache_梁导入为梁
在一些社区,这似乎是一个很好的选择
首先,如果您不是使用不推荐的Python2而是使用Python3.7,则不会发生此错误。我想虽然Python2.7是必需的
我创建了一个Dockerfile,例如:
FROM python:2
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD [ "python", "./tests.py" ]
然后,更改了cloudbuild.yaml
:
steps:
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'gcr.io/$PROJECT_ID/quickstart-image', '.' ]
成功构建的:
Step 5/6 : COPY . .
---> 082407f9672d
Step 6/6 : CMD [ "python", "./tests.py" ]
---> Running in 7d53e96370f9
Removing intermediate container 7d53e96370f9
---> d60a6d473d21
Successfully built d60a6d473d21
嗨,你查过我的答案了吗?你还是有同样的问题吗?嗨,胡安基,谢谢你的回答。它确实解决了我的问题谢谢你花些时间来复习这个帖子!快乐编码!
Step 5/6 : COPY . .
---> 082407f9672d
Step 6/6 : CMD [ "python", "./tests.py" ]
---> Running in 7d53e96370f9
Removing intermediate container 7d53e96370f9
---> d60a6d473d21
Successfully built d60a6d473d21