gitlab CI中的LFTP：文件不会在FTP服务器上更新，即使在上次提交时更改了这些文件

在服务器上提交新修改时，GITLAB-CI管道出现问题。我们希望自动更新FTP服务器上的修改。

为此，我们在gitlaci.yaml文件中使用了一个简单的CI配置：

部署：
脚本：
-bash./deploy.sh
仅：
-发展
-主人

deploy.sh的内容也非常简单：

#/bin/bash
回显“部署启动”；
echo“***目标分支测试：”
target=“null”
中的案例“$CI\u COMMIT\u REF\u NAME”
“开发”）
目标=$DEV
;;
“船长”）
目标=$PROD
;;
*)
echo“cette分行附近的Pas解除部署”
出口1；
;;
以撒
回显“***$target”
echo“***安装客户端FTP”
apt-get-update-qq&&apt-get-install-y-qq-lftp
回显“***将文件复制到目标”
lftp-c“设置ftp:ssl不允许；打开-u$USERNAME，$PASSWORD$HOST；镜像-Rnev./$target--忽略时间--并行=10--排除glob.git*--排除.git/”

（变量将在CI变量中设置，并且大部分时间都在工作）

当我们尝试通过管道部署更改时，我们有

• 有时他会发送文件
• 有时不发送任何内容，零文件，甚至是上次提交时更改的文件

有什么想法吗？

作业成功，但提交的文件未通过FTP发送…
这有点奇怪。。。如果没有文件被更改，我如何拥有管道^^？？？

你知道发生了什么事吗？

谢谢你，
问候，
尼古拉斯

我上一期的附加日志。gitLab中更新的相关文件是trombi/index.php，它没有出现在日志中。。。在服务器上，它根本没有改变：

 $ bash ./deploy.sh
 *** Déploiement de xxx - start:
 *** Target branche:
 *** ./www/xxxDev
 *** voir le resultat sur: xxx/Dev
 *** Install client FTP
 debconf: delaying package configuration, since apt-utils is not installed
 Selecting previously unselected package libtcl8.6:amd64.
 (Reading database ... 23968 files and directories currently installed.)
 Preparing to unpack .../libtcl8.6_8.6.9+dfsg-2_amd64.deb ...
 Unpacking libtcl8.6:amd64 (8.6.9+dfsg-2) ...
 Selecting previously unselected package tcl8.6.
 Preparing to unpack .../tcl8.6_8.6.9+dfsg-2_amd64.deb ...
 Unpacking tcl8.6 (8.6.9+dfsg-2) ...
 Selecting previously unselected package tcl-expect:amd64.
 Preparing to unpack .../tcl-expect_5.45.4-2_amd64.deb ...
 Unpacking tcl-expect:amd64 (5.45.4-2) ...
 Selecting previously unselected package expect.
 Preparing to unpack .../expect_5.45.4-2_amd64.deb ...
 Unpacking expect (5.45.4-2) ...
 Selecting previously unselected package lftp.
 Preparing to unpack .../lftp_4.8.4-2_amd64.deb ...
 Unpacking lftp (4.8.4-2) ...
 Setting up lftp (4.8.4-2) ...
 Setting up libtcl8.6:amd64 (8.6.9+dfsg-2) ...
 Setting up tcl8.6 (8.6.9+dfsg-2) ...
 Setting up tcl-expect:amd64 (5.45.4-2) ...
 Setting up expect (5.45.4-2) ...
 Processing triggers for mime-support (3.62) ...
 Processing triggers for hicolor-icon-theme (0.17-2) ...
 Processing triggers for libc-bin (2.28-10) ...
 *** Copy file to target
 Mirroring directory `.gitlab'
 Mirroring directory `adm'
 Mirroring directory `xxx'
 Mirroring directory `communaute'
 Mirroring directory `conso'
 Mirroring directory `cron'
 Mirroring directory `css'
 Mirroring directory `doc'
 Mirroring directory `eve'
 Mirroring directory `img'
 Finished mirror `img'
 Mirroring directory `js'
 Mirroring directory `ad/comi'
 Mirroring directory `comm/img'
 Finished mirror `conso'
 Mirroring directory `materiel'
 Finished mirror `js'
 Mirroring directory `monCompte'
 Mirroring directory `.gitlab/issue_templates'
 Finished mirror `association'
 Mirroring directory `money'
 Finished mirror `css'
 Mirroring directory `xxx'
 Mirroring directory `documents/parsedown-contents'
 Mirroring directory `events/championnats'
 Finished mirror `cron'
 Mirroring directory `proto'
 Finished mirror `adm/comite'
 Mirroring directory `adm/doc-legaux'
 Finished mirror `communaute/img'
 Mirroring directory `communaute/jou'
 Finished mirror `materiel'
 Mirroring directory `public'
 Finished mirror `.gitlab/issue_templates'
 Finished mirror `.gitlab'
 Mirroring directory `te'
 Finished mirror `monCompte'
 Mirroring directory `webServices'
 Finished mirror `xxx'
 Mirroring directory `admin/emails'
 Finished mirror `documents/parsedown-contents'
 Mirroring directory `documents/parsedown-master'
 Mirroring directory `money/cotisations'
 Finished mirror `events/championnats'
 Mirroring directory `events/festivals'
 Mirroring directory `proto/auth'
 Finished mirror `adm/do'
 Mirroring directory `adn/jeux'
 Finished mirror `communaute/jou'
 Mirroring directory `communaute/mem'
 Mirroring directory `public/noel'
 Mirroring directory `te/con'
 Finished mirror `adm/emails'
 Mirroring directory `adm/ppp'
 Finished mirror `events/fest'
 Mirroring directory `events/tour'
 Finished mirror `proto/auth'
 Finished mirror `proto'
 Mirroring directory `money/devis'
 Mirroring directory `communaute/membres/Ajouter'
 Finished mirror `te/cont'
 Finished mirror `te'
 Mirroring directory `communaute/membres/messageCollectif'
 Finished mirror `adm/popp'
 Mirroring directory `adm/reglement-interieur'
 Finished mirror `events/tour'
 Finished mirror `events'
 Mirroring directory `communaute/membres/trombi'
 Finished mirror `communaute/membres/Ajouter'
 Mirroring directory `communaute/membres/urgence'
 Finished mirror `communaute/membres/messageCollectif'
 Mirroring directory `communaute/outils'
 Finished mirror `adm/reglement-interieur'
 Mirroring directory `admin/roles-engagements-avantages'
 Finished mirror `communaute/membres/trombi'
 Mirroring directory `communaute/partenaires'
 Mirroring directory `public/noel/css'
 Finished mirror `communaute/membres/urgence'
 Finished mirror `communaute/membres'
 Mirroring directory `communaute/sponsors'
 Mirroring directory `communaute/outils/association'
 Finished mirror `admin/roles-engagements-avantages'
 Mirroring directory `admin/settings'
 Finished mirror `communaute/partenaires'
 Mirroring directory `communaute/outils/classements'
 Finished mirror `public/noel/css'
 Mirroring directory `public/noel/fonts'
 Finished mirror `communaute/sponsors'
 Mirroring directory `communaute/outils/smash-rules-sets'
 Finished mirror `communaute/outils/association'
 Mirroring directory `communaute/outils/teams'
 Finished mirror `admin/settings'
 Mirroring directory `public/noel/img'
 Finished mirror `communaute/outils/classements'
 Mirroring directory `public/noel/js'
 Mirroring directory `wwws/compte'
 Mirroring directory `money/cotisations/fiches'
 Finished mirror `admin/jeux'
 Finished mirror `documents/parsedown-master'
 Finished mirror `admin'
 Finished mirror `documents'
 Mirroring directory `money/img'
 Mirroring directory `money/note2Frais'
 Finished mirror `public/noel/fonts'
 Mirroring directory `money/tresorerie'
 Finished mirror `communaute/outils/smash-rules-sets'
 Mirroring directory `wwws/database'
 Finished mirror `communaute/outils/teams'
 Finished mirror `communaute/outils'
 Finished mirror `communaute'
 Mirroring directory `wwws/devis'
 Finished mirror `public/noel/img'
 Mirroring directory `wwws/discord'
 Finished mirror `public/noel/js'
 Finished mirror `public/noel'
 Finished mirror `public'
 Mirroring directory `wwws/email'
 Finished mirror `money/cotisations/fiches'
 Finished mirror `money/cotisations'
 Mirroring directory `wwws/emoji'
 Finished mirror `wwws/compte'
 Mirroring directory `wwws/files'
 Finished mirror `money/img'
 Mirroring directory `wwws/googleMaps'
 Mirroring directory `money/note2Frais/min'
 Finished mirror `money/tresorerie'
 Mirroring directory `money/note2Frais/dons'
 Finished mirror `wwws/devis'
 Mirroring directory `wwws/jou'
 Finished mirror `wwws/discord'
 Mirroring directory `wwws/membres'
 Finished mirror `money/devis'
 Mirroring directory `wwws/permissions'
 Finished mirror `wwws/dda'
 Mirroring directory `wwws/public'
 Finished mirror `wwws/email'
 Mirroring directory `wwws/resssppp'
 Finished mirror `wwws/emoji'
 Finished mirror `wwws/googleMaps'
 Finished mirror `wwws/files'
 Finished mirror `mon/note2Frais/aport'
 Finished mirror `mon/note2Frais/dons'
 Finished mirror `mon/note2Frais'
 Finished mirror `mon'
 Finished mirror `wwws/jou'
 Finished mirror `wwws/membres'
 Finished mirror `wwws/public'
 Finished mirror `wwws/permissions'
 Finished mirror `wwws/resetPasswords'
 Finished mirror `wwws'
 *** End
 ********* ATTENTION si pas de ligne entre 'End' et 'Copy file to target' alors il faut re lancer le déploiement
Running after script
00:02
Saving cache
00:02
Uploading artifacts for successful job
00:01
 Job succeeded

我确认，管道正在为已更新文件的提交运行：

Running with gitlab-runner 12.9.0-rc1 (a350f628)
   on docker-auto-scale ed2dce3a
Preparing the "docker+machine" executor
00:32
 Using Docker executor with image ruby:2.5 ...
 Pulling docker image ruby:2.5 ...
 Using docker image sha256:5a76bd71024b1c46b1e8871b205d025b03abe85ea for ruby:2.5 ...
Preparing environment
00:05
 Running on runner-ed2dce3a-project-10794060-concurrent-0 via runner-ed2dce3a-srm-1584724660-6891f84d...
Getting source from Git repository
00:03
 $ eval "$CI_PRE_CLONE_SCRIPT"
 Fetching changes...
 Initialized empty Git repository in /builds/xxx/prod/.git/
 Created fresh repository.
 From https://gitlab.com/xxx/prod
  * [new ref]         refs/pipelines/128249282 -> refs/pipelines/128249282
  * [new branch]      develop                  -> origin/develop
  * [new branch]      master                   -> origin/master
 Checking out 0f5997df as develop...

提交0f5997df包含：

---

显示1个已更改的文件，其中包括3个添加和3个删除

我建议在

deploy.sh

的开头添加

set-euo pipefail

，这样管道就不会忽略脚本错误。你说过不管命令结果如何，状态总是成功的

看起来其他人有问题，有一个建议可以用来解决这个问题。请注意，这可能需要安装软件包

您还可以尝试增加详细级别

-v/-vv/-vvv

，看看到底出了什么问题

更新： 我的建议是安装带有apt get的

expect

包，并与lftp一起使用

unbuffer

命令，以确保将所有日志刷新到控制台。 您可以尝试更新

deploy.sh

install行并复制文件行，如下所示：

echo“***安装客户端FTP并期望”
apt-get-update-qq&&apt-get-install-y-qq-lftp-expect
echo“***将文件复制到目标并取消缓冲lftp输出”
解除缓冲lftp-c“”

关于详细性，我从

mirror

命令开始：

#您可以从一级冗长开始，并在需要时增加它
lftp-c“…镜像-v…”
lftp-c“…镜像-vv…”
lftp-c“…镜像-vvv…”

错误仍然存在 目前，我只找到了一个解决方法，使用此代码在需要时向我发出警告：

echo "*** Install client FTP"
apt-get update -qq && apt-get install -y -qq lftp expect

echo "*** Copy file to target"
unbuffer lftp -c "set ftp:ssl-allow no; open -u $USERNAME,$PASSWORD $HOST; mirror -vvv -Rnev ./ $target --ignore-time --parallel=10 --exclude-glob .git* --exclude .git/" >> log_${CI_JOB_ID}_lftp.txt

cat log_${CI_JOB_ID}_lftp.txt

count=$(grep -c "' (" log_${CI_JOB_ID}_lftp.txt)
echo $count

# lesser than 2: log file + at least one file
if [ $count -lt 2 ]
then
    echo "erreur: need to FORCE REDEPLOY"
    echo "erreur: need to FORCE REDEPLOY"
    echo "erreur: need to FORCE REDEPLOY"
    echo "erreur: need to FORCE REDEPLOY"
    exit 1;
else
    echo "End ***"
fi

我测试是否有更新的文件，如果没有更新的文件（计数<2，计数=1或0），退出1；为了强制用户重新部署。。。
但是开发人员仍然必须创建新的提交，而不仅仅是重新启动管道。

为什么-lt 2而不是1？因为日志文件也将添加到服务器上。所以我需要检查至少有2个文件发送，而不是只有1个。

---

（不要担心，以前的日志文件将被删除，因为镜像在源代码中看不到它）

如果这是一项成本较低的ftp服务，我希望它不稳定且不可靠。为了使其发挥最佳效果，您可以尝试将文件传输减少到最低限度

经过一些研究，我发现了一个似乎最适合您使用的工具——每次提交后只将更改的文件部署到ftp服务器

也许值得一试。甚至还有预装的

deploy\u生产：
图片：dotsunite/git ftp
阶段：部署
仅：
-主人
脚本：
-git ftp push-v--用户$ftp\u部署\u用户--passwd$ftp\u部署\u密码$ftp\u部署\u主机

最后，我决定将我的projet从gitlab移动到github。 CI/CD基本相同，只是需要/可能将yaml文件拆分为每个目标环境

这就是我所做的，一个文件用于dev，另一个文件用于prod。 这两个文件的内容非常相同，我向您展示了DEV文件的内容，唯一不同的是URL\u DEV secret值

on:
  push:
    branches:
      - develop
name: Publish on https://www.mydomain.fr/myprojectname
jobs:
  FTP-Deploy-Action:
    name: FTP-Deploy-Action
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2.1.0
      with:
        fetch-depth: 2
    - name: FTP-Deploy-Action
      uses: SamKirkland/FTP-Deploy-Action@3.1.1
      with:
        ftp-server: ${{ secrets.FTP_HOSTNAME }}/${{ secrets.URL_DEV }}/
        ftp-username: ${{ secrets.FTP_USERNAME }}
        ftp-password: ${{ secrets.FTP_PASSWORD }}

所有的秘密变量都和我在gitlab中的一样

通过进行此更改，我解决了与此票证相关的问题+我消除了gitlab shared runner minute的限制

---

拜拜GitLab，您好GitHub…感谢所有帮助我找到解决方案的人

不知道？是否只有我一个人在gitLab上使用FTP客户端？？：（也许我可以尝试调整我的deploy.sh脚本以获取答案，并查看是否有文件被复制…如果没有…也许我应该重试…但如果第一次它不工作，我怀疑它将第二次工作管道示例：仍然没有人…它如何工作此FTP客户端？它如何知道哪些文件是相关的？？？git ci中有没有办法查看最近更新的文件？非常感谢您回答makozaki。我为pipefail做了设置…我们会看到的。我更喜欢先使用LFTP继续…然后再搜索其他工具。我可以问您如何提高详细程度（-v/-vv/-vvv）要查看到底出了什么问题？谢谢我想使用

unbuffer

和

lftp

来确保日志被刷新并且可见。冗长可以用于许多命令。我希望

on:
  push:
    branches:
      - develop
name: Publish on https://www.mydomain.fr/myprojectname
jobs:
  FTP-Deploy-Action:
    name: FTP-Deploy-Action
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2.1.0
      with:
        fetch-depth: 2
    - name: FTP-Deploy-Action
      uses: SamKirkland/FTP-Deploy-Action@3.1.1
      with:
        ftp-server: ${{ secrets.FTP_HOSTNAME }}/${{ secrets.URL_DEV }}/
        ftp-username: ${{ secrets.FTP_USERNAME }}
        ftp-password: ${{ secrets.FTP_PASSWORD }}