Web scraping 在Nixos上使用systemd计时器运行脚本

我有一个小的shellscript

scrape.sh

，它可以刮取一个网站并将结果数据放入一个新目录：

website='test.com'
dir_name=“datev_data/$（日期-u+%Y-%m-%dT%H:%m:%S）”
mkdir$dirname
wget--目录前缀=“$dir_name”“$website”

（我真的不在乎数据在哪里结束，只要它每次都有一个新的目录，我就可以访问数据。因此，我现在把它放在我的主目录

/home/kaligule

中。）

手动运行这个脚本效果很好，所以现在我想在我的nixos服务器上每小时运行一次这个脚本。因此，我将以下内容放在我的配置中（灵感来源于）：

systemd.services.test\u systemd\u计时器={
serviceConfig.Type=“oneshot”；
脚本=“”
echo“现在将开始刮削。”
{pkgs.bash}/bin/bash/home/kaligule/scrap.sh
echo“完成刮削”
'';
};
systemd.timers.test_systemd_timers={
wantedBy=[“timers.target”]；
partOf=[“test_systemd_timers.service”]；
timerConfig.OnCalendar=[“*-*-***:00:00]”；
};

然后我测试它：

sudo nixos-rebuild switch # everything is fine
sudo systemctl start test_systemd_timers # run it immediatelly for testing

我得到：

Job for test_systemd_timers.service failed because the control process exited with error code.
See "systemctl status test_systemd_timers.service" and "journalctl -xe" for details.

建议的第一个命令给出了以下信息：

● test_systemd_timers.service
     Loaded: loaded (/nix/store/f8348svxpnn6qx08adrv5s7ksc2zy1sk-unit-test_systemd_timers.service/test_systemd_timers.service; linked; vendor preset: enabled)
     Active: failed (Result: exit-code) since Fri 2021-04-02 14:50:02 CEST; 2min 36s ago
TriggeredBy: ● test_systemd_timers.timer
    Process: 5686 ExecStart=/nix/store/4smyxxxlhnnmw8l6l3nnfjyvmg0wxcwh-unit-script-test_systemd_timers-start/bin/test_systemd_timers-start (code=exited, status=127)
   Main PID: 5686 (code=exited, status=127)
         IP: 0B in, 0B out
        CPU: 11ms

Apr 02 14:50:02 regulus systemd[1]: Starting test_systemd_timers.service...
Apr 02 14:50:02 regulus test_systemd_timers-start[5686]: Will start scraping now.
Apr 02 14:50:02 regulus test_systemd_timers-start[5687]: /nix/store/4smyxxxlhnnmw8l6l3nnfjyvmg0wxcwh-unit-script-test_systemd_timers-start/bin/test_systemd_timers-start: line 3: {pkgs.bash}/bin/bash: No such file or directory
Apr 02 14:50:02 regulus systemd[1]: test_systemd_timers.service: Main process exited, code=exited, status=127/n/a
Apr 02 14:50:02 regulus systemd[1]: test_systemd_timers.service: Failed with result 'exit-code'.
Apr 02 14:50:02 regulus systemd[1]: Failed to start test_systemd_timers.service.

第二个建议的命令给了我：

Apr 02 14:54:42 regulus systemd[1]: Starting test_systemd_timers.service...
░░ Subject: A start job for unit test_systemd_timers.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit test_systemd_timers.service has begun execution.
░░ 
░░ The job identifier is 34454.
Apr 02 14:54:42 regulus test_systemd_timers-start[5734]: Will start scraping now.
Apr 02 14:54:42 regulus test_systemd_timers-start[5735]: /nix/store/4smyxxxlhnnmw8l6l3nnfjyvmg0wxcwh-unit-script-test_systemd_timers-start/bin/test_systemd_timers-start: line 3: {pkgs.bash}/bin/bash: No such file or directory
Apr 02 14:54:42 regulus systemd[1]: test_systemd_timers.service: Main process exited, code=exited, status=127/n/a
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ An ExecStart= process belonging to unit test_systemd_timers.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 127.
Apr 02 14:54:42 regulus systemd[1]: test_systemd_timers.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit test_systemd_timers.service has entered the 'failed' state with result 'exit-code'.
Apr 02 14:54:42 regulus systemd[1]: Failed to start test_systemd_timers.service.
░░ Subject: A start job for unit test_systemd_timers.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit test_systemd_timers.service has finished with a failure.
░░ 
░░ The job identifier is 34454 and the job result is failed.
Apr 02 14:54:42 regulus sudo[5731]: pam_unix(sudo:session): session closed for user root

因此，因为我得到了

将立即开始抓取。

在日志中，我认为作业已启动，但无法运行脚本。我的问题是：

• 我应该将脚本放在哪里（以及具有哪些权限）
• 我应该如何添加我的配置，以便脚本按照我所描述的那样运行

---

（当然我很感谢对我的方法的每一个反馈。）

你的问题是，在你的脚本中，

{pkgs.bash}

应该是

${pkgs.bash}

；如果没有

$

，您将无法获得变量插值。

在NixOS上，systemd服务以最小的默认环境运行。您可以在中添加缺少的包

---

你说得对，我忘了。输出改变了。非常感谢。对于那些在我之后出现的人：实际的helpfull错误消息出现在第一个命令的输出中，但到目前为止，它还没有显示在我的80字符终端上。

    systemd.services.test_systemd_timers = {
      serviceConfig.Type = "oneshot";
      path = [
        pkgs.wget
        pkgs.gawk
        pkgs.jq
      ];
      script = ''
        echo "Will start scraping now."
        ${pkgs.bash}/bin/bash /home/kaligule/scrape.sh
        echo "Done scraping."
      '';
    };