Ruby on rails Rsync显示文件已同步,但在AWS S3中找不到
我使用的是Capistrano3和RubyonRails3.2.14,以及Ruby2.1.0。我编写了一个配方来检查S3装载状态,然后运行一组特定的操作:Ruby on rails Rsync显示文件已同步,但在AWS S3中找不到,ruby-on-rails,ruby,amazon-s3,capistrano3,Ruby On Rails,Ruby,Amazon S3,Capistrano3,我使用的是Capistrano3和RubyonRails3.2.14,以及Ruby2.1.0。我编写了一个配方来检查S3装载状态,然后运行一组特定的操作: 如果bucket已卸载,且文件夹(public/system)为空,则使用s3fs 如果bucket已卸载且文件夹不为空,则使用rsync将文件从public/system移动到s3 bucket,然后清空文件夹public/system,然后重新安装 rsync显示文件以增量方式发送,但我在bucket中找不到它们。这是我得到的输出:
(public/system)
为空,则使用s3fs
public/system
移动到s3 bucket,然后清空文件夹public/system
,然后重新安装 05 rsync -av /home/deploy/unmounted /home/deploy/mounted
05 sending incremental file list
05 unmounted/
05 unmounted/ckeditor/
05 unmounted/ckeditor/pictures/
05 unmounted/ckeditor/pictures/16736/
05 unmounted/ckeditor/pictures/16736/bike.jpg
05 unmounted/ckeditor/pictures/16736/content_bike.jpg
05 unmounted/ckeditor/pictures/16736/thumb_bike.jpg
05
05 sent 750,943 bytes received 93 bytes 166,896.89 bytes/sec
05 total size is 750,396 speedup is 1.00
配方对于安装s3,我还没有对其进行优化,因为首先我只想检查它是否工作
desc "Mounting S3"
task :mounting_s3 do
on main_server, in: :sequence, wait: 5 do |host|
as 'deploy' do
within "#{code_folder_path}/current" do
code_folder = "#{code_folder_path}/current"
path = "#{code_folder_path}/current/public/system"
unmount_path = "#{code_folder_path}/shared/public/system"
# check first if public/system exists
puts "** Checking if public/system folder exists on #{host}"
puts "** testing if it is there #{test("[ -d #{path} ]")}"
if test("[ -d #{path} ]")
puts "** public/system folder exists.."
puts "** Checking if S3 is mounted"
result = capture "cd #{code_folder} && mountpoint public/system" , raise_on_non_zero_exit: false
puts "result #{result}"
if result.match('is a mountpoint') != nil
puts "** S3 is mounted at #{path} on #{host}"
elsif result.match('is not a mountpoint') != nil
puts "** #{path} is not a mountpoint on #{host}"
result = capture "ls -A #{unmount_path}"
if result.length > 0
puts "** public/system is not empty"
if test("[ -d /home/deploy/mounted ]")
puts "** mounted folder exists"
puts "checking if folder is mounted"
result = capture "mountpoint /home/deploy/mounted" , raise_on_non_zero_exit: false
puts "result #{result}"
if result.match('is a mountpoint') != nil
puts "** mounted folder is mounted so unmounting it"
execute! "fusermount -u /home/deploy/mounted"
puts "** mounted folder is unmounted now"
puts "** Deleting mounted folder"
execute! "cd /home/deploy/ && sudo rm -r mounted"
puts "** mounted folder deleted"
elsif result.match('is not a mountpoint') != nil
puts "** Deleting mounted folder"
execute! "sudo rm -r /home/deploy/mounted"
puts "** mounted folder deleted"
end
end
if test("[ -d /home/deploy/unmounted ]")
puts "** deleting unmounted folder"
execute! "cd /home/deploy/ && sudo rm -r unmounted"
puts "** unmounted folder deleted"
end
puts "** mounting bucket"
execute! "cd /home/deploy/ && mkdir unmounted && mkdir mounted"
execute! "s3fs s3_bucket /home/deploy/mounted"
puts "** bucket mounted on /home/deploy/mounted folder"
puts "** copying files from public/system to unmounted"
execute! "cp -r #{unmount_path}/* /home/deploy/unmounted/."
puts "** files copied"
puts "** syncing unmounted files to s3"
execute! "rsync -av /home/deploy/unmounted /home/deploy/mounted"
puts "** Files synced"
puts "** unmounting mounted folder now for safety"
execute! "fusermount -u /home/deploy/mounted"
puts "** folder unmounted"
puts "** deleting unmounted folder now"
execute! "sudo rm -r /home/deploy/unmounted"
puts "** folders deleted\n"
puts "** emptying public/system"
execute! "sudo rm -r #{unmount_path}/../system/*"
puts "** folder is empty, mounting public/system"
execute! "s3fs s3-bucket #{path}"
puts "** s3 is mounted"
else
puts "** public/system is empty"
execute! "cd #{code_folder} && s3fs s3-bucket #{path}"
puts "** s3 is mounted"
end
end
else
puts "** public/system folder does not exists.."
puts "** Mounting S3"
execute! "cd #{code_folder} && s3fs s3-bucket #{path}"
end
end
end
运行cap staging deploy:mounting_s3--trace之后得到的输出:
** Invoke www_data (first_time)
** Execute www_data
** Invoke load:defaults (first_time)
** Execute load:defaults
** Invoke rvm:hook (first_time)
** Execute rvm:hook
** Invoke rvm:check (first_time)
** Execute rvm:check
rvm 1.27.0 (latest) by Wayne E. Seguin <wayneeseguin@gmail.com>, Michal Papis <mpapis@gmail.com> [https://rvm.io/]
ruby-2.1.0
ruby 2.1.0p0 (2013-12-25 revision 44422) [x86_64-linux]
** Invoke bundler:map_bins (first_time)
** Execute bundler:map_bins
** Invoke deploy:set_rails_env (first_time)
** Execute deploy:set_rails_env
** Invoke deploy:set_linked_dirs (first_time)
** Execute deploy:set_linked_dirs
** Invoke deploy:set_rails_env
** Invoke deploy:mounting_s3 (first_time)
** Execute deploy:mounting_s3
** Checking if public/system folder exists on webby
** testing if it is there true
** public/system folder exists..
** Checking if S3 is mounted
result public/system is not a mountpoint
** /home/deploy/apps/web_cc_cap1/current/public/system is not a mountpoint on webby
** public/system is not empty
** mounted folder exists
checking if folder is mounted
result /home/deploy/mounted is not a mountpoint
** Deleting mounted folder
00:00 deploy:mounting_s3
01 sudo rm -r /home/deploy/mounted
✔ 01 webby 0.649s
** mounted folder deleted
** mounting bucket
02 cd /home/deploy/ && mkdir unmounted && mkdir mounted
✔ 02 webby 0.631s
03 s3fs s3-bucket /home/deploy/mounted
✔ 03 webby 0.619s
** bucket mounted on /home/deploy/mounted folder
** copying files from public/system to unmounted
04 cp -r /home/deploy/apps/web_cc_cap1/shared/public/system/* /home/deploy/unmounted/.
✔ 04 webby 0.651s
** files copied
** syncing unmounted files to s3
05 rsync -av /home/deploy/unmounted /home/deploy/mounted
05 sending incremental file list
05 unmounted/
05 unmounted/ckeditor/
05 unmounted/ckeditor/pictures/
05 unmounted/ckeditor/pictures/16736/
05 unmounted/ckeditor/pictures/16736/bike.jpg
05 unmounted/ckeditor/pictures/16736/content_bike.jpg
05 unmounted/ckeditor/pictures/16736/thumb_bike.jpg
05
05 sent 750,943 bytes received 93 bytes 166,896.89 bytes/sec
05 total size is 750,396 speedup is 1.00
✔ 05 webby 4.299s
** Files synced
** unmounting mounted folder now for safety
06 fusermount -u /home/deploy/mounted
✔ 06 webby 0.631s
** folder unmounted
** deleting unmounted folder now
07 sudo rm -r /home/deploy/unmounted
✔ 07 webby 0.621s
** folders deleted
** emptying public/system
08 sudo rm -r /home/deploy/apps/web_cc_cap1/shared/public/system/../system/*
✔ 08 webby 0.649s
** folder is empty, mounting public/system
09 s3fs s3-bucket /home/deploy/apps/web_cc_cap1/current/public/system
✔ 09 webby 0.623s
** s3 is mounted
**调用www\u数据(第一次)
**执行www_数据
**调用加载:默认值(第一次)
**执行加载:默认值
**调用rvm:hook(第一次)
**执行rvm:hook
**调用rvm:检查(第一次)
**执行rvm:检查
rvm 1.27.0(最新版本)作者:韦恩E.塞根,迈克尔·帕皮斯[https://rvm.io/]
ruby-2.1.0
ruby 2.1.0p0(2013-12-25修订版44422)[x86_64-linux]
**调用绑定器:映射容器(第一次)
**执行bundler:map\u bin
**调用部署:设置轨道环境(第一次)
**执行deploy:set_rails_env
**调用部署:设置链接目录(第一次)
**执行部署:设置链接目录
**调用部署:设置\u rails\u env
**调用部署:装载\u s3(第一次)
**执行deploy:mounting_s3
**检查webby上是否存在公用/系统文件夹
**测试它是否为真
**公用/系统文件夹已存在。。
**检查是否安装了S3
结果公共/系统不是装入点
**/home/deploy/apps/web\u cc\u cap1/current/public/system不是webby上的挂载点
**公共/系统不为空
**已装入文件夹
检查文件夹是否已装入
结果/主/部署/装载不是装载点
**删除装入的文件夹
00:00部署:安装\u s3
01 sudo rm-r/home/deploy/mounted
✔ 01韦比0.649s
**已删除装入的文件夹
**安装斗
02 cd/home/deploy/&mkdir已卸载和mkdir已安装
✔ 02韦比0.631s
03 s3fs s3铲斗/主/展开/安装
✔ 03韦比0.619s
**铲斗安装在/home/deploy/mounted文件夹上
**将文件从公共/系统复制到卸载
04 cp-r/home/deploy/apps/web\u cc\u cap1/shared/public/system/*/home/deploy/unmounted/。
✔ 04韦比0.651s
**复制的文件
**将卸载的文件同步到s3
05 rsync-av/home/deploy/unmounted/home/deploy/mounted
05发送增量文件列表
05未安装/
05未安装的/未安装的编辑器/
05未安装的/ckeditor/图片/
05未安装的/ckeditor/pictures/16736/
05未安装的/ckeditor/pictures/16736/bike.jpg
05未安装的/ckeditor/pictures/16736/content_bike.jpg
05未安装的/ckeditor/pictures/16736/thumb_bike.jpg
05
05发送750943字节接收93字节166896.89字节/秒
05总尺寸为750396加速比为1.00
✔ 05韦比4.299s
**同步的文件
**为安全起见,现在卸载已装入的文件夹
06 fusermount-u/主/部署/安装
✔ 06韦比0.631s
**文件夹卸载
**正在删除未安装的文件夹
07 sudo rm-r/home/deploy/unmounted
✔ 07韦比0.621s
**已删除文件夹
**清空公共/系统
08 sudo rm-r/home/deploy/apps/web\u cc\u cap1/shared/public/system//系统/*
✔ 08韦比0.649s
**文件夹为空,正在装入公用/系统
09 s3fs s3 bucket/home/deploy/apps/web\u cc\u cap1/current/public/system
✔ 09韦比0.623s
**s3已安装
为什么不将文件发送到s3存储桶?我实际上犯了一个错误,我的目的是复制未安装的文件夹的内容,而不是复制文件夹本身。因此,我修复了rsync代码:rsync-av/home/deploy/unmounted/*/home/deploy/mounted
,它按预期工作。我实际上犯了一个错误,我的目的是复制unmounted文件夹的内容,而不是文件夹本身。因此,我修复了rsync代码:rsync-av/home/deploy/unmounted/*/home/deploy/mounted
并按预期工作。尝试rsync-avvv/home/deploy/unmounted/home/deploy/mounted
。您将得到一个非常详细的日志。也许它会告诉你一些事情。谢谢@folse,我实际上犯了一个错误,我的目的是复制未安装文件夹的内容,而不是文件夹本身。所以我修复了rsync代码:rsync-av/home/deploy/unmounted/*/home/deploy/mounted
,它按预期工作rsync-av
基本上满足了您的需求。请尝试rsync-avvv/home/deploy/unmounted/home/deploy/mounted
。您将得到一个非常详细的日志。也许它会告诉你一些事情。谢谢@folse,我实际上犯了一个错误,我的目的是复制未安装文件夹的内容,而不是文件夹本身。所以我修复了rsync代码:rsync-av/home/deploy/unmounted/*/home/deploy/mounted
,它按预期工作rsync-av
基本上满足了您的需求。