Python subprocess.popen和rdiff备份
我想用python创建一个rdiff备份包装程序，用于将windows机器备份到linux服务器 我想在包装程序中处理rdiff备份的输出。但是，当使用子流程模块执行rdiff备份并将stdout和stderr管道化到包装器时，stdout总是以管道的末端结束 在不使用管道的情况下使用子流程时，标准输出和标准输出将在控制台中以正确的顺序打印 我还注意到，在本地使用rdiff备份而不使用ssh管道时，stdout和stderr的顺序是正确的。Rdiff备份还使用subprocess.popen打开ssh会话并将数据传输到服务器。我怀疑由于某种原因，在ssh会话关闭之前，stdout-get会被阻止 这是我的代码，这是一个简化版本，真正的程序使用线程来侦听标准输出：Python subprocess.popen和rdiff备份,python,subprocess,popen,rdiff-backup,Python,Subprocess,Popen,Rdiff Backup,我想用python创建一个rdiff备份包装程序，用于将windows机器备份到linux服务器 我想在包装程序中处理rdiff备份的输出。但是，当使用子流程模块执行rdiff备份并将stdout和stderr管道化到包装器时，stdout总是以管道的末端结束 在不使用管道的情况下使用子流程时，标准输出和标准输出将在控制台中以正确的顺序打印 我还注意到，在本地使用rdiff备份而不使用ssh管道时，stdout和stderr的顺序是正确的。Rdiff备份还使用subprocess.popen打开
import sys
import subprocess
class Rdiffbackup(object):
def __init__(self):
#self.io_q = Queue()
self.exe = 'F:\\workspace\\pysubprocess\\bin\\rdiff-backup\\rdiff-backup.exe'
self.verbosity = '-v5'
self.ssh_exe = './bin/openssh/bin/ssh'
self.quiet = '-q'
self.compression = '-C'
self.port = '-p 5555'
self.key = '-i ./keys/rdiffbackup'
self.options = '-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'
self.remote_schema = self.ssh_exe + ' ' + self.quiet + ' ' + self.compression + ' ' + self.port + ' ' + self.key + ' ' + self.options + ' %s rdiff-backup --server'
def start(self,source,dest):
with subprocess.Popen([self.exe,self.verbosity,'--remote-schema',self.remote_schema,source,dest],stdout=subprocess.PIPE,stderr=subprocess.STDOUT) as self.proc:
for line in self.proc.stdout:
sys.stdout.write(line.decode("utf-8"))
if __name__ == '__main__':
rdb = Rdiffbackup()
source = "C:/Users/vdrmrt/Desktop/data"
dest = "vdrmrt@hostname::backup"
rdb.start(source,dest)
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists Off
Extended attributes Off
Windows access control lists On
Case sensitivity Off
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Unable to import win32security module. Windows ACLs
not supported by filesystem at backup/rdiff-backup-data/rdiff-backup.tmp.0
escape_dos_devices not required by filesystem at backup/rdiff-backup-data/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Ownership changing Off
Hard linking On
fsync() directories On
Directory inc permissions On
High-bit permissions On
Symlink permissions Off
Extended filenames On
Windows reserved filenames Off
Access control lists On
Extended attributes On
Windows access control lists Off
Case sensitivity On
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Backup: must_escape_dos_devices = 0
Using rdiff-backup version 1.2.8
Executing ./bin/openssh/bin/ssh -q -C -p 5555 -i ./keys/rdiffbackup -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null vdrmrt@hostname rdiff-backup --server
Hardlinks disabled by default on Windows
Unable to import module xattr.
Extended attributes not supported on filesystem at C:/Users/vdrmrt/Desktop/data
Unable to import module posix1e from pylibacl package.
POSIX ACLs not supported on filesystem at C:/Users/vdrmrt/Desktop/data
escape_dos_devices not required by filesystem at C:/Users/vdrmrt/Desktop/data
Symbolic links excluded by default on Windows
Starting increment operation C:/Users/vdrmrt/Desktop/data to backup
Using rdiff-backup version 1.2.8
Executing ./bin/openssh/bin/ssh -q -C -p 5555 -i ./keys/rdiffbackup -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null vdrmrt@hostname rdiff-backup --server
Hardlinks disabled by default on Windows
Unable to import module xattr.
Extended attributes not supported on filesystem at C:/Users/vdrmrt/Desktop/data
Unable to import module posix1e from pylibacl package.
POSIX ACLs not supported on filesystem at C:/Users/vdrmrt/Desktop/data
escape_dos_devices not required by filesystem at C:/Users/vdrmrt/Desktop/data
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists Off
Extended attributes Off
Windows access control lists On
Case sensitivity Off
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Unable to import win32security module. Windows ACLs
not supported by filesystem at backup/rdiff-backup-data/rdiff-backup.tmp.0
escape_dos_devices not required by filesystem at backup/rdiff-backup-data/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Ownership changing Off
Hard linking On
fsync() directories On
Directory inc permissions On
High-bit permissions On
Symlink permissions Off
Extended filenames On
Windows reserved filenames Off
Access control lists On
Extended attributes On
Windows access control lists Off
Case sensitivity On
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Backup: must_escape_dos_devices = 0
Symbolic links excluded by default on Windows
Starting increment operation C:/Users/vdrmrt/Desktop/data to backup
尝试对
stdoutstderrstdoutstderrRdiff备份在将stdout和stderr通过管道传输到其他程序时对其进行缓冲。 解决方案是使用一个额外的py2exe选项重建rdiff备份：“未缓冲”：True
我还必须添加一个额外的选项，以便在使用windows 7构建时能够正常工作。
下面是rdiff备份版本的final setup.py中的一个片段
if '--single-file' in sys.argv[1:]:
sys.argv.remove('--single-file')
extra_options.update({
'options': {'py2exe': {'bundle_files': 1,
'unbuffered': True,
'dll_excludes': [ "mswsock.dll", "powrprof.dll" ]}},
'zipfile': None
})
我已经试过了。通常rdiff备份首先有一些标准输出，然后是一些标准输出，然后再是标准输出。当我在线程中分别读取它们以便在程序运行时处理它们时，stdout只在最后填充。@vdrmrt然后保存stderr输出，直到stdout完成，然后打印出来。我想在rdiff运行时处理输出。