Python 运行shell命令并捕获输出
我想编写一个函数,它将执行shell命令并以字符串形式返回其输出,不管是错误消息还是成功消息。我只想得到与命令行相同的结果 什么样的代码示例可以做这样的事情 例如:Python 运行shell命令并捕获输出,python,shell,subprocess,Python,Shell,Subprocess,我想编写一个函数,它将执行shell命令并以字符串形式返回其输出,不管是错误消息还是成功消息。我只想得到与命令行相同的结果 什么样的代码示例可以做这样的事情 例如: def run_command(cmd): # ?????? print run_command('mysqladmin create test -uroot -pmysqladmin12') # Should output something like: # mysqladmin: CREATE DATABASE fai
def run_command(cmd):
# ??????
print run_command('mysqladmin create test -uroot -pmysqladmin12')
# Should output something like:
# mysqladmin: CREATE DATABASE failed; error: 'Can't create database 'test'; database exists'
>>> subprocess.check_output('cat books/* | wc', shell=True, text=True)
' 1299377 17005208 101299376\n'
诸如此类:
def runProcess(exe):
p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while(True):
# returns None while subprocess is running
retcode = p.poll()
line = p.stdout.readline()
yield line
if retcode is not None:
break
注意,我正在将stderr重定向到stdout,这可能不是您想要的,但我也需要错误消息
这个函数会逐行产生(通常您必须等待子进程完成才能获得整体输出)
就您的情况而言,用法如下:
for line in runProcess('mysqladmin create test -uroot -pmysqladmin12'.split()):
print line,
诸如此类:
def runProcess(exe):
p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while(True):
# returns None while subprocess is running
retcode = p.poll()
line = p.stdout.readline()
yield line
if retcode is not None:
break
注意,我正在将stderr重定向到stdout,这可能不是您想要的,但我也需要错误消息
这个函数会逐行产生(通常您必须等待子进程完成才能获得整体输出)
就您的情况而言,用法如下:
for line in runProcess('mysqladmin create test -uroot -pmysqladmin12'.split()):
print line,
在所有官方维护的Python版本中,最简单的方法是使用函数:
>>> subprocess.check_output(['ls', '-l'])
b'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
check\u output
运行一个只接受参数作为输入的程序。1它返回打印到stdout
的结果。如果需要将输入写入stdin
,请跳到run
或Popen
部分。如果要执行复杂的shell命令,请参阅本答案末尾有关shell=True
的说明
check\u输出
函数在所有官方维护的Python版本中都有效。但对于较新的版本,可以使用更灵活的方法
Python的现代版本(3.5或更高版本):run
如果您使用的是Python 3.5+,并且不需要向后兼容,则官方文档建议对大多数任务使用。它为子流程
模块提供了一个非常通用的高级API。要捕获程序的输出,请将subprocess.PIPE
标志传递给stdout
关键字参数。然后访问返回对象的stdout
属性:
返回值是一个bytes
对象,因此如果需要正确的字符串,则需要对其进行解码。假设被调用进程返回UTF-8编码的字符串:
>>> result.stdout.decode('utf-8')
'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
如果需要,可将其压缩为一个衬里:
>>> subprocess.run(['ls', '-l'], stdout=subprocess.PIPE).stdout.decode('utf-8')
'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
如果要将输入传递给进程的stdin
,可以将bytes
对象传递给input
关键字参数:
>>> cmd = ['awk', 'length($0) > 5']
>>> ip = 'foo\nfoofoo\n'.encode('utf-8')
>>> result = subprocess.run(cmd, stdout=subprocess.PIPE, input=ip)
>>> result.stdout.decode('utf-8')
'foofoo\n'
您可以通过传递stderr=subprocess.PIPE
(捕获到result.stderr
)或stderr=subprocess.STDOUT
(捕获到result.STDOUT
)以及常规输出来捕获错误。如果希望run
在进程返回非零退出代码时引发异常,可以传递check=True
。(或者您可以检查上面的result
的returncode
属性。)如果不考虑安全性,您还可以通过传递shell=True
来运行更复杂的shell命令,如本答案末尾所述
Python的更高版本进一步简化了上述内容。在Python 3.7+中,上述一行代码的拼写如下:
>>> subprocess.run(['ls', '-l'], capture_output=True, text=True).stdout
'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
使用run
这种方式与旧的做事方式相比只增加了一点复杂性。但是现在,您几乎可以单独使用run
函数执行任何需要执行的操作
Python的旧版本(3-3.4):更多关于check\u输出的信息
如果您使用的是较旧版本的Python,或者需要适度的向后兼容性,则可以使用上文简要介绍的check_output
函数。它从Python 2.7开始就可用
subprocess.check_output(*popenargs, **kwargs)
它采用与Popen
(见下文)相同的参数,并返回包含程序输出的字符串。这个答案的开头有一个更详细的用法示例。在Python 3.5+中,check\u output
相当于使用check=True
和stdout=PIPE
执行run
,并仅返回stdout
属性
您可以传递stderr=subprocess.STDOUT
,以确保返回的输出中包含错误消息。如果不考虑安全性,还可以通过传递shell=True
来运行更复杂的shell命令,如本答案末尾所述
如果需要从stderr
进行管道传输或将输入传递给流程,check\u output
将无法完成任务。在这种情况下,请参见下面的Popen
示例
复杂的应用程序和Python的旧版本(2.6及以下版本):Popen
如果您需要深层向后兼容性,或者如果您需要比check\u output
或run
提供更复杂的功能,那么您必须直接使用Popen
对象,这些对象封装了子流程的低级API
Popen
构造函数接受一个不带参数的命令,或一个列表,该列表包含一个命令作为其第一项,后跟任意数量的参数,每个参数作为列表中的一个单独项。可以帮助将字符串解析为适当格式的列表Popen
对象也接受进程IO管理和低级配置
要发送输入和捕获输出,通信
几乎总是首选方法。例如:
output = subprocess.Popen(["mycmd", "myarg"],
stdout=subprocess.PIPE).communicate()[0]
或
如果设置了stdin=PIPE
,communicate
还允许您通过stdin
将数据传递给流程:
>>> cmd = ['awk', 'length($0) > 5']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
... stderr=subprocess.PIPE,
... stdin=subprocess.PIPE)
>>> out, err = p.communicate('foo\nfoofoo\n')
>>> print out
foofoo
注意,这表明在某些系统上,您可能需要将stdout
、stderr
和stdin
全部设置为PIPE
(或DEVNULL
),才能使通信工作
在某种程度上
run(cmd, [stdout=etc...], input=other_output)
Popen(cmd, [stdout=etc...]).communicate(other_output)
import commands
print commands.getstatusoutput('wc -l file')
from subprocess import Popen, PIPE
output = Popen(["date"],stdout=PIPE)
response = output.communicate()
print response
def run_command(command):
p = subprocess.Popen(command,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
return iter(p.stdout.readline, b'')
command = 'mysqladmin create test -uroot -pmysqladmin12'.split()
for line in run_command(command):
print(line)
import subprocess
def run_command(cmd):
"""given shell command, returns communication tuple of stdout and stderr"""
return subprocess.Popen(cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
stdin=subprocess.PIPE).communicate()
run_command('tracert 11.1.0.1')[0]
def run_command(cmd):
"""given shell command, returns communication tuple of stdout and stderr"""
# instantiate a startupinfo obj:
startupinfo = subprocess.STARTUPINFO()
# set the use show window flag, might make conditional on being in Windows:
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
# pass as the startupinfo keyword argument:
return subprocess.Popen(cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
stdin=subprocess.PIPE,
startupinfo=startupinfo).communicate()
run_command('tracert 11.1.0.1')
import os
import subprocess
# Define a function for running commands and capturing stdout line by line
# (Modified from Vartec's solution because it wasn't printing all lines)
def runProcess(exe):
p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
return iter(p.stdout.readline, b'')
# Get all filenames in working directory
for filename in os.listdir('./'):
# This command will be run on each file
cmd = 'nm ' + filename
# Run the command and capture the output line by line.
for line in runProcess(cmd.split()):
# Eliminate leading and trailing whitespace
line.strip()
# Split the output
output = line.split()
# Filter the output and print relevant lines
if len(output) > 2:
if ((output[2] == 'set_program_name')):
print filename
print line
import os
os.system('sample_cmd > tmp')
print open('tmp', 'r').read()
os.remove('tmp')
import subprocess
from time import sleep
def run_command(command):
p = subprocess.Popen(command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
shell=True)
# Read stdout from subprocess until the buffer is empty !
for line in iter(p.stdout.readline, b''):
if line: # Don't print blank lines
yield line
# This ensures the process has completed, AND sets the 'returncode' attr
while p.poll() is None:
sleep(.1) #Don't waste CPU-cycles
# Empty STDERR buffer
err = p.stderr.read()
if p.returncode != 0:
# The run_command() function is responsible for logging STDERR
print("Error: " + str(err))
for line in run_command(cmd):
print(line)
import subprocess
output = subprocess.getoutput("ls -l")
print(output)
import os
os.popen('your command here').read()
import subprocess
p = subprocess.Popen("Your command", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()[0]
print p.split("\n")
def execute(cmd, output=True, DEBUG_MODE=False):
"""Executes a bash command.
(cmd, output=True)
output: whether print shell output to screen, only affects screen display, does not affect returned values
return: ...regardless of output=True/False...
returns shell output as a list with each elment is a line of string (whitespace stripped both sides) from output
could be
[], ie, len()=0 --> no output;
[''] --> output empty line;
None --> error occured, see below
if error ocurs, returns None (ie, is None), print out the error message to screen
"""
if not DEBUG_MODE:
print "Command: " + cmd
# https://stackoverflow.com/a/40139101/2292993
def _execute_cmd(cmd):
if os.name == 'nt' or platform.system() == 'Windows':
# set stdin, out, err all to PIPE to get results (other than None) after run the Popen() instance
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
else:
# Use bash; the default is sh
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, executable="/bin/bash")
# the Popen() instance starts running once instantiated (??)
# additionally, communicate(), or poll() and wait process to terminate
# communicate() accepts optional input as stdin to the pipe (requires setting stdin=subprocess.PIPE above), return out, err as tuple
# if communicate(), the results are buffered in memory
# Read stdout from subprocess until the buffer is empty !
# if error occurs, the stdout is '', which means the below loop is essentially skipped
# A prefix of 'b' or 'B' is ignored in Python 2;
# it indicates that the literal should become a bytes literal in Python 3
# (e.g. when code is automatically converted with 2to3).
# return iter(p.stdout.readline, b'')
for line in iter(p.stdout.readline, b''):
# # Windows has \r\n, Unix has \n, Old mac has \r
# if line not in ['','\n','\r','\r\n']: # Don't print blank lines
yield line
while p.poll() is None:
sleep(.1) #Don't waste CPU-cycles
# Empty STDERR buffer
err = p.stderr.read()
if p.returncode != 0:
# responsible for logging STDERR
print("Error: " + str(err))
yield None
out = []
for line in _execute_cmd(cmd):
# error did not occur earlier
if line is not None:
# trailing comma to avoid a newline (by print itself) being printed
if output: print line,
out.append(line.strip())
else:
# error occured earlier
out = None
return out
else:
print "Simulation! The command is " + cmd
print ""
from subprocess import check_output
from shlex import split
res = check_output(split('git log -n 5 --since "5 years ago" --until "2 year ago"'))
print(res)
>>> b'commit 7696ab087a163e084d6870bb4e5e4d4198bdc61a\nAuthor: Artur Barseghyan...'
def sh(cmd, input=""):
rst = subprocess.run(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, input=input.encode("utf-8"))
assert rst.returncode == 0, rst.stderr.decode("utf-8")
return rst.stdout.decode("utf-8")
sh("ls -a")
import subprocess
def command_caller(command=None)
sp = subprocess.Popen(command, stderr=subprocess.PIPE, stdout=subprocess.PIPE, shell=False)
out, err = sp.communicate()
if sp.returncode:
print(
"Return code: %(ret_code)s Error message: %(err_msg)s"
% {"ret_code": sp.returncode, "err_msg": err}
)
return sp.returncode, out, err
import subprocess
import os
import tempfile
def execute_to_file(command):
"""
This function execute the command
and pass its output to a tempfile then read it back
It is usefull for process that deploy child process
"""
temp_file = tempfile.NamedTemporaryFile(delete=False)
temp_file.close()
path = temp_file.name
command = command + " > " + path
proc = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
if proc.stderr:
# if command failed return
os.unlink(path)
return
with open(path, 'r') as f:
data = f.read()
os.unlink(path)
return data
if __name__ == "__main__":
path = "Somepath"
command = 'ecls.exe /files ' + path
print(execute(command))
#!/usr/bin/env bash
# Usage: gdrive_dl.sh <url>
urlBase='https://drive.google.com'
fCookie=tmpcookies
curl="curl -L -b $fCookie -c $fCookie"
confirm(){
$curl "$1" | grep jfk-button-action | sed -e 's/.*jfk-button-action" href="\(\S*\)".*/\1/' -e 's/\&/\&/g'
}
$curl -O -J "${urlBase}$(confirm $1)"
import subprocess
def run_command(cmd_and_args, print_constantly=False, cwd=None):
"""Runs a system command.
:param cmd_and_args: the command to run with or without a Pipe (|).
:param print_constantly: If True then the output is logged in continuous until the command ended.
:param cwd: the current working directory (the directory from which you will like to execute the command)
:return: - a tuple containing the return code, the stdout and the stderr of the command
"""
output = []
process = subprocess.Popen(cmd_and_args, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=cwd)
while True:
next_line = process.stdout.readline()
if next_line:
output.append(str(next_line))
if print_constantly:
print(next_line)
elif not process.poll():
break
error = process.communicate()[1]
return process.returncode, '\n'.join(output), error
from simppl.simple_pipeline import SimplePipeline
sp = SimplePipeline(start=0, end=100):
sp.print_and_run('<YOUR_FIRST_OS_COMMAND>')
sp.print_and_run('<YOUR_SECOND_OS_COMMAND>') ```
commands = ['<YOUR_FIRST_OS_COMMAND>', '<YOUR_SECOND_OS_COMMAND>']
max_number_of_processes = 4
sp.run_parallel(commands, max_number_of_processes) ```
from example_module import example_tool
sp.print_and_run_clt(example_tool.run, ['first_number', 'second_nmber'],
{'-key1': 'val1', '-key2': 'val2'},
{'--flag'}) ```
import logging
from logging.config import dictConfig
logging_config = dict(
version = 1,
formatters = {
'f': {'format':
'%(asctime)s %(name)-12s %(levelname)-8s %(message)s'}
},
handlers = {
'h': {'class': 'logging.StreamHandler',
'formatter': 'f',
'level': logging.DEBUG}
},
root = {
'handlers': ['h'],
'level': logging.DEBUG,
},
)
dictConfig(logging_config)
from simppl.simple_pipeline import SimplePipeline
sp = SimplePipeline(0, 100)
sp.print_and_run('ls')