Question

我想创建一个程序来生成多个子进程，然后读取它们的输出。每个子进程将使用 curl 从服务器读取数据并每隔几秒钟刷新一次该数据。为简单起见，可以将其替换为 subproc1 的简单 cat <filename1>，subproc2 的 cat <filename2> 等。然后（父）程序进程从子进程 stdin 读取，保持顺序，进行一些解析并打印到 stdout。为简单起见，假设它回显以“subproc[1-N]”为前缀的输入

我已经尝试使用 subprocess.Popen、Queue 和守护线程进行读写。但我无法让它按预期工作。

这是一个使用 select 的例子。

#!/usr/local/bin/python3
import subprocess, shlex
import sys, os
import select
import pdb

cmds=[ "ls -l", "ls -a", "./print_nums.sh"]

procs = [ subprocess.Popen(shlex.split(cmd), stdin=subprocess.PIPE, stdout=subprocess.PIPE) for cmd in cmds ]
streams = [p.stdout for p in procs]
streams2Cmds = {k:v for (k,v) in zip(streams, cmds)}

def handler(line, stream):
    print('{}:\n{}'.format(streams2Cmds[stream], line.decode('utf-8')))

while True:
    rstreams,_,_ = select.select(streams, [], [])
    for stream in rstreams:
        print('handle input from {}'.format(streams2Cmds[stream]))
        line = stream.readline()
        handler(line, stream)
    if all(p.poll is not None for p in procs):
        break

for stream in streams:
    handler(stream.read(), stream)

它工作正常，只是它没有为每一行输入添加前缀。输出将类似于：

handle input from ls -l
ls -l:
total 120

handle input from ls -a
ls -a:
.

ls -l:
-rwxr-xr-x  1 i500695  staff   779 Jan 24 14:31 popen2.orig.py
-rwxr-xr-x  1 i500695  staff  1000 Jan 24 14:17 popen2.orig2.py
-rwxr-xr-x  1 i500695  staff   990 Jan 24 14:13 popen2.py
-rwxr-xr-x  1 i500695  staff  1553 Mar  1  2020 popen2_no_select.py
-rw-r--r--  1 i500695  staff  2264 Mar  1  2020 popen2_no_select.py2
-rw-r--r--  1 i500695  staff  1519 Jan 24 14:25 popenv2.py
-rw-r--r--  1 i500695  staff   775 Jan 24 14:23 popenv3.py
-rwxr-xr-x  1 i500695  staff    53 Jan 24 13:25 print_nums.sh
-rw-r--r--  1 i500695  staff   227 Jan 13  2020 proc_a.py
prw-r--r--  1 i500695  staff     0 Jan 24 12:10 proc_a_input
-rw-r--r--  1 i500695  staff   227 Jan 13  2020 proc_b.py
prw-r--r--  1 i500695  staff     0 Jan 24 12:10 proc_b_input
-rw-r--r--  1 i500695  staff  1075 Jan 24 12:16 readTwoStdings.py
-rwxr-xr-x  1 i500695  staff   503 Jan 13  2020 simple_popen.py
-rw-r--r--  1 i500695  staff  3013 Jan 24 12:12 two_subprocesses.py
-rw-r--r--  1 i500695  staff   712 Jan 13  2020 two_subprocesses2.py
-rw-r--r--  1 i500695  staff  1461 Jan 13  2020 two_subprocesses_with_output.py

ls -a:
..
popen2.orig.py
popen2.orig2.py
popen2.py
popen2_no_select.py
popen2_no_select.py2
popenv2.py
popenv3.py
print_nums.sh
proc_a.py
proc_a_input
proc_b.py
proc_b_input
readTwoStdings.py
simple_popen.py
two_subprocesses.py
two_subprocesses2.py
two_subprocesses_with_output.py

./print_nums.sh:
 1
 2
 3
 4
 5

处理来自多个命令行输入源的输入

0 个答案: