线程子进程的奇怪输出

时间:2014-03-11 15:22:30

标签: python multithreading parsing subprocess output

感谢您花时间看这个,对此有任何帮助将非常感谢!

我试图通过ping一个名为ips的IP地址列表来获取一些网络统计数据。然而我遇到的问题是我的输出是一个包含几个'None'的列表。在实现线程以运行子进程命令之前,下面显示的输出是一系列数字。如果有人可以看一下我的源代码并对这个问题有所了解,我将非常感激!

提前谢谢!

import subprocess
import re
import threading
from multiprocessing import Pool, Lock
from multiprocessing.dummy import Pool as ThreadPool



def get_ips():

    # Fill empty list with IP address
    ips = []
    with open('C:\Python26\ARPips.prn','r')as f:
        for line in f:
            line = line[:-1]
            if line != "end":
                ips.append(line)
        return ips

def ping(pingArgs):


    lock = Lock()
    lock.acquire()

    # Ping with "pingArgs" as the arguments
    ping = subprocess.Popen(pingArgs,
        stdout = subprocess.PIPE,
        stderr = subprocess.PIPE,
        shell=True)

    # Get and parse output
    out = ping.communicate()
    out = ''.join((out))

    lost = re.findall(r"Lost = (\d+)", out)
    minimum = re.findall(r"Minimum = (\d+)", out)
    maximum = re.findall(r"Maximum = (\d+)", out)
    avg = re.findall(r"Average = (\d+)", out)
    no =  re.findall(r"Sent = (\d+)", out)

    # Change output to integers
    lost = [int(x) for x in lost]
    minimum = [int(x) for x in minimum]
    maximum = [int(x) for x in maximum]
    avg = [int(x) for x in avg]
    no = [int(x) for x in no]



    print "%s \t \t %s \t \t%s \t \t %s \t \t%s" % (no, lost, maximum, minimum, avg)
    lock.release()

def main():


    # grab IP address list
    ips = get_ips()

    # Declare global variables
    global position, newIP, pingArgs

    position = 0
    newIP = ips[position]
    position += 1
    pingArgs = ["ping", "-n", "1", "-l", "1", "-w", "100", newIP]


    # Header for output
    print "Packets \t loss(%) \t Max(ms) \t Min(ms) \t Average(ms)"

    # Instantiate Pool objects, and set size of pool
    pool = Pool(processes=12)


    #Ping ips in own threads and return the results
    result = pool.map(ping, ips)


    # Close the pool and wait for work to finish
    pool.close()
    pool.join()

    # print the results
    print result

if __name__ == '__main__':
    main()
    print get_ips()

输出如下所示:

Packets      loss(%)     Max(ms)     Min(ms)     Average(ms)
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[]       []         []       []         []
[None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]
['10.10.10.1', '10.10.10.41', '10.10.10.42', '10.10.10.43', '10.10.10.49', '10.10.10.51', '10.10.10.61', '10.10.10.71', '10.10.10.91', '10.10.10.92', '10.10.10.201', '10.10.10.205', '10.10.10.208', '10.10.10.209', '10.10.10.213', '10.10.10.214']

Process finished with exit code 0

2 个答案:

答案 0 :(得分:2)

  

我遇到的问题是我的输出是一个包含几个'无'的列表

Noneprint result生成result = pool.map(ping, ips),因为ping()函数不返回任何内容(这意味着它在Python中返回None)。

  

在实现线程以运行子进程命令之前,下面显示的输出是一系列数字。

您正在将IP地址传递给Popen()而不是要运行的完整ping命令。

您定义的global pingArgs未在任何地方使用(pingArgs中的本地ping()遮盖它)。您可以通过在同一进程和同一线程中创建结果列表的内置pool.map调用替换map,以查看结果是否相同(正则表达式不匹配)且错误不是与线程/进程有关。

本地 lock = Lock()没用。

以下是解决上述问题的代码:

#!/usr/bin/env python
import re
from multiprocessing.dummy import Pool # use threads
from subprocess import Popen, PIPE

def get_ips(filename):
    ips = []
    with open(filename) as f:
        for line in f:
            line = line.strip()
            if line and line != "end":
                ips.append(line)
    return ips

def ping(ip):
    cmd = ["ping", "-n", "1", "-l", "1", "-w", "100", ip]
    ping = Popen(cmd, stdout=PIPE, stderr=PIPE)

    # Get and parse output
    output, err = ping.communicate()
    out = ''.join([output, err])

    lost = re.findall(r"Lost = (\d+)", out)
    minimum = re.findall(r"Minimum = (\d+)", out)
    maximum = re.findall(r"Maximum = (\d+)", out)
    avg = re.findall(r"Average = (\d+)", out)
    no =  re.findall(r"Sent = (\d+)", out)

    # Change output to integers
    lost = [int(x) for x in lost]
    minimum = [int(x) for x in minimum]
    maximum = [int(x) for x in maximum]
    avg = [int(x) for x in avg]
    no = [int(x) for x in no]

    return "%s \t \t %s \t \t%s \t \t %s \t \t%s" % (
        no, lost, maximum, minimum, avg)

def main():
    # grab IP address list
    ips = get_ips(r'C:\Python26\ARPips.prn')

    # Header for output
    print "Packets \t loss(%) \t Max(ms) \t Min(ms) \t Average(ms)"

    # Instantiate Pool objects, and set size of pool
    pool = Pool(processes=12)

    #Ping ips in own threads and return the results
    results = pool.map(ping, ips)

    # Close the pool and wait for work to finish
    pool.close()
    pool.join()

    # print the results
    print "\n".join(results)

if __name__ == '__main__':
    main()

答案 1 :(得分:0)

这就是我(unix)系统的工作原理

import subprocess
import re
from multiprocessing import Pool, Lock

lock = Lock()
pingArgs = "ping -n -l 1 -w 100 "

def get_ips():
    # Fill empty list with IP address
    ips = []
    with open('C:\\Python26\\ARPips.prn', 'r')as f:
        for line in f:
            line = line[:-1]
            if line != "end":
                ips.append(line)
        return ips

def ping(ip):

    global lock
    pingCommand = pingArgs + ip
    lock.acquire()
    # Ping with "pingArgs" as the arguments
    ping = subprocess.Popen(pingCommand,
        stdout = subprocess.PIPE,
        stderr = subprocess.PIPE,
        shell=True)

    # Get and parse output
    out = ping.communicate()
    out = ''.join((out))

    lost = re.findall(r"Lost = (\d+)", out)
    minimum = re.findall(r"Minimum = (\d+)", out)
    maximum = re.findall(r"Maximum = (\d+)", out)
    avg = re.findall(r"Average = (\d+)", out)
    no =  re.findall(r"Sent = (\d+)", out)

    # Change output to integers
    lost = [int(x) for x in lost]
    minimum = [int(x) for x in minimum]
    maximum = [int(x) for x in maximum]
    avg = [int(x) for x in avg]
    no = [int(x) for x in no]

    lock.release()
    return "%s \t \t %s \t \t%s \t \t %s \t \t%s" % (no, lost, maximum, minimum, avg)

def main():

    # grab IP address list
    ips = get_ips()

    # Header for output
    print("Packets \t loss(%) \t Max(ms) \t Min(ms) \t Average(ms)")

    # Instantiate Pool objects, and set size of pool
    pool = Pool(processes=12)

    #Ping ips in own threads and return the results
    results = pool.map(ping, ips)

    # Close the pool and wait for work to finish

    pool.close()
    pool.join()
    # print the results
    for result in results:
        print(result)

if __name__ == '__main__':
    main()
    print(get_ips())

我遇到的代码问题是ping参数不正确(-n没有接受任何其他参数),lock应该是全局的,shell=True时它不需要一个列表,只有一个字符串,而ping(pingArgs)可以覆盖全局参数,而你希望池只将ip发送给worker,然后worker应该将它添加为最终参数。

我也注意到你没有逃避你在dist上的文件路径,我没有使用/测试那部分,但包括它正确转义以供参考。

编辑:也改变了它,因此每个函数只返回要打印的值,并在完成时打印所有这些值。而不是将它们打印在函数内部以及函数的结果而不返回(Nones列表)。