有些东西在从不同进程访问fifo时阻塞线程

时间:2018-06-12 12:20:05

标签: linux pandas perl python-multithreading multiprocess

我有一个Perl脚本和Python脚本。 Perl脚本通过FIFO提供Python脚本。 Python脚本执行Perl脚本,并运行一些线程来处理Perl的数据输出。我将FIFO直接读入pandas DataFrame。

有时它会运行,有时会卡住。也许是死锁。我做错了什么?

Python代码:

from threading import Thread
import os
import subprocess
import numpy as np
import pandas as pd

class MyThread(Thread):
    def __init__(self, fifo_name):
        self.fifo_name = fifo_name
        self.results = None
        super(MyThread, self).__init__()

    def run(self):
        try:
            os.mkfifo(self.fifo_name)
        except FileExistsError:
            pass
        self.results = pd.read_csv(self.fifo_name)

def Main():
    t11= MyThread("fifo1_msg1")
    t11.start()

    t12= MyThread("fifo1_msg2")
    t12.start()

    t21= MyThread("fifo2_msg1")
    t21.start()

    t22= MyThread("fifo2_msg2")
    t22.start()

    # run perl scripts that writes to fifos
    subprocess.Popen(["perl", "dumpData.pl", file1, fifo1_msg1, fifo1_msg2, bufsize=1, stderr=subprocess.STDOUT)
    subprocess.Popen(["perl", "dumpData.pl", file2, fifo2_msg1, fifo2_msg2], bufsize=1, stderr=subprocess.STDOUT)

    t11.join()
    t12.join()
    t21.join()
    t22.join()

    t_plot = Thread(target=make_plots, args=(t11.results, t12.results, t21.results, t22.results))
    t_plot.start()
    t_plot.join()

if __name__ == "__main__":
    Main()

Perl代码:

use warnings;
use strict;
$|++;  # turn on autoflush
sub dump_file
{
    my $filename = shift;
    my $fifo_msg1 = shift;
    my $fifo_msg2 = shift;

    open(my $fifo1, ">", $fifo_msg1 ) || die();
    open(my $fifo2, ">", $fifo_msg2) || die();

    print $fifo1 "date,t,p\n";
    print $fifo2 "time,x,y,z\n";
    # some pseudo code as I don't want to put whole code
    # while not eof, read line..
       # read variables from line 
       if(case_bla) {
           printf $fifo1 "%d,%d,%d\n", date,t,p;
       }
       else {
           printf $fifo2 "%d,%d,%d,%d\n", time,x,y,z;
       }
    # end while
    close($fifo1);
    close($fifo2);
}
dump_file($inputfile, $fifo_msg1, $fifo_msg2);

0 个答案:

没有答案