Question

我正在处理带有函数的文件，并按照以下方式启动我的线程：

for my $file (@files){
  $threads[$k] = threads->create('function', $file);
  $k++;
}

我想限制并行进程的数量。怎么做的？我查看了很多信号量/队列示例，但找不到任何对我来说简单的事情。

有什么想法我可以简单地限制线程数吗？

Answer 1

限制并行性的最直接的方法，也是部署线程的更有效方法之一，是操作“工作线程”＆＃39;模型。

具体来说 - 让一个线程处于循环中，读取队列并对其进行操作。

这就是这样的事情：

#!/usr/bin/perl

use strict;
use warnings;

use threads;

use Thread::Queue;

my $nthreads = 5;

my $process_q = Thread::Queue->new();
my $failed_q  = Thread::Queue->new();

#this is a subroutine, but that runs 'as a thread'.
#when it starts, it inherits the program state 'as is'. E.g.
#the variable declarations above all apply - but changes to
#values within the program are 'thread local' unless the
#variable is defined as 'shared'.
#Behind the scenes - Thread::Queue are 'shared' arrays.

sub worker {
    #NB - this will sit a loop indefinitely, until you close the queue.
    #using $process_q -> end
    #we do this once we've queued all the things we want to process
    #and the sub completes and exits neatly.
    #however if you _don't_ end it, this will sit waiting forever.
    while ( my $server = $process_q->dequeue() ) {
        chomp($server);
        print threads->self()->tid() . ": pinging $server\n";
        my $result = `/bin/ping -c 1 $server`;
        if ($?) { $failed_q->enqueue($server) }
        print $result;
    }
}

#insert tasks into thread queue.
open( my $input_fh, "<", "server_list" ) or die $!;
$process_q->enqueue(<$input_fh>);
close($input_fh);

#we 'end' process_q  - when we do, no more items may be inserted,
#and 'dequeue' returns 'undefined' when the queue is emptied.
#this means our worker threads (in their 'while' loop) will then exit.
$process_q->end();

#start some threads
for ( 1 .. $nthreads ) {
    threads->create( \&worker );
}

#Wait for threads to all finish processing.
foreach my $thr ( threads->list() ) {
    $thr->join();
}

#collate results. ('synchronise' operation)
while ( my $server = $failed_q->dequeue_nb() ) {
    print "$server failed to ping\n";
}

信号量实际上是关于仲裁对有限资源的访问，以及用于保护＆＃39;过程的一部分。

因此，如果您希望在代码中包含 - 比如 - ssh操作，但又不想同时拥有超过20个连接，那么您可以：

my $ssh_limit = Thread::Semaphore -> new (20);

在你的主题中：

$ssh_limit -> down;
 #do ssh thing
$ssh_limit -> up;

每个线程都将阻塞，直到有可用资源。

但这不是控制整个线程的有效方法。 - 答案就是首先启动正确的号码，并使用队列来提供数据。

Perl线程和信号量

1 个答案: