Warning: file_get_contents(/data/phpspider/zhask/data//catemap/4/kotlin/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Multithreading Perl线程有时会卡住(其中一些线程正在等待futex)_Multithreading_Perl_Unix_Pipe_Futex - Fatal编程技术网

Multithreading Perl线程有时会卡住(其中一些线程正在等待futex)

Multithreading Perl线程有时会卡住(其中一些线程正在等待futex),multithreading,perl,unix,pipe,futex,Multithreading,Perl,Unix,Pipe,Futex,我正试图运行的多线程Perl应用程序(在Redhat7.4上使用Perl5.10.1)遇到了一个问题。该问题已通过以下设置重现(设置类似于原始Perl应用程序): 有两个文件:main.pl、module1.pm main.pl: 完成一组任务的顶级脚本,作为“main.pl”调用 我无法理解为什么会发生这种情况(可能与线程::队列或管道等有关),如果有人能帮我解决这个问题,那就太好了。我最终希望能够运行所有测试,而不出现任何futex等待/挂起问题。我试过: 使用IPC::Run的超时选

我正试图运行的多线程Perl应用程序(在Redhat7.4上使用Perl5.10.1)遇到了一个问题。该问题已通过以下设置重现(设置类似于原始Perl应用程序):

有两个文件:main.pl、module1.pm

main.pl

  • 完成一组任务的顶级脚本,作为“main.pl”调用
我无法理解为什么会发生这种情况(可能与线程::队列或管道等有关),如果有人能帮我解决这个问题,那就太好了。我最终希望能够运行所有测试,而不出现任何futex等待/挂起问题。我试过:

  • 使用IPC::Run的超时选项检查控件是否返回,但此处没有运气
  • 使用open3()代替IPC::Run:-问题仍然存在,并且比IPC::Run更频繁
注:

  • 当我使用Perl5.16.3时,这个问题不会出现。但是我需要使用Perl 5.10.1使原始应用程序正常工作,因此如果有人能帮助我理解如何解决这个问题,那将是非常棒的:)
  • 在module1::dummy()中,如果将$loCmd更改为“sleep”,将$loArgs更改为(比如)“10”,则问题不会出现(给人的印象是它可能与管道/IO缓冲区有关)
  • 被卡住的线程数可能因不同的运行而有所不同,有时所有线程都会在没有任何问题的情况下完成(可能存在争用条件)

如果使用最新的perl版本5.32运行,是否会遇到相同的问题?
运行['bsub'、'-q regresse'、'-R ws70'、'-K'、“$loCmd$loArgs”]
此命令的作用是什么?其他命令也有同样的问题吗?例如,尝试替换一个伪命令,如
sleep
实际上,我们需要一些有意义的替换来替换
bsub
命令,或者一个指向如何安装它的指针,否则代码在测试设置中不容易使用。我必须使用bsub命令在远程LSF计算设施上运行lsfScript.pl脚本。我将编辑脚本以删除此命令,并在本地计算机上运行一个更简单的命令。我已删除
bsub
命令,并将其更改为一个简单的
echo
命令。
# main.pl
#!/apps/perl/5.10.1/bin/perl

use strict;
use warnings;
use v5.10.1;
use threads;
use threads::shared;

use module1;

use lib "<path to Thread::Queue module>"; 
use Thread::Queue;

my $glNumTests = shift();       # Number of tests(jobs) to run
my $glMaxThreads = shift();     # What is the max allowed number of workers (threads) for queue mode

my $q = Thread::Queue->new();    # A new empty queue

# Worker thread
sub worker
{
    # Thread will loop until no more work
    while (defined(my $item = $q->dequeue())) {
        # Do work on $item
        sleep(5); # dummy pre-work
        $item->dummy(); # Actual task that needs to be run by each thread
    }
}

sub start_threads
{
    my $loNumThreads = shift();
    $loNumThreads = $loNumThreads > $glMaxThreads ? $glMaxThreads : $loNumThreads;
    print "Creating $loNumThreads threads...";
    for ( 1..$loNumThreads )
    {
        threads->create(\&worker);
    }
    print "done\n";
}

sub initialize
{
    my @lotests;
    my $loNumTests = shift();
    for my $i (1..$loNumTests)
    {
        push(@lotests,"Test_".$i);
    }
    return \@lotests;
}

sub launchjobs
{
    my @lotests = @{shift()};
    my $tests = {};
    
    # Create objects
    foreach my $lotest (@lotests)
    {
        $tests->{$lotest}->{"obj"} = module1->new($lotest);
    }
    
    # Start the threads before adding work to the queue
    start_threads(scalar(@lotests));
    
    # Adding work to queue
    foreach my $lotest (@lotests)
    {
        $q->enqueue($tests->{$lotest}->{"obj"});
    }
    # No more work to be added
    $q->end();

    # Wait for threads to finish
    foreach my $thr ( threads -> list() )
    {
        $thr->join();
    }   
}

launchjobs(initialize($glNumTests));
# module1.pm
package module1;

use lib "<InstallationPath>/IPC-Run-20200505.0/lib";
use IPC::Run qw( run ); 

sub new
{
    my $class = shift();
    my $test = shift();
    my $self = {};
    $self->{"testName"} = $test;
    system("\\mkdir -p test_output/$self->{testName}");
    # Create new file track.log for this object
    open(my $OFH,">","test_output/$self->{testName}/track.log") || die "Cannot open track.log for writing in new\n";
    close($OFH);
    bless($self,$class);
    return $self;
}

sub logTracker
{
    # Writes out the message to track.log file 
    my $self = shift();
    my $message = shift();  
    open(my $OFH,">>","test_output/$self->{testName}/track.log") || die"Cannot open track.log for writing\n";
    print $OFH $message;
    close($OFH);    
}

sub dummy
{
    my $self = shift();
    print "running $self->{testName}\n";

    my $loCmd = "echo"; # Command to be run
    my $loArgs = "This is test $self->{testName}"; # Arguments to the above command
    
    $self->logTracker("Calling run\n");
    run [$loCmd,$loArgs],'>&',"test_output/$self->{testName}/output";
    $self->logTracker("run completed\n");
}
1;
strace: Process 41187 attached with 8 threads
[pid 42343] read(22,  <unfinished ...>
[pid 42292] read(20,  <unfinished ...>
[pid 42291] read(25,  <unfinished ...>
[pid 42282] read(24,  <unfinished ...>
[pid 42234] read(43,  <unfinished ...>
[pid 42212] read(18,  <unfinished ...>
[pid 41187] futex(0x7ff2597939d0, FUTEX_WAIT, 42212, NULL <unfinished ...>
[pid 42338] read(16,
-perl,41187 main.pl 1000 128
    |-perl,42614 main.pl 1000 128
    |-perl,42615 main.pl 1000 128
    |-perl,42616 main.pl 1000 128
    |-perl,42617 main.pl 1000 128
    |-perl,42618 main.pl 1000 128
    |-perl,42620 main.pl 1000 128
    |-perl,42621 main.pl 1000 128
    |-{perl},42212
    |-{perl},42234
    |-{perl},42282
    |-{perl},42291
    |-{perl},42292
    |-{perl},42338
    `-{perl},42343