multithreading - perl system call causing hang when using threads -

i newbie perl, please excuse ignorance. (i'm using windows 7)

i have borrowed echicken's threads example script , wanted use basis script make number of system calls, have run issue beyond understanding. illustrate issue seeing, doing simple ping command in example code below.

• $nb_process number or simultaneous running threads allowed.
• $nb_compute number of times want run sub routine (i.e total number of time issue ping command).

when set $nb_compute , $nb_process same value each other, works perfectly.

however when reduce $nb_process (to restrict number of running threads @ 1 time), seems lock once number of threads defined in $nb_process have started.

it works fine if remove system call (ping command).

i see same behaviour other system calls (it'd not ping).

please help? have provided script below.

#!/opt/local/bin/perl -w    use threads;    use strict;    use warnings;     @a = ();    @b = ();      sub sleeping_sub ( $ $ $ );    print "starting main program\n";     $nb_process = 3;    $nb_compute = 6;    $i=0;    @running = ();    @threads;    while (scalar @threads < $nb_compute) {         @running = threads->list(threads::running);        print "loop $i\n";        print "  - begin loop >> nb running threads = ".(scalar @running)."\n";         if (scalar @running < $nb_process) {            $thread = threads->new( sub { sleeping_sub($i, \@a, \@b) });            push (@threads, $thread);            $tid = $thread->tid;            print "  - starting thread $tid\n";        }        @running = threads->list(threads::running);        print "  - after starting >> nb running threads = ".(scalar @running)."\n";        foreach $thr (@threads) {            if ($thr->is_running()) {                $tid = $thr->tid;                print "  - thread $tid running\n";            }            elsif ($thr->is_joinable()) {                $tid = $thr->tid;                $thr->join;                print "  - results thread $tid:\n";                print "  - thread $tid has been joined\n";            }        }         @running = threads->list(threads::running);        print "  - end loop >> nb threads = ".(scalar @running)."\n";        $i++;    }     print "\njoining pending threads\n";    while (scalar @running != 0) {       foreach $thr (@threads) {            $thr->join if ($thr->is_joinable());        }        @running = threads->list(threads::running);   }    print "nb started threads = ".(scalar @threads)."\n";    print "end of main program\n";      sub sleeping_sub ( $ $ $ ) {      @res2 = `ping 136.13.221.34`;      print "\n@res2";     sleep(3);    }  

the main problem program have busy loop tests whether thread can joined. wasteful. furthermore, reduce amount of global variables better understand code.

other eyebrow-raiser:

• never ever use prototypes, unless know exactly mean.
• the sleeping_sub not use of arguments.
• you use threads::running list lot without contemplating whether correct.

it seems want run n workers @ once, want launch m workers in total. here elegant way implement this. main idea have queue between threads threads finished can enqueue thread id. thread joined. limit number of threads, use semaphore:

use threads; use strict; use warnings; use feature 'say';  # "say" works "print", appends newline. use thread::queue; use thread::semaphore;  @pieces_of_work = 1..6; $num_threads = 3; $finished_threads = thread::queue->new; $semaphore = thread::semaphore->new($num_threads);  $task (@pieces_of_work) {   $semaphore->down;  # wait permission launch thread    "starting new thread...";    # create new thread in scalar context   threads->new({ scalar => 1 }, sub {     $result = worker($task);                # run actual task     $finished_threads->enqueue(threads->tid);  # report joinable "in second"     $semaphore->up;                            # allow thread launched     return $result;   });    # maybe join threads   while (defined( $thr_id = $finished_threads->dequeue_nb )) {     join_thread($thr_id);   } }  # wait threads finished, "down"ing semaphore: $semaphore->down 1..$num_threads; # end finished thread id queue: $finished_threads->enqueue(undef);  # join threads left: while (defined( $thr_id = $finished_threads->dequeue )) {   join_thread($thr_id); } 

with join_thread , worker defined as

sub worker {   ($task) = @_;   sleep rand 2; # sleep random amount of time   return $task + rand; # return number }  sub join_thread {   ($tid) = @_;   $thr = threads->object($tid);   $result = $thr->join;   "thread #$tid returned $result"; } 

we output:

starting new thread... starting new thread... starting new thread... starting new thread... thread #3 returned 3.05652608754778 starting new thread... thread #1 returned 1.64777186731541 thread #2 returned 2.18426146087901 starting new thread... thread #4 returned 4.59414651998983 thread #6 returned 6.99852684265667 thread #5 returned 5.2316971836585 

(order , return values not deterministic).

the usage of queue makes easy tell which thread has finished. semaphores make easier protect resources, or limit amount of parallel somethings.

the main benefit of pattern far less cpu used, when contrasted busy loop. shortens general execution time.

while big improvement, better! spawning threads expensive: fork() without copy-on-write optimizations on unix systems. whole interpreter copied, including variables, state etc. have created.

therefore, threads should used sparingly, , spawned possible. introduced queues can pass values between threads. can extend few worker threads pull work input queue, , return via output queue. difficulty have last thread exit finish output queue.

use threads; use strict; use warnings; use feature 'say'; use thread::queue; use thread::semaphore;  # define i/o queues $input_q  = thread::queue->new; $output_q = thread::queue->new;  # spawn workers $num_threads = 3; $all_finished_s = thread::semaphore->new(1 - $num_threads); # negative start value! @workers; (1 .. $num_threads) {   push @workers, threads->new( { scalar => 1 }, sub {     while (defined( $task = $input_q->dequeue )) {       $result = worker($task);       $output_q->enqueue([$task, $result]);     }     # here when input queue exhausted.     $all_finished_s->up;     # end output queue if last thread (the semaphore > 0).     if ($all_finished_s->down_nb) {       $output_q->enqueue(undef);     }   }); }  # fill input queue tasks @pieces_of_work = 1 .. 6; $input_q->enqueue($_) @pieces_of_work;  # finish input queue $input_q->enqueue(undef) 1 .. $num_threads;  # data while (defined( $result = $output_q->dequeue )) {   ($task, $answer) = @$result;   "task $task produced $answer"; }  # join workers: $_->join @workers; 

with worker defined before, get:

task 1 produced 1.15207098293783 task 4 produced 4.31247785766295 task 5 produced 5.96967474718984 task 6 produced 6.2695013168678 task 2 produced 2.02545636412421 task 3 produced 3.22281619053999 

(the 3 threads joined after output printed, output boring).

this second solution gets bit simpler when detach threads – main thread won't exit before threads have exited, because listening input queue finished last thread.