SizedQueue

Ruby
1

Does anyone know about threads and SizedQueue?

I want to limit or control the number of threads created in a script. I
am not sure what is the best way to handle this.

I wrote a test script using SizedQueue to limit the number of threads.

require ‘thread’

@items = %w{item1 item2 item3 item4 item5}
@queue = SizedQueue.new(1)
@thread = []
@nap = 3

def testthread(item)
@thread << Thread.new(item) do | i |
puts Thread.list
puts “#{i} will sleep for #{@nap} seconds”
sleep(@nap)
puts “#{i} woke up”
@queue.deq
end
end

@items.each do | item |
@queue.enq( testthread(item) )
end

@thread.each { | th | th.join }

I set the SizedQueue max to one but the output I get is this.

#Thread:0x28955c8
#Thread:0x28aa420
item1 will sleep for 3 seconds
#Thread:0x28aa420
#Thread:0x28955c8
#Thread:0x28aa0d8
item2 will sleep for 3 seconds
item2 woke up
item1 woke up
#Thread:0x28aa420
#Thread:0x28955c8
#Thread:0x28aa060
item3 will sleep for 3 seconds
#Thread:0x28aa060
#Thread:0x28955c8
#Thread:0x28a98f8
item4 will sleep for 3 seconds
item4 woke up
item3 woke up
#Thread:0x28955c8
#Thread:0x28a9b08
item5 will sleep for 3 seconds
item5 woke up

The question is why are there three threads running concurrently from
the output. I know one is the main thread. Then I expect only one
other thread to be running. Is this right or am I doing something
wrong?

Thanks for any suggestions or input.

2

On 6/4/06, Paul T. [email protected] wrote:

Does anyone know about threads and SizedQueue?

I want to limit or control the number of threads created in a script. I
am not sure what is the best way to handle this.

Subclass Thread, then use a class variable to hold a thread
counter. Then, simply return from the constructor when the counter
reaches a certain value.

3

2006/6/3, Paul T. [email protected]:

Does anyone know about threads and SizedQueue?

I want to limit or control the number of threads created in a script. I
am not sure what is the best way to handle this.

You probably haev a misconception on how a SizedQueue works. The limit
on the queue ensures that at most n elements reside in the queue at
any given point in time. You typically do that if the queue is filled
faster than it is emptied. When the queue is full an enqueue operation
blocks the calling thread.

What you probably want is a leigth weight processing framework. The
basic idea is to have a single queue (sized limited or not) and a
fixed number of threads reading from the queue.

A typical setting:

require ‘thread’

MAX_THREADS = 10
QUEUE = Queue.new

threads = (1…MAX_THREADS).map do |i|
Thread.new(QUEUE) do |q|
until ( q == ( task = q.deq ) )
# process task
print “thread-”, i, " ", task.inspect, “\n”
end
end
end

generate tasks

1000.times {|i| QUEUE.enq i}

send terminators down the queue

threads.size.times { QUEUE.enq QUEUE}

wait for threads to finish

threads.each {|t| t.join}

eof

HTH

robert

4

Robert K. wrote:

Thanks Robert. I will give it a try.

Paul

5

On Fri, Jan 22, 2010 at 11:17 AM, Lune L. [email protected]
wrote:

   print "thread-", a, " ", task.inspect, "\n"

In my loop “30 times” 5 * 6 threads are created.
But when I add “10 times” I should see 10 * what I had earlier. This is
not happening. I have my 5 * 6 threads created and that’s all. Could you
explain why ?

The first time around you insert 30 elements and then one end-of-work
token (the QUEUE itself) per thread. Then you join the threads, so the
main thread will wait at that point. The 5 threads will consume the 30
tokens and eventually consume the end-of-work token and exit. After
that, the main thread goes through the next 9 iterations inserting
items in the queue, but there are no more threads to consume the queue
items.

Jesus.

6

Hi Robert

Thanks for your example.

I wanted to play with it to really understand how that works. So I tried
this :
QUEUE = Queue.new
threads = (1…5).map do |a|
Thread.new(QUEUE) do |q|
until ( q == (task = q.deq))
print “thread-”, a, " ", task.inspect, “\n”
end
end
end

10.times do |t|
puts “t #{t}”
30.times { |s| puts “s #{s}”; QUEUE.enq(s) }
threads.size.times { QUEUE.enq QUEUE }
threads.each {|t| t.join}
end

In my loop “30 times” 5 * 6 threads are created.
But when I add “10 times” I should see 10 * what I had earlier. This is
not happening. I have my 5 * 6 threads created and that’s all. Could you
explain why ?

Thanks