I’m rewriting some old Python scripts in Ruby. Just to learn a bit more.
I want to thread this code to check websites concurrently. Could somone
offer advice? What are reasonable limits on threads in Ruby?
ips.each do |ip|
begin
Timeout::timeout(5) do
Net::HTTP.start(ip) do |site|
data = site.get( '/' ).body.downcase
if data.include?(minolta) and
data.include?(pagescope)
puts printer + “\t” + ip
elsif data.include?(xerox) and
data.include?(printer)
puts printer + “\t” + ip
else
puts “website\t” + ip
end
end
end
# Don’t really care about exceptions just time outs mostly.
rescue Timeout::Error
puts “Timeout\t” + ip
rescue Exception
puts “General Exception\t” + ip
end
end
What are reasonable limits on threads? Say I’m scanning a class b
network (roughly 65K hosts). How would you break up the threads? I seem
to get too many execution expired errors if I have more than 500 hosts
in one thread. It seems to work best with 256 groups of 256 hosts each
or 254 if you exclude the 0’s and 255’s
What are reasonable limits when working with threads in Ruby? Any tips?
What are reasonable limits on threads? Say I’m scanning a class b
network (roughly 65K hosts). How would you break up the threads? I seem
to get too many execution expired errors if I have more than 500 hosts
in one thread. It seems to work best with 256 groups of 256 hosts each
or 254 if you exclude the 0’s and 255’s
What are reasonable limits when working with threads in Ruby? Any tips?
No tips there… it always depends on the task on hand - just use what
works
for you
if data.include?(minolta) and
# Don't really care about exceptions just time outs mostly.
Looks like you’re trying to perform a network client operation
simultaneously across a lot of different servers. For a non-threaded
approach, look at the EventMachine library. Sync to the latest source
and
look at EventMachine::Deferrable. That should give you considerably more
scalability and performance than trying to solve this with threads. The
Deferrable pattern works like Python’s Twisted. If it doesn’t make sense
to
you, let me know and I can send you some sample code.
I’m rewriting some old Python scripts in Ruby. Just to learn a bit more.
I want to thread this code to check websites concurrently. Could somone
offer advice? What are reasonable limits on threads in Ruby?
Here’s an EventMachine code sample that should do what you need. Notice,
this code is nonthreaded, but it still does all the HTTP GETs
simultaneously. Of course you’ll want to do something more interesting
in
the http.callback block.
what would the preferrer way by to shared data from http.callback? does
it
need protection? how about sharing with ruby green threads?
I’m not exactly sure what you’re asking, Ara. What happens in this code
is
that the HTTP requests are fired off simultaneously, and as they
complete,
the callback gets called for each completion, always on the same thread.
(There are no additional green or native threads being spun here.) So
there’s no contention and no need for mutex protection. If you wanted
for
some reason to run this code simultaneously with unrelated code on other
threads, then of course you’d use the normal thread-safe procedures to
sync
this data with your other threads.
If you wanted for some reason to run this code simultaneously with unrelated
code on other threads, then of course you’d use the normal thread-safe
procedures to sync this data with your other threads.
simultaneously. Of course you’ll want to do something more interesting in
“www.microsoft.com”
puts response[:status]
puts response[:headers]
puts response[:content].length
}
end
