So I am using the instructions here to have mongrel_cluster start my
various mongrel-deployed apps on server boot.
http://mongrel.rubyforge.org/wiki/MongrelCluster
There is a problem mentioned here:
http://www.ruby-forum.com/topic/105849
In summary—if the server (or the mongrel processes) die ungracefully
leaving their pids behind, then on next boot, mongrel_cluster won’t
succesfully start them up, choking on the leftover pids.
What is the reccommended solution to this? I’d like to have my mongrels
automatically start on boot, in a way that forces them to start even if
there are stale pids. Thanks for any advice!
[ PS: I just discovered this problem the hard way, when my server
unexpetedly lost power, and my production apps failed to start up on the
next boot! Very unpleasant. ]
Jonathan
–
Jonathan R.
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886
rochkind (at) jhu.edu
Newer versions of mongrel_cluster have a --clean option which will
remove stale pids if it finds them. Provide that option to
mongrel_rails cluster::* in your scripts.
Actually, I have investigated further, and am pretty sure I am
duplicating the problem condition properly.
mongrel_cluster_ctl start --clean doesn’t seem to work properly, it’s
throwing that error. Hmm.
I am guessing that the script I find in the mongrel_cluster gem at
resources/mongrel_cluster, which is reccommended on that wiki page… is
not actually what you all are using. And perhaps shouldn’t be included
in the distro anymore? What are others using to ensure mongrel
clusters start at boot?
Jonathan
Jonathan R. wrote:
from /usr/bin/mongrel_rails:18:in `load'
Jonathan
leaving their pids behind, then on next boot, mongrel_cluster won’t
the
410.516.8886
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users
–
Jonathan R.
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886
rochkind (at) jhu.edu
I’m doing everything (everything!) via either monit (on linux) or SMF
(on Solaris).
Using a process manager is a good idea… and having only one
‘production’ way to start mongrel is generally a good thing, too 
-n
I use mongrel_rails cluster::* instead of mongrel_cluster_ctl, however
it looks like you’re using an older version of mongrel_cluster. You
might try upgrading to 1.0.5.
Thanks that’s helpful, I’m investigating that.
Using the instructions at:
http://mongrel.rubyforge.org/wiki/MongrelCluster#OnBootInitializationSetup
I had a little bash script at /etc/init.d/mongrel_cluster that was
copied from
/path/to/mongrel_cluster_gem/resources/mongrel_cluster
That bash script calls mongrel_cluster_ctl with various options. I have
added a “–clean” option to the “start” condition. Would it make sense
to suggest this be added to resources/mongrel_cluster in the actual gem,
to not require this change?
However, now I’m trying to set up an intentionally created stale pid, to
test this feature. I am invoking mongrel_cluster_ctl manually (but in
the manner of the resources/mongrel_cluster script) with the --clean
option. I get this:
/usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:83:in
unlink': No such file or directory - tmp/pids/mongrel.3000.pid (Errno::ENOENT) from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:83:in start’
from
/usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:79:in
each' from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:79:in start’
from
/usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:232:in
run' from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/command.rb:212:in run’
from
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281
from /usr/bin/mongrel_rails:18:in `load’
from /usr/bin/mongrel_rails:18
mongrel_rails cluster::start returned an error.
That was using a mongrel_config.yml file that did have ‘cwd’ set. There
certainly is a tmp/pids/mongrel.3000.pid off of the ‘cwd’ location. But
perhaps the ‘unlink’ isn’t being executed off the ‘cwd’ location as it
ought to be?
Is this a bug in mongrel_cluster_ctl --clean, or perhaps just a bug in
my attempt to duplicate the problem condition and make sure I’ve solved
it?
Thanks for any help.
Jonathan
Eden Li wrote:
if
http://rubyforge.org/mailman/listinfo/mongrel-users
Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users
–
Jonathan R.
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886
rochkind (at) jhu.edu
Believe it or not, I have indeed upgraded to 1.0.5 of mongrel_cluster
and 1.1.4 of mongrel. At least I think I have. I have installed the
newer versions of the gems, and I believe it’s what my scripts are
using.
What makes you think it looks like I am not? Because maybe I’m not even
though I intend to be.
I wonder again, if the sample script distributed with the gem at
resources/mongrel_cluster doesn’t work/isn’t supported… should someone
with commit privs remove it? Or add a comment to it warning it might not
work?
Jonathan
Eden Li wrote:
mongrel_cluster_ctl start --clean doesn’t seem to work properly, it’s
Jonathan
copied from
script) with the --clean option. I get this:
mongrel_cluster/init.rb:79:in `start’
There is a problem mentioned here:
even
Mongrel-users mailing list
Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users
Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users
–
Jonathan R.
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886
rochkind (at) jhu.edu
Yeah, I think that closing of that ticket is right. As far as
mongrel_rails is concerned, it makes sense to use the --clean option,
rather than put it in there to happen by default. (although actually,
I’d have --clean be default).
My problem is that I’m using mongrel_cluster_ctl instead of
mongrel_rails. The nice feature of mongrel_cluster_ctl here is that you
can give it a -c argument which is a directory containing mongrel
cluster yml files. And it will do the invoked operation to ALL of those
config files. I like this. I want to use it.
And mongrel_cluster_ctl’s --clean option appears to be broken to me. I
guess I should figure out how to fix it and submit a patch?
The other problem is that the sample script at
resources/mongrel_cluster, which invokes mongrel_cluster_ctl, doesn’t
invoke the --clean option (which wouldn’t work right even if it did, but
once it does). So I guess I should fix that and submit a patch too?
I’m new to submitting patches for this kind of stuff. Can anyone
reccommend patch etiquette/procedure to do this properly? And/or confirm
that I’m approaching this sanely?
Thanks for the help, all.
Jonathan
Simon S. wrote:
but it was not committed because “mongrel_rails now supports a --clean”.
my mongrel_rails does not support it, but I may have an old version.
I think it would be right for mongrel to clean up it’s stale pid files. Every
program I know of does that, and I patched my mongrel.
Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users
–
Jonathan R.
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886
rochkind (at) jhu.edu
Your backtrace:
/usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/
mongrel_cluster/init.rb
You might try removing stale gems by running sudo gem uninstall mongrel_cluster and removing anything that’s not mongrel_cluster
1.0.5. Also make sure your /usr/bin/mongrel_* are pointing to the
right version.
to write a script which checks the running pids and pid files and if
there is staled pid just remove it?
(and after let the normal starting process run)
mongrel_rails and mongrel_cluster_ctl are written in ruby,
you will not hurt the original library with this modification
if the --clean is not working well…
istvan,
http://weho.st
On Wednesday 05 March 2008 17:59:49 Jonathan R. wrote:
What is the reccommended solution to this?
I don’t know, put there is a patch here:
http://textsnippets.com/posts/show/931
and a ticket got submitted here:
http://rubyforge.org/tracker/index.php?func=detail&aid=11098&group_id=1306&atid=5147
but it was not committed because “mongrel_rails now supports a --clean”.
my mongrel_rails does not support it, but I may have an old version.
I think it would be right for mongrel to clean up it’s stale pid files.
Every
program I know of does that, and I patched my mongrel.
Aha. Thanks. This is curious. I would actually like to have both
versions of mongrel_cluster installed at the moment, so I can easily
switch back if needed. gem is supposed to be capable of this of course.
I’m confused as to why my scirpt in /usr/bin/mongrel_cluster_ctl is
calling the 1.0.2 version, when I have 1.0.5 installed. When I look at
the very brief code in /usr/bin/mongrel_cluster_ctl, it looks like it
should use the latest mongrel_cluster stuff installed. Here’s what’s in
/usr/bin/mongrel_cluster, anyone know why this would be using 1.0.2 even
though I have 1.0.5 installed?
#!/usr/bin/env ruby
This file was generated by RubyGems.
The application ‘mongrel_cluster’ is installed as part of a gem, and
this file is here to facilitate running it.
require ‘rubygems’
version = “> 0”
if ARGV.size > 0 && ARGV[0][0]==95 && ARGV[0][-1]==95
if Gem::Version.correct?(ARGV[0][1…-2])
version = ARGV[0][1…-2]
ARGV.shift
end
end
require_gem ‘mongrel_cluster’, version
load ‘mongrel_cluster_ctl’
Eden Li wrote:
right version.
Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users
–
Jonathan R.
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886
rochkind (at) jhu.edu
try
sudo rm /usr/lib/ruby/gems/1.8/specifications/
mongrel_cluster-1.0.2.gemspec
i think your specs are stale
Okay, I figured out more. Sorry for spamming the list. This is
frustrating me.
I deleted mongrel_cluster 1.0.2 entirely. But check out this stack
trace. Why the heck is mongrel-1.1.4/bin/mongrel_rails requiring
mongrel_rails 1.0.2 and refusing to use mongrel_rails 1.0.5?
/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
gem_original_require': no such file to load -- /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb (LoadError) from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in require’
from
/usr/lib/ruby/gems/1.8/gems/gem_plugin-0.2.3/lib/gem_plugin.rb:134:in
load' from /usr/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:142:in each’
from /usr/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:142:in
each' from /usr/lib/ruby/gems/1.8/gems/gem_plugin-0.2.3/lib/gem_plugin.rb:112:in load’
from
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:278
from /usr/bin/mongrel_rails:18:in `load’
from /usr/bin/mongrel_rails:18
Jonathan R. wrote:
that I’m approaching this sanely?
What is the reccommended solution to this?
I think it would be right for mongrel to clean up it’s stale pid files. Every
program I know of does that, and I patched my mongrel.
Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users
–
Jonathan R.
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886
rochkind (at) jhu.edu
Okay, you’re right. That did work, I didn’t do a good job of removing
mongrel_cluster-1.0.2 before. The question still remains, why the heck
do I need to actually REMOVE 1.0.2 to get mongrel 1.1.4 to use it, when
there’s a newer mongrel_cluster 1.0.5 installed? This shouldn’t actually
be, should it? Aren’t you supposed to be able to have both installed in
gems at once? Anyone have any explanation for why mongrel was
preferring the older one when both were installed? [It’s this kind of
thing that drives me nuts with ruby/rails, especially with deployment
issues].
Now that I’ve gotten it to use mongrel_cluster 1.0.5,
mongrel_cluster_ctl is indeed properly dealing with the --clean
argument, that bug is gone.
Very frustrating. Thanks for the help all. Still plan to submit a patch
for the resources/mongrel_cluster script to use --clean. Does this make
sense? That resource/mongrel_cluster script is mentioned in the
mongrel_cluster README still, so it does seem like it’s supposed to be
still a non-deprecated thing?
Jonathan
Eden Li wrote:
Okay, I figured out more. Sorry for spamming the list. This is
(LoadError)
`each’
those
once it does). So I guess I should fix that and submit a patch too?
Simon S. wrote:
I don’t know, put there is a patch here:
program I know of does that, and I patched my mongrel.
Jonathan R.
Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users
–
Jonathan R.
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886
rochkind (at) jhu.edu
I’m not sure why it prefers the oldest version, but the
mongrel_cluster_ctl script effectively runs:
require_gem “mongrel_cluster”, “> 0”
which seems like it would prefer the oldest. reading the script a tad
more, it looks like you can specify which version to run by passing in
VERSION (e.g. 1.0.5) as the first argument to mongrel_cluster_ctl
Should I try to patch this to prefer the newest version instead?
If anyone’s actually been following along, I filed a patch to fix
resources/mongrel_cluster to pass --clean in.
http://rubyforge.org/tracker/index.php?func=detail&aid=18627&group_id=1306&atid=5147
Jonathan
Eden Li wrote:
preferring the older one when both were installed? [It’s this kind of
make
sense? That resource/mongrel_cluster script is mentioned in the
mongrel_cluster README still, so it does seem like it’s supposed to be
still a non-deprecated thing?
Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users
–
Jonathan R.
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886
rochkind (at) jhu.edu
On Thu, Mar 6, 2008 at 10:29 AM, Jos B. [email protected] wrote:
If only Mongrel/mongrel_cluster would work with daemontools/runit. Pid files
are evil.
Mongrel runs perfectly well with runit.
–
Alexey V.
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
RubyWorks [http://rubyworks.thoughtworks.com]
On Wed, Mar 05, 2008 at 11:57:44AM -0600, Nate V. wrote:
I’m doing everything (everything!) via either monit (on linux) or SMF
(on Solaris).
Using a process manager is a good idea… and having only one
‘production’ way to start mongrel is generally a good thing, too 
If only Mongrel/mongrel_cluster would work with daemontools/runit. Pid
files
are evil.
(Maybe Thin will grow support for this;
#40 Support running thin under daemontools/runit - Thin - thin)
On Thu, Mar 6, 2008 at 9:29 AM, Jos B. [email protected] wrote:
(Maybe Thin will grow support for this;
#40 Support running thin under daemontools/runit - Thin - thin)
Mongrel works great with process supervision. We use runit
extensively; it has eliminated all the pidfile & deployment headaches
discussed here.
jeremy