Passing file descriptors when exec()ing

Ruby JRuby
1

I think I’m about ready to give up on this approach given that JRuby
doesn’t claim to be able to fork. But I’d be interested if anyone had
an explanation for this behaviour, and whether there might be a
workaround.

Here’s reader1:

File.open(“testfile”, “w”) { |fh| fh.print “Hello World!” }

p $$

f1 = File.open(“testfile”)
print f1.sysread(6)
Process.waitpid2(fork { exec(“/usr/bin/ruby1.8”, “reader2”,
f1.fileno.to_s) })

Here’s reader2:

begin
f2 = IO.for_fd(ARGV[0].to_i)
print f2.sysread(6)
print “\n”
rescue Errno::EBADF => badf
print “File descriptor #{ARGV[0]}?\n”
end

p $$

So reader1 should open a file, display 6 bytes from it, and then pass
the same file handle to reader2 via fork/exec call, wait for the process
to exit, then exit itself.

The result under MRI is what I expect, two different process IDs and all
the data.

mattbee@desk4:~$ ruby reader1
26983
Hello World!
26984

JRuby 1.5.5 does this, and needs interrupting:

mattbee@desk4:~$ /usr/local/jruby-1.5.5/bin/jruby
-J-Djruby.fork.enabled=true reader1
WARNING: fork is highly unlikely to be safe or stable on the JVM.
Have fun!
27026
Hello File descriptor 4?
27054
^C

And my system JRuby (1.1.6) terminates, but gets muddled in a different
way:

mattbee@desk4:~$ jruby -J-Djruby.fork.enabled=true reader1
WARNING: fork is highly unlikely to be safe or stable on the JVM.
Have fun!
27110
Hello reader1:7: No child processes - No child processes
(Errno::ECHILD)
mattbee@desk4:~$ File descriptor 4?
27153

The fork is happening on JRuby, as I can see different pids but I’m not
sure why the same FD is invalid in the child process, and whether
there’s any point in trying to fix it.

If I take out the fork, and change reader1 to simply finish with:

exec(“/usr/bin/ruby1.8”, “reader2”, f1.fileno.to_s)

Then JRuby still gives me:

mattbee@desk4:~$ /usr/local/jruby-1.5.5/bin/jruby reader1
28969
Hello File descriptor 4?
28998

i.e. it gives me a new pid as if I had forked, and the same file
descriptor still isn’t valid.

Behind the scenes I see JRuby handing off to a POSIX library, which
looks like it is binding pretty directly to fork and exec. But if that
were happening it ought to work :slight_smile: I think the correct answer is “stop
trying to supervise processes from Java” but I would be interested to
know if it ever could work, and why it doesn’t at the moment.

–
Matthew B. Bytemark Hosting
http://www.bytemark.co.uk/
tel: +44 (0) 1904 890890

2

On Wed, Dec 1, 2010 at 6:30 AM, Matthew B. [email protected]
wrote:

I think I’m about ready to give up on this approach given that JRuby doesn’t
claim to be able to fork. But I’d be interested if anyone had an
explanation for this behaviour, and whether there might be a workaround.

fork is in theory “ok” to do on the JVM provided that you immediately
exec, since the child process will not inherit very important
threads from the parent like GC, signal handling, jitting.

However fork + exec will probably never be safe from Ruby code in
JRuby (and maybe even from Java code) because of all those other JVM
threads. fork only brings along the calling thread, which means that
if anything… anything happens in that thread between the ‘fork’
and ‘exec’ that requires the other JVM threads be active and
functional, the child process will fail horribly.

exec is also currently simulated in JRuby so that it will work
without requiring a separate native call-out. By simulated, I mean we
don’t actually exec(3) POSIX call, which would replace the whole
process and inherit file descriptors, etc. Instead, we just use the
JVM’s process-launching and wait for that child process to terminate.
JVM’s process-launching does its own fork + exec in native code
(preventing possible thread-switching between the two), so it does
inherit most file descriptors, but the JVM code goes to great
lengths to prevent it from sharing stdio descriptors (in order to
really isolate the child as an external process). As a result, it can
get a little wacky…at least when comparing to how fork + exec works
normally.

Your best option would be to use (and help me improve) the “spoon”
gem, which uses FFI to wrap posix_spawn. posix_spawn is a
semi-standard POSIX function that does fork + exec in a single call,
as you want here, with various options for how the parent file
descriptors, etc, should be inherited. This allows you to get around
what we “simulate” for exec, the instability of manually forking the
JVM, and the problems of the JVM’s own process-launching. In fact,
here’s an example for launching ‘vi’, which properly inherits tty,
stdio, and all…something impossible to do with the JVM’s built-in
process-launching facilities.

require ‘spoon’

Spoon.spawnp ‘vi’
Process.waitall

Does that explain it well enough?

(a version of this probably should go on the wiki)

  • Charlie
3

Thanks for the fast reply Charles! I’m trying to write an inetd type
service, which you can see has been difficult.

I had read your post from last year on the subject, but I didn’t know
that “exec” isn’t the system exec call under Java (my test found that
out, I just didn’t know what it was doing instead). So it basically
acts like system() followed by exit(), and makes sure that I don’t have
the old stdin? That is a bummer for an inetd.

More generally, though, it seems fair to say that fork & exec aren’t
safe in any multithreaded environment unless you can reliably stop all
other threads while you do it, and also suppress any signals.

Furthermore in JRuby or managed languages, “Thread.critical” or other
stop-the-world facilities won’t protect me from external threads messing
up a fork/exec. Therefore posix_spawn will be the only safe option.

Therefore therefore … the only sane option to make them work as
intended will be to skirt around the JVM and call the libc functions
through FFI.

I think that’s just a rewording of what you told me.

So - I will try harder on that front and try to send you a better spoon
(just downloaded it, mutter, gems, mutter mutter). However my instinct
will be, rather than keep a separate gem, to redefine Kernel.fork & exec
to call the real ones whose behaviour will match what Ruby/UNIX
programmers will expect. The mapping of Ruby Kernel.exec onto
java.lang.Runtime’s exec seems a bit misleading to me, if that’s what
was going on.

Regards,

–
Matthew B. Bytemark Hosting
http://www.bytemark.co.uk/
tel: +44 (0) 1904 890890

4

On 01/12/10 15:57, Matthew B. wrote:

There is definitely something else going on with file descriptors,
unrelated to fork/exec troubles. Even after switching to posix_spawn, my
accepted() socket descriptors are reported as non-sockets in the child
process - does IO#fileno correspond to actual libc-level file
descriptors, or does Java indirect them somehow?

FWIW, JRuby, ffi, bad file descriptor - Ruby - Ruby-Forum answers my question

  • the file descriptors aren’t real, and there’s no way JRuby can get
    them! Sharing File Descriptors between Java and JNI suggests a
    hacky way to get at the real ones through JNI, but I’m not sure that’s
    worth the bother.

If IO#fileno is lying maybe it could start the integer values at 1000 to
make it obvious? :slight_smile:

–
Matthew B. Bytemark Hosting
http://www.bytemark.co.uk/
tel: +44 (0) 1904 890890

5

There is definitely something else going on with file descriptors,
unrelated to fork/exec troubles. Even after switching to posix_spawn,
my accepted() socket descriptors are reported as non-sockets in the
child process - does IO#fileno correspond to actual libc-level file
descriptors, or does Java indirect them somehow? Sorry, I will
investigate a bit more thoroughly once the rest of the project is
working.

Regards,

–
Matthew B. Bytemark Hosting
http://www.bytemark.co.uk/
tel: +44 (0) 1904 890890