Sigh! I'm depressed. Debian vs Ruby and the backtick and subshells

[amin1426]

I have just strace’d weird bug to it’s root cause.

For decades bash was the “sh” of choice for linux systems.

Then somebody threw some money at the Debian foundation to make Debian
boot faster.

The solution was to tweak the init scripts to be perfectly POSIX
Lowest Common Denominator “sh” compatible, and then make dash the
default “sh”.

To allow for legacy scripts that expect /bin/sh === /bin/bash you can
sudo dpkg-reconfigure dash
and switch back to sh === bash

Thus on Debian (and Debian derivatives like Ubuntu) Rubies backtick
(iff there is a glob or a redirection present) is HARDCODED to /bin/sh
but will (depending on the users whim) give you dash or bash.

For example the &> redirection is a bashism…
ls -l /bin/sh
lrwxrwxrwx 1 root root 4 2008-08-12 16:54 /bin/sh -> dash
build@ws1419:~/AutomatedBuild/test$ ruby -e ‘system(“ls
NOFILEWITHTHISNAME&>foo”)’
ls: cannot access NOFILEWITHTHISNAME: No such file or directory
build@ws1419:~/AutomatedBuild/test$ cat foo
build@ws1419:~/AutomatedBuild/test$ sudo dpkg-reconfigure dash
Removing diversion of /bin/sh to /bin/sh.distrib by dash' Removingdiversion of /usr/share/man/man1/sh.1.gz to
/usr/share/man/man1/sh.distrib.1.gz by dash’
build@ws1419:~/AutomatedBuild/test$ ruby -e ‘system(“ls
NOFILEWITHTHISNAME&>foo”)’
build@ws1419:~/AutomatedBuild/test$ cat foo
ls: cannot access NOFILEWITHTHISNAME: No such file or directory

AAARGH!

As I always say… if a bug is hard to find… you won’t know what to
do with it once you find it.

John C. Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : [email protected]
New Zealand

[amin1426]

On 12.08.2008 07:48, John C. wrote:

lrwxrwxrwx 1 root root 4 2008-08-12 16:54 /bin/sh -> dash
AAARGH!

As I always say… if a bug is hard to find… you won’t know what to
do with it once you find it.

You should probably mention that the bug is actually in your code: if
you rely on bash features present in /bin/sh then that’s the error.

Kind regards

robert

[amin1426]

I am using Debian sid, which sh is bash.
And, in fact, I hate ruby’s way of system call. Ugly and hard to use and
stupid hardcoded.

[amin1426]

On 12 Aug., 08:49, Magicloud M.
[email protected] wrote:

I am using Debian sid, which sh is bash.
And, in fact, I hate ruby’s way of system call. Ugly and hard to use and
stupid hardcoded.

/bin/sh is the least common denominator for all Unix like systems. So
in terms of portability it is a good choice to make this the default
shell. And if you do not like that shell, you can use another one
anytime:

09:06:03 ~$ ruby -e ‘system “/bin/bash”, “-c”, “echo "$
{BASH_VERSINFO[]}""’
3 2 39 19 release i686-pc-cygwin
09:06:06 ~$ ruby -e 'system ENV[“SHELL”], “-c”, "echo "$
{BASH_VERSINFO[]}"”’
3 2 39 19 release i686-pc-cygwin
09:06:07 ~$

Put that into a method and you can use your bash whenever you want.
No need to blame Ruby here when in fact it behaves reasonable.

Kind regards

robert

[amin1426]

Le 12 août 2008 à 09:07, Robert K. a écrit :

Put that into a method and you can use your bash whenever you want.
No need to blame Ruby here when in fact it behaves reasonable.

Not that you can overload the method, too…

module Kernel
def `(v)
…
end
end

(Maybe it’s a bit eeeeevil…)

Fred

[amin1426]

build@ws1419:~/AutomatedBuild/test$ ruby -e ‘system(“ls
NOFILEWITHTHISNAME&>foo”)’

Magicloud M. wrote:

I am using Debian sid, which sh is bash.
And, in fact, I hate ruby’s way of system call. Ugly and hard to use and
stupid hardcoded.

The choice of `` is deliberate. It’s to inspire you to get off your lazy
butt
and learn the Ruby-internal way to do something, such as Pathname.glob.
There’s
always a way to avoid a shell command…

[amin1426]

On Tue, 2008-08-12 at 14:48 +0900, John C. wrote:

lrwxrwxrwx 1 root root 4 2008-08-12 16:54 /bin/sh → dash
AAARGH!

As I always say… if a bug is hard to find… you won’t know what to
do with it once you find it.

Why does Debian use dash? I’ve never even heard of dash until I saw
this thread! Is there some religious war against bash?

M. Edward (Ed) Borasky
ruby-perspectives.blogspot.com

“A mathematician is a machine for turning coffee into theorems.” –
Alfréd Rényi via Paul Erdős

[amin1426]

On Tue, Aug 12, 2008 at 8:54 AM, M. Edward (Ed) Borasky
[email protected] wrote:

On Tue, 2008-08-12 at 14:48 +0900, John C. wrote:

Then somebody threw some money at the Debian foundation to make Debian
boot faster.

Why does Debian use dash? I’ve never even heard of dash until I saw
this thread! Is there some religious war against bash?

Almquist shell

“Dash, like ash, executes scripts faster than bash and depends on
fewer libraries. It is believed to be more reliable in case of upgrade
problems or disk failures.”

https://wiki.ubuntu.com/DashAsBinSh
“The major reason to switch the default shell was efficiency. bash is
an excellent full-featured shell appropriate for interactive use;
indeed, it is still the default login shell. However, it is rather
large and slow to start up and operate by comparison with dash. A
large number of shell instances are started as part of the Ubuntu boot
process. Rather than change each of them individually to run
explicitly under /bin/dash, a change which would require significant
ongoing maintenance and which would be liable to regress if not paid
close attention, the Ubuntu core development team felt that it was
best simply to change the default shell.”

[amin1426]

On Tue, Aug 12, 2008 at 7:37 AM, Phlip [email protected] wrote:

butt and learn the Ruby-internal way to do something, such as Pathname.glob.
There’s always a way to avoid a shell command…

This is a ridiculous claim. I’d spend more time writing a response,
but I’m hoping you’re just being sarcastic.

Michael G.

[amin1426]

On Tue, Aug 12, 2008 at 1:48 AM, John C. [email protected]
wrote:

Thus on Debian (and Debian derivatives like Ubuntu) Rubies backtick
(iff there is a glob or a redirection present) is HARDCODED to /bin/sh
but will (depending on the users whim) give you dash or bash.

For as long as I’ve been working with *nix systems the rule was
always: either explicitly invoke Bash, or understand and use only
the lowest common denominator bourne-compatible shell features. In
fact, the best practice is to only use features which are common to
both Bourne and CSH-compatible shells. It may come as a shock, but
there are actually more *NIXen than Linux, and many of them don’t use
Bash as the default shell.

If you expect portability, you shouldn’t be be doing anything that
depends on bash-specific features in your #system()/backtick calls.
If you need to invoke a specific shell, the popen() family of methods
are your friends.

–
Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

[amin1426]

I am using Debian sid, which sh is bash.
And, in fact, I hate ruby’s way of system call. Ugly and hard to use and
stupid hardcoded.

The choice of `` is deliberate. It’s to inspire you to get off your lazy
butt and learn the Ruby-internal way to do something, such as Pathname.glob.
There’s always a way to avoid a shell command…

The backtick syntax is a convenience inherited from Bash and Perl. It
makes one very common simple case of shell interaction easy and
concise. I typically only use backticks in quick one-off scripts. If
I need to interact with the shell in a larger program I avoid it and
use either system() (for clarity) or popen*() (for greater control).

–
Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

[amin1426]

On Aug 11, 2008, at 11:48 PM, John C. wrote:

Thus on Debian (and Debian derivatives like Ubuntu) Rubies backtick
(iff there is a glob or a redirection present) is HARDCODED to /bin/sh
but will (depending on the users whim) give you dash or bash.

i could easily be wrong, but i have a dim recollection that the
location of sh is a POSIX standard itself.

backticks seem to respect the SHELL var here

cfp:~ > SHELL=tcsh ruby -e’ puts echo $SHELL 2>out ’
tcsh

a @ http://codeforpeople.com/

[amin1426]

On Tue, Aug 12, 2008 at 1:33 PM, John C. [email protected]
wrote:

On Tue, 12 Aug 2008, M. Edward (Ed) Borasky wrote:

“A mathematician is a machine for turning coffee into theorems.” –
Alfréd Rényi via Paul Erdõs

I turn coffee into Ruby.

How is rubby formed? How code get prorgam?

martin

[amin1426]

On Tue, 12 Aug 2008, M. Edward (Ed) Borasky wrote:

Why does Debian use dash? I’ve never even heard of dash until I saw
this thread! Is there some religious war against bash?

I believe it’s sole claim / reason is it’s “fastest POSIX sh compatible
shell”

But as I tell people, the Lowest Common Denominator amongst the *sh is
sh, and it’s very very Low indeed.

“A mathematician is a machine for turning coffee into theorems.” –
Alfréd Rényi via Paul Erdős

I turn coffee into Ruby.

John C. Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : [email protected]
New Zealand

[amin1426]

On Tue, 12 Aug 2008, ara.t.howard wrote:

i could easily be wrong, but i have a dim recollection that the location of
sh is a POSIX standard itself.

backticks seem to respect the SHELL var here

cfp:~ > SHELL=tcsh ruby -e’ puts echo $SHELL 2>out ’
tcsh

At risk of being a little rude to one who doesn’t deserve it, I
couldn’t resist (purely for humorous reasons) framing my illustration
of the problem with that line like so…

I_AM_EASILY_WRONG=‘yes you are :-)’ ruby -e’ puts echo $I_AM_EASILY_WRONG 2>out’
yes you are

The location at /bin/sh is standard and fixed. SHELL is what you set
that users default shell to in your /etc/password.

In unix Ruby backtick ignores ENV[“SHELL”] and just runs whatever is at
/bin/sh

John C. Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : [email protected]
New Zealand

[amin1426]

On Aug 12, 2008, at 3:10 PM, John C. wrote:

At risk of being a little rude to one who doesn’t deserve it, I

In unix Ruby backtick ignores ENV[“SHELL”] and just runs whatever is
at /bin/sh

John C. Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : [email protected]
New Zealand

heh - right you are!

a @ http://codeforpeople.com/

[amin1426]

On Tue, 12 Aug 2008, Phlip wrote:

and learn the Ruby-internal way to do something, such as Pathname.glob.
There’s always a way to avoid a shell command…

There are two reasons I use ``.

1. Quick and easy. Much quicker to type than system " "

2. When I want the output of a program as a string, much easier to type
   blah than IO.read("|blah")

Whilst shell globbing has some shorthandedness about it, the main
reason is are file redirects. > &> >2&1

I’m quite happy to override `` with that trick someone mentioned above
if it grants me deterministic behaviour between client boxes. (I don’t
care if its the wrong behaviour, so long as it’s the same
behaviour!)

I often use Dir[“bla*”] and glob with glee.

Anyone have a short and easy Ruby way of doing file redirects?

John C. Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : [email protected]
New Zealand

[amin1426]

The location at /bin/sh is standard and fixed.

Could you point out where it is defined? I could understand if the FHS
wants to impose such locations, but someone claimed that POSIX in itself
makes that claim.

I would be delighted to hear more, because as of today I only thought
the only thing that imposes something is FHS (and glibc which relies on
a hardcoded path. For more info about this read NixOS design choices, i
think /bin/bash or something else is hardcoded because the glibc guys
mandated that)

[amin1426]

On 12.08.2008 22:55, John C. wrote:

Anyone have a short and easy Ruby way of doing file redirects?

Well, define it once and include it in all scripts where you need it.
For example:

robert@fussel /cygdrive/c/Temp
$ ./redir.rb
Done #<Process::Status: pid=5384,exited(0)>
Done #<Process::Status: pid=5464,exited(2)>

robert@fussel /cygdrive/c/Temp
$ cat output
drwx------+ 5 robert Kein 0 Aug 13 08:42 .

robert@fussel /cygdrive/c/Temp
$ cat output2
ls: cannot access does_not_exis: No such file or directory
drwx------+ 5 robert Kein 0 Aug 13 08:42 .

robert@fussel /cygdrive/c/Temp
$ cat redir.rb
#!/bin/env ruby

module Kernel
private
def system_redir(args, fout = ::STDOUT, ferr = ::STDERR, fin =
::STDIN)
child = fork do
redir_fd ::STDIN, fin, “r”
redir_fd ::STDOUT, fout, “w”
redir_fd ::STDERR, ferr, “w”
exec *args
end
Process.wait2(child).last
end

def redir_fd(io, target, mode = nil)
case target
when String
io.reopen(File.open(target, mode))
when IO
io.reopen target
when io
# do nothing
when nil
io.close
else
raise ArgumentError, “Invalid target argument #{target.inspect}”
end
end
end

res = system_redir %w{ls -ld}, “output”
printf “Done %p\n”, res

File.open “output2”, “w” do |io|
res = system_redir %w{ls -ld . does_not_exis}, io, io
printf “Done %p\n”, res
end

robert@fussel /cygdrive/c/Temp

Kind regards

robert

[amin1426]

On Aug 12, 2008, at 5:18 PM, Marc H. wrote:

Could you point out where it is defined? I could understand if the FHS
wants to impose such locations, but someone claimed that POSIX in
itself
makes that claim.

[ahoward@localhost ~]$ PAGER=cat man system
SYSTEM
(3)
Linux Programmer’s
Manual

SYSTEM(3)

NAME
system - execute a shell command

SYNOPSIS
#include <stdlib.h>

    int system(const char *string);

DESCRIPTION
system() executes a command specified in string by
calling /bin/sh -c string, and returns after the command has been
completed. During execution of the command,
SIGCHLD will be blocked, and SIGINT and SIGQUIT will be ignored.

RETURN VALUE
The value returned is -1 on error (e.g. fork failed), and the
return status of the command otherwise. This latter return status is
in the format specified in wait(2).
Thus, the exit code of the command will be
WEXITSTATUS(status). In case /bin/sh could not be executed, the exit
status will be that of a command that does exit(127).

    If the value of string is NULL, system() returns nonzero if

the shell is available, and zero if not.

    system() does not affect the wait status of any other children.

CONFORMING TO
ANSI C, POSIX.2, BSD 4.3

NOTES
As mentioned, system() ignores SIGINT and SIGQUIT. This may
make programs that call it from a loop uninterruptable, unless they
take care themselves to check the exit
status of the child. E.g.

        while(something) {
            int ret = system("foo");

            if (WIFSIGNALED(ret) &&
                (WTERMSIG(ret) == SIGINT || WTERMSIG(ret) ==

SIGQUIT))
break;
}

    Do not use system() from a program with suid or sgid

privileges, because strange values for some environment variables
might be used to subvert system integrity. Use
the exec(3) family of functions instead, but not execlp(3)
or execvp(3). system() will not, in fact, work properly from programs
with suid or sgid privileges on sys-
tems on which /bin/sh is bash version 2, since bash 2 drops
privileges on startup. (Debian uses a modified bash which does not do
this when invoked as sh.)

    The check for the availability of /bin/sh is not actually

performed; it is always assumed to be available. ISO C specifies the
check, but POSIX.2 specifies that the
return shall always be non-zero, since a system without the
shell is not conforming, and it is this that is implemented.

    It is possible for the shell command to return 127, so that

code is not a sure indication that the execve() call failed.

SEE ALSO
sh(1), signal(2), wait(2), exec(3)

C process control

a @ http://codeforpeople.com/