[BUG:trunk] encoding for stdio's

Ruby Development
1

Yuguie$B$G$9!#e(B

stdout, stderre$B$Ne(Bexternal
encodinge$B$,e(Bnile$B$J$N$O$3$&$$$&$b$N$G$7$?$C$1e(B?
default_externale$B$r4|BT$7$F$^$7$?!#e(B

e$B$=$l$+$i!“e(B-Eext:inte$B$7$?$H$-$K$O>/$J$/$H$be(Binternal_encodinge$B$KH?1G$5$l$?J}e(B
e$B$,;H$$$d$9$$$H;W$&$N$G$9$,!”$$$+$,$G$7$g$&$+!#e(B

% cat test.rb
p [STDIN, STDOUT, STDERR].map{|io|
[io.external_encoding, io.internal_encoding]
}

% ruby-trunk -v test.rb

ruby 1.9.1 (2008-12-22 p-5000 trunk-r20912) [i386-darwin9.6.0]
[[#Encoding:UTF-8, nil], [nil, nil], [nil, nil]]

% ruby-trunk -vEcp932 test.rb
ruby 1.9.1 (2008-12-22 p-5000 trunk-r20912) [i386-darwin9.6.0]
[[#Encoding:Windows-31J, nil], [nil, nil], [nil, nil]]

% ruby-trunk -vE:euc-jp test.rb
ruby 1.9.1 (2008-12-22 p-5000 trunk-r20912) [i386-darwin9.6.0]
[[#Encoding:UTF-8, nil], [nil, nil], [nil, nil]]

% ruby-trunk -vEcp932:euc-jp test.rb

ruby 1.9.1 (2008-12-22 p-5000 trunk-r20912) [i386-darwin9.6.0]
[[#Encoding:Windows-31J, nil], [nil, nil], [nil, nil]]

2

e$B$^$D$b$He(B e$B$f$-$R$m$G$9e(B

In message “Re: [ruby-dev:37576] [BUG:trunk] encoding for stdio’s”
on Wed, 24 Dec 2008 18:20:49 +0900, “Yugui (Yuki S.)”
[email protected] writes:

|stdout, stderre$B$Ne(Bexternal encodinge$B$,e(Bnile$B$J$N$O$3$&$$$&$b$N$G$7$?$C$1e(B?
|default_externale$B$r4|BT$7$F$^$7$?!#e(B
|
|e$B$=$l$+$i!“e(B-Eext:inte$B$7$?$H$-$K$O>/$J$/$H$be(Binternal_encodinge$B$KH?1G$5$l$?J}e(B
|e$B$,;H$$$d$9$$$H;W$&$N$G$9$,!”$$$+$,$G$7$g$&$+!#e(B

e$B<jMn$A$@$H;W$$$^$9!#@_Dj$5$l$k$Y$-$G$7$g$&!#e(B

3

On 12/29/08 6:22 PM, Yukihiro M. wrote:

e$B$^$D$b$He(B e$B$f$-$R$m$G$9e(B
|stdout, stderre$B$Ne(Bexternal encodinge$B$,e(Bnile$B$J$N$O$3$&$$$&$b$N$G$7$?$C$1e(B?
|default_externale$B$r4|BT$7$F$^$7$?!#e(B

e$B@.@%$5$s$+$i$O!Ve(Bstdout,
stderre$B$N=PNO$,<+F0JQ49$5$l$J$$$h$&$K!W$H$$$&0U?^e(B
e$B$@$C$?JV;v$rD:$-$^$7$?!#e(B
Rubye$B$H$7$F$I$A$i$,NI$$$b$N$G$7$g$&$+!#e(B

|e$B$=$l$+$i!“e(B-Eext:inte$B$7$?$H$-$K$O>/$J$/$H$be(Binternal_encodinge$B$KH?1G$5$l$?J}e(B
|e$B$,;H$$$d$9$$$H;W$&$N$G$9$,!”$$$+$,$G$7$g$&$+!#e(B

e$B<jMn$A$@$H;W$$$^$9!#@_Dj$5$l$k$Y$-$G$7$g$&!#e(B

internal_encodinge$B$O<jMn$A$@$H$O;W$$$^$9$,!"$H$O$$$(e(B$stdout,
$stderre$B$K$De(B
e$B$$$F$Oe(Bexternal_encodinge$B$,e(Bnile$B$N$^$^$@$H0UL#$,$J$$$G$9$M!#e(B

4

e$B$^$D$b$He(B e$B$f$-$R$m$G$9e(B

In message “Re: [ruby-dev:37653] Re: [BUG:trunk] encoding for stdio’s”
on Mon, 29 Dec 2008 18:35:15 +0900, “Yugui (Yuki S.)”
[email protected] writes:

|> |stdout, stderre$B$Ne(Bexternal encodinge$B$,e(Bnile$B$J$N$O$3$&$$$&$b$N$G$7$?$C$1e(B?
|> |default_externale$B$r4|BT$7$F$^$7$?!#e(B
|
|e$B@.@%$5$s$+$i$O!Ve(Bstdout, stderre$B$N=PNO$,<+F0JQ49$5$l$J$$$h$&$K!W$H$$$&0U?^e(B
|e$B$@$C$?JV;v$rD:$-$^$7$?!#e(B

default_internale$B$,F3F~$5$l$k$^$G$O!V<+F0JQ49!W$O$J$+$C$?$o$1e(B
e$B$G$9$7!"e(Bdefault_externale$B$,@_Dj$5$l$J$$M}M3$K$O$J$i$J$$$N$G$Oe(B
e$B$J$$$+$H;W$$$^$9!#e(B

|Rubye$B$H$7$F$I$A$i$,NI$$$b$N$G$7$g$&$+!#e(B

default_internale$B$rF3F~$7$?0J>e!"F~=PNO$K$D$$$F$O<+F0JQ49$O$de(B
e$B$`$J$7$G$O$J$$$+$H;W$$$^$9!#e(B

5

e$B@.@%$G$9!#e(B

Yukihiro M. wrote:

default_internale$B$,F3F~$5$l$k$^$G$O!V<+F0JQ49!W$O$J$+$C$?$o$1e(B
e$B$G$9$7!"e(Bdefault_externale$B$,@_Dj$5$l$J$$M}M3$K$O$J$i$J$$$N$G$Oe(B
e$B$J$$$+$H;W$$$^$9!#e(B

|Rubye$B$H$7$F$I$A$i$,NI$$$b$N$G$7$g$&$+!#e(B

default_internale$B$rF3F~$7$?0J>e!"F~=PNO$K$D$$$F$O<+F0JQ49$O$de(B
e$B$`$J$7$G$O$J$$$+$H;W$$$^$9!#e(B

default_internal
e$B$OF3F~$5$l$^$7$?$,!"<+F0JQ49$O%G%U%)%k%H$O%%U$K$J$C$F$$$^$9!#e(B
e$B$3$N$?$a!“F~NO%(%s%3!<%G%#%s%0$N<+F0JQ49$O%G%U%)%k%H$@$HF/$-$^$;$s!#e(B
e$B$3$N>uBV$G!”=PNO$@$1<+F0JQ49$r$9$k$H$$$&$N$O$$+$7$$$h$&$K;W$$$^$9!#e(B

e$B$^$?!"e(Bdefault_internal e$B$,e(B nil
e$B$G$b=PNO<+F0JQ49$r9T$&$H$7$?>l9g!“e(B
e$B<B:]LdBj$H$7$F<+F0JQ49$r$7$FM_$7$/$J$$>l9g$N;XDj$,:$Fq$K$J$k$H$$$&LdBj$b$”$j$^$9!#e(B

e$B$7$+$7!"0lJ}$G!“e(BIO#external_encoding
e$B$N5sF0$,4qL/$J$N$b3N$+$G!”$?$H$($P!“e(B
% ruby19 -e’open(“fuga.txt”,“r+”){|f|p f.external_encoding}’
nil
% ruby19 -e’open(“fuga.txt”,“r”){|f|p f.external_encoding}’
#Encoding:UTF-8
e$B$3$s$JLdBj$b$”$j!“e(Bexternal_encoding
e$B$O@_Dj$5$l$kI,MW$,$”$j$=$&$G$9!#e(B

e$B$D$^$k$H$3$m!"e(B

  • e$B8=:_$N<+F0JQ49<~$j$N5sF0$OJQ$($J$$e(B
  • e$B$G$be(B external_encoding e$B$O=PNO$G$b>o$K@_Dj$9$ke(B
    e$B$,BEEv$G$O$J$$$+$H;W$$$^$9!#e(B

e$B$3$3$G!"e(Bexternal_encoding e$B$,>o$K??$K$J$k$h$&$K$J$k$?$a!"e(B
e$B<+F0JQ49$NM-L5$N%U%i%0$r!"e(Bexternal_encoding
e$B$+$i2?$rMQ$$$k$h$&$K$9$k$+$,LdBj$G$9$,!"e(B
internal_encoding e$B$NM-L5$r8+$l$PBg>fIW$+$J$!$H;W$C$F$$$^$9!#e(B

6

e$B%A%1%C%He(B #924 e$B$,99?7$5$l$^$7$?!#e(B (by Takashi T.)

e$B$3$s$P$s$Oe(B sheepman e$B$G$9!#e(B

e$B7k6I!"e(B1.9.2 e$B$Ge(B stdio e$B$Ne(B encoding
e$B$O$I$&$J$ke(B/e$B$J$C$?$N$G$7$g$&$+!#e(B

http://redmine.ruby-lang.org/issues/show/924

7

e$B%A%1%C%He(B #924 e$B$,99?7$5$l$^$7$?!#e(B (by Yui NARUSE)

e$B$^$:!"8=:_$N$H$3$m$OJQ$o$C$F$$$^$;$s!#e(B

e$B:#8e$G$9$,!"e(B

  • e$B8=:_$N<+F0JQ49<~$j$N5sF0$OJQ$($J$$e(B
  • IO#external_encodinge$B$NLa$jCM$rJQ99$9$ke(B
    (e$BL@<(E*$K;XDj$5$l$F$$$J$$:]$K!"e(Bdefault_externale$B$rJV$7$F$$$k$N$r$d$a$ke(B)
    e$B$H$$$&<gD%$r@.@%$O;}$C$F$$$k$N$G!"8e$G%Q%C%A$r=P$7$^$9!#e(B

http://redmine.ruby-lang.org/issues/show/924

8

e$B%A%1%C%He(B #924 e$B$,99?7$5$l$^$7$?!#e(B (by Takashi T.)

e$B@.@%$5$s$,0JA0=q$+$l$?0J2<$NA0Ds$OJQ$o$C$?$H;W$&$N$G$9$,!"e(B
e$B8=:_$G$bI8=`F~=PNO$r<+F0JQ49$7$J$$M}M3$r$b$&>/$7>$7$/=q$$$F$$$?$@$1$J$$$G$7$g$&$+!#e(B

e$B$"$HI8=F~=PNO$r<+F0JQ49$7$J$$>l9g!"Hse(B ASCII e$BJ8;z$rI8==PNO$K=PNO$7$?$$;~$K$O!"e(B
e$B$I$N$h$&$K=q$/$N$,%Y%9%H%W%i%/%F%#%9$J$s$G$7$g$&$+!#e(B

default_internal e$B$OF3F~$5$l$^$7$?$,!"<+F0JQ49$O%G%U%)%k%H$O%%U$K$J$C$F$$$^$9!#e(B
e$B$3$N$?$a!“F~NO%(%s%3!<%G%#%s%0$N<+F0JQ49$O%G%U%)%k%H$@$HF/$-$^$;$s!#e(B
e$B$3$N>uBV$G!”=PNO$@$1<+F0JQ49$r$9$k$H$$$&$N$O$$+$7$$$h$&$K;W$$$^$9!#e(B

e$B:#$@$He(B default_internal
e$B$r;XDj$7$F$bI8=`F~NO$OJQ49$5$l$^$;$s$,!"%U%!%$%k$NF~NO$OJQ49$5$l$^$9!#e(B

$ echo $LANG
ja_JP.eucJP

$ nkf --guess t.txt
EUC-JP

$ cat e01.rb
p $stdin.read.encoding
p File.read(‘t.txt’).encoding

$ echo e$B4A;ze(B | ruby-1.9 -vE :utf-8 e01.rb
ruby 1.9.2dev (2009-07-24 trunk 24263) [i686-linux]
#Encoding:EUC-JP
#Encoding:UTF-8

http://redmine.ruby-lang.org/issues/show/924

9

e$B%A%1%C%He(B #924 e$B$,99?7$5$l$^$7$?!#e(B (by Yui NARUSE)

e$B%9%F!<%?%9e(B Opene$B$+$ie(BAssignede$B$KJQ99e(B

e$B$H$j$"$($:!"e(Br24337e$B$K$F0J2<$N$h$&$JJQ99$rF~$l$^$7$?!#e(B

Set encodings of stdio after setting default internal and external.

  • io.c (rb_stdio_set_default_encoding): added.

  • ruby.c (process_options): call rb_stdio_set_default_encoding
    after setting defualt internal and external.

http://redmine.ruby-lang.org/issues/show/924

10

e$B%A%1%C%He(B #924 e$B$,99?7$5$l$^$7$?!#e(B (by Yui NARUSE)

e$B%9%F!<%?%9e(B Assignede$B$+$ie(BClosede$B$KJQ99e(B

e$B%A%1%C%H$N%4!<%k$,L@3N$G$J$+$C$?$?$a!“ESCf$G%4!<%k$,DI2C$5$l$?$?$a$K;D$C$F$7$^$C$?$H$$$&46$8$G$9$+$M$'!#e(B
e$B<+F0JQ49$H$+!”%Q%C%A%o!<%/E*$J8=9T;EMM$N%I%-%e%a%s%H2=$H$+$H@0M}$H$+$b$“$j$^$9$,!”$^$!$=$l$OJL$N%A%1%C%H$H8@$&$3$H$G!#e(B

http://redmine.ruby-lang.org/issues/show/924

11

e$B%A%1%C%He(B #924 e$B$,99?7$5$l$^$7$?!#e(B (by Yusuke E.)

e$B@.@%$5$se(B
e$B1sF#$G$9!#e(B

e$B$3$N%A%1%C%H$O2?$N$?$a$K;D$C$F$$$k$N$G$7$g$&$+!#e(B
e$B$Q$C$H8+$G$o$+$k$h$&$K!";D$k8!F$;v9`!&LdBj$Ne(B summary
e$B$r=q$$$Fe(B
e$B$b$i$($k$H=u$+$j$^$9!#e(B

e$B$b$7;EMM%l%Y%k$N8!F$;v9`$,;D$k$@$1$J$i$P!"$9$G$K;EMM%U%j!<%:e(B
e$B$7$F$7$^$C$?$N$G!"e(Btarget e$B$re(B 1.9.x e$B$K$7$F$/$@$5$$!#e(B

–
Yusuke E. [email protected]

http://redmine.ruby-lang.org/issues/show/924