Rails core going multiligual...ever?

Julian,

A lot of plugins will rely on overriding Rails behavior. If it breaks,
we fix it, or it gets swallowed into core. Another option would be to
make the overrides configurable, as are the unicode normalization
routines currently (depending on whether the gem is installed).

Joshua Sierles

On 12 Nov 2005, at 17:09, Jamis B. wrote:

This really fascinates me. I can completely understand that you may
need i18n for your apps, and without i18n support, you can’t get
your work done. But why does i18n need to be in the core? (I’m not
saying it shouldn’t be, just wondering why it must be.)

I work on projects that are cross-EU and even within the UK several
of the big projects that I am lining up now have Welsh audiences. If
I haven’t got a clear strategy to handle multi-lingual work within a
particular framework, for those projects, that framework can’t go
near it.

This isn’t preference: on some projects I work on it is law. I can,
in theory, go to prison or have to repay a multi-million budget if I
ship code that isn’t multi-lingual-friendly.

Plus, I have a hard time thinking of a web application that would
need to be mono-lingual these days. I know everybody thinks English
is the lingua franca of the web, but in the same way you wouldn’t
dare ship a web app that is not time-zone aware, I can’t think how
anybody could think multi-lingualism isn’t a critical capability of
any code that ships these days. But that’s me.

The assumption, it seems, is that if something is in the core, it
will be just what you need. This is definitely not going to be the
case for everyone–as has been pointed out repeatedly, i18n has
many application-specific needs, and just because there may someday
be an implementation of i18n in core, that does not imply that it
may be the implementation you need. I see the proliferation of
3rd party i18n efforts as a good thing, in many ways.

Sure, lots of third party solutions is OK, and I’ve been looking at
those today - as I pointed out in my initial e-mail, I have mailed
Josh to send me his SVN URL so I can see if that hits the spot. What
I’m saying I suppose is that without i18n in there, or at least a
clear set of third-party solutions (which atm, there isn’t a clear
path for this set out, not that I can see anyway) there are many,
many, many projects for which Rails just couldn’t be considered.
That’s not a bad thing or a good thing: it’s just a fact.

I don’t care if it’s in core of it’s in a plugin. I just want it to
exist and for the solution to the problem to be well understood. For
me, it makes sense for i18n to be in core, but I’m just one of the
5.7 billion people on the planet that doesn’t live in the USA. :slight_smile:

May I politely suggest that people stop complaining about the
lack of i18n in Rails, and please do something about it. Use an
existing solution. Join an existing effort to build something else.
Or even build something else yourself.

I’ve already explained that whilst I haven’t looked in a few months
because I’ve been busy on other projects, I’ve started looking at
this again. I don’t have the resources to throw at this right now,
but I’m not saying “do this for me”, I’m saying “this needs to be
thought about, otherwise people might not take the framework
seriously and that would be dumb”, that’s all.

Some members of the core team have put forward i18n solutions that
work for them. Other rails users have proposed several other
solutions, either complete or in-progress. If they don’t work for
you, put together something that does. Once we have a few really
good solutions that have been tested in the field, we can start to
decide what kind of solution ought to reside in core, if any. (The
key here is “tested in the field.”)

Yup, understood. I think for me the key is to move everything - error
messages, language in template code, everything - out into an
industry-standard i18n format.

done as much as we’re going to at this point, I think. The ball is
in your court now (where “you” refers to “anyone needing an i18n
solution”.)

I think you’re confusing me with somebody going around apportioning
blame. I’m not. I’m telling you that for Rails to truly be taken as
seriously as some of the Java (or even Python) frameworks out there
the core team should consider thinking about i18n and working out a
way of being able to make it as easy as the rest of Rails, perhaps
based on an existing solution, and that if I were in your position I
wouldn’t ship 1.0 until it had been thought about. Shipping 1.0 isn’t
a job I’ve signed up to, so my opinion counts for nothing. :slight_smile:

However, what strikes me as strange about your response is that you
appear to be suggesting we should just shut up about it. All I was
doing was adding to the discussion, and your response has a hint of
hostility about it. Perhaps I’m reading it wrong, but it doesn’t seem
like a response that is about opening up the debate. RoR aint a
democracy, sure, but I find it odd that the first response to my
first mail to the list is basically a long way of telling me to shut
up… if I’ve misread you, my apologies in advance.

Anyway, I shall go and dig through the i18n solutions already out
there and see what comes of it…


Paul R. - http://iconoplex.com
“Cogito, ergo sum.” – Descartes

Jamis,

I think part of the problem is documentation. If I want i18n in my Rails
application I have options

  • integrate i18n in my app - not easily portable or updateable
  • write an plugin - where is the documentation/HowTos/tutorials about
    writing plugins and how they link to the core? (I don’t mean that as a
    demand. It is just a question. I know plugins are new.)
  • write an engine - recently discouraged approach, complicated. In a
    way,
    discouraging is as good as forbiding or might have that affect when
    people
    genuinely want to do things “the rails way” or “David’s way”.

So what do I do? I say “Hey, this is an issue that 6 billion people can
benefit from solving well. Perhaps this is a core issue.”

I definitely take your point that if the core developers do not need
i18n
then they should not develop this aspect of Rails. But pehaps the core
devs
will be open to cooperative discussions identifying every topic of i18n
that
must be addressed in a solution. And then discussing ideas for how to
start
moving the core towards i18n in a piecewise way.

I’m not looking for the core to do the work. But some guidence on
implementing a solution to this important issue would be very helpful.

Peter

±Le 12/11/2005 10:09 -0700, Jamis B. a dit :
| This really fascinates me. I can completely understand that you may need
| i18n for your apps, and without i18n support, you can’t get your work
| done. But why does i18n need to be in the core? (I’m not saying it
| shouldn’t be, just wondering why it must be.)

I totally agree with you, I feel that it does not need to be in core, I
have a framework working pretty nicely here, working with gettext::mo
just
to store things.

It’s composed of an RR/lang/extract.rb http://p.mat.cc/C which gets
the
hell out of the app file, a RR/lang/Makefile http://p.mat.cc/A which
automates the updating task, a RR/lib/i18n.rb http://p.mat.cc/D which
is
the whole framework, a RR/config/environments/i18n_env.rb
http://p.mat.cc/E which sets defaults, I have :
require ‘environments/i18n_env’
require ‘i18n’
I18N::load_localized_strings

in my RR/config/environment.rb

and to finish, an “include I18N” in my
RR/app/helpers/application_helper.rb
and RR/app/controllers/application.rb

whenever I need some i18n string, I just say :
l(:levels)
or
l(‘Error %s : %s’, e.class, e.to_str)

when I have to add some language, I just touch lang/xx.po and run make
in
the lang directory, it’ll fill it with empty strings and it’ll rock.

(maybe I will be punished with the task to put that in the rails wiki)

On 11/12/05, Jamis B. [email protected] wrote:

I strongly advise against talking this thing to death. Yes,
discussion is helpful, and often necessary, but don’t let discussion
take the place of experimentation.

From there, development is an iterative/evolutionary process

that builds on each previous cycle.

I agree! This is exactly why I asked if the core devs are open allowing
i18n
solutions to creep into the core over time. The problem being that if I
come
up with a great solution to i18n that requires the core to keep track of
a
locale or culture or something similar, but the core devs won’t hear
anything of it, then was my work was in vain?

I found that the symphony framework has already imbraced some i18n into
the
core. That at least indicates some other frameworks do think i18n is a
core
issue that needs to be incorporated in today’s web development world.

http://www.symfony-project.com/content/book/page/i18n.html

Some of these ideas are really great.

  • Peter

On Nov 12, 2005, at 12:08 PM, Peter M. wrote:

Jamis,

I think part of the problem is documentation. If I want i18n in my
Rails application I have options

  • integrate i18n in my app - not easily portable or updateable
  • write an plugin - where is the documentation/HowTos/tutorials
    about writing plugins and how they link to the core? (I don’t mean
    that as a demand. It is just a question. I know plugins are new.)

You’re dead on–we need better docs here. And there will be some.

I definitely take your point that if the core developers do not
need i18n then they should not develop this aspect of Rails. But
pehaps the core devs will be open to cooperative discussions
identifying every topic of i18n that must be addressed in a
solution. And then discussing ideas for how to start moving the
core towards i18n in a piecewise way.

I strongly advise against talking this thing to death. Yes,
discussion is helpful, and often necessary, but don’t let discussion
take the place of experimentation. (FWIW, there is a thread currently
on the rails-core mailing list about i18n, and anyone that has a
stake in i18n is encouraged to follow, if not participate in, that
thread.)

For many years I labored under the delusion that it was possible to
specify something completely, up front, simply by thinking and
talking about it. I’ve since been disabused of that notion. As Truly
Scrumptious said to Caractacus Potts in Chitty-Chitty-Bang-Bang,
“It’s a beautiful dream, but I don’t see how it’s going to help…”
Sure, you want to have some idea of what you need to implement, but
that can be as simple as a few HTML mockups of a real application
(“real” is key) demonstrating where and how internationalized text
ought to be used. In my experience, that’s as much as you need, up-
front. From there, development is an iterative/evolutionary process
that builds on each previous cycle.

It’s called “Getting Real” [1].

  • Jamis

[1] http://getreal.37signals.com

On 12 Nov 2005, at 20:19, Peter M. wrote:

I found that the symphony framework has already imbraced some i18n
into the core. That at least indicates some other frameworks do
think i18n is a core issue that needs to be incorporated in today’s
web development world.

http://www.symfony-project.com/content/book/page/i18n.html

Some of these ideas are really great.

What on the symfony page? I’d prefer ‘ugly’ over ‘great’ :-), but
yes, the principles are interesting. Very interesting, but I think it
can be made a little more elegant if it is deserving of the title of
going into Rails… One tactic might be to put something into rake to
parse the templates in the views directory to find strings and
create .fr.html and .de.html and so on based on the text strings
found. Another tactic might be to figure a way of separating out the
content of a view in a text sense from the layout of a view, but in a
way that makes sense in a Rails-style.

I agree this is a big area to bite off and chew. We might need to
start collecting ideas on a wiki, although today I’ve seen three or
four approaches that all look like they’re half-reasonable, but not
in a way I would advocate going into the core of Rails because there
would be too much lifting going on for everybody else. That’s the
main reason why it would be nice if core said ‘no way chaps, keep it
separate’ in which case we already have a bunch of solutions mostly
worked out.

±Le 12/11/2005 20:23 +0000, Paul R. a dit :
| On 12 Nov 2005, at 19:45, Mathieu A. wrote:
|
|> (maybe I will be punished with the task to put that in the rails wiki)
|
| Yes, that might be one work-around. It doesn’t seem very Rails-ish, but
| it’s a solution that will work well for most cases. It just makes
| development a pain in the proverbial which kind of negates the point of
| using Rails.

Well, I don’t find it to be particularely a pita, but I’m used to it :slight_smile:

| I think what we need for it to be a Rails solution is that Rails somehow
| just “looks after it” for you - convention over configuration, etc.
| which is why I figured it was a core issue. I appear to be in a minority
| though, so I think I’ll end up aping your solution. If you don’t whack
| it into the wiki, I’m sure I will in a day or two when I’ve played with
| it some more.

Well, in i18n, you have to extract what has to be translated one way or
the
other and merge it with what’s already translated, and that’s a pita :slight_smile:
You also have to have it translated one way or the other. I used po/mo
files because there is already nice and powerful utilities to deal with
them. (and I used a Makefile and not a Rakefile because it’s something
ported from a perl/mason app).

El Sábado 12 Noviembre 2005 20:08, Peter M. escribió:

discouraging is as good as forbiding or might have that affect when people
genuinely want to do things “the rails way” or “David’s way”.

So what do I do? I say “Hey, this is an issue that 6 billion people can
benefit from solving well. Perhaps this is a core issue.”

I definitely take your point that if the core developers do not need i18n
then they should not develop this aspect of Rails. But pehaps the core devs
will be open to cooperative discussions identifying every topic of i18n
that must be addressed in a solution. And then discussing ideas for how to
start moving the core towards i18n in a piecewise way.

Well, I posted a mail with subject “[RFC] Multi-language rails” almost
1
month ago and got no comments.

I took some notes at
http://wiki.rubyonrails.com/rails/pages/RailsLocalization
and I’m planning to do some more work when I get some more free time :slight_smile:

I’m not looking for the core to do the work. But some guidence on
implementing a solution to this important issue would be very helpful.

Peter


Jorge B. [email protected]
Warp Networks S.L.
http://www.warp.es/
[EN] http://koke.amedias.org/
[ES] http://www.amedias.org/koke

On 12 Nov 2005, at 19:45, Mathieu A. wrote:

(maybe I will be punished with the task to put that in the rails wiki)

Yes, that might be one work-around. It doesn’t seem very Rails-ish,
but it’s a solution that will work well for most cases. It just makes
development a pain in the proverbial which kind of negates the point
of using Rails.

I think what we need for it to be a Rails solution is that Rails
somehow just “looks after it” for you - convention over
configuration, etc. which is why I figured it was a core issue. I
appear to be in a minority though, so I think I’ll end up aping your
solution. If you don’t whack it into the wiki, I’m sure I will in a
day or two when I’ve played with it some more.


Paul R. - http://iconoplex.com
“Cogito, ergo sum.” – Descartes

On 11/12/05, Peter M. [email protected] wrote:

  • write an engine - recently discouraged approach, complicated. In a way,
    discouraging is as good as forbiding or might have that affect when people
    genuinely want to do things “the rails way” or “David’s way”.

Incidentally, an engine really wouldn’t be a suitable way of doing
i18n - engines are (or should be) focused and relatively
self-contained. “Vertical application slices” is a phrase I throw
about sometimes. I18n is far more “horizontal” (i.e. across the whole
application like a layer of peanut butter). Engines (and other
focussed plugins/components) might want to support i18n, but not
provide it. That’s the job of something far more application-wide
(or, as it seems to be moving towards, in the core itself).

  • james

There’s been a lot of talk about login systems and components/engines in
the last few weeks, so I though I’d bring up a related topic: single
sign-on.

David Heinemeier H. wrote:

Let’s not mix these together. I agree that a i18n definitely stands a
shot at making it into the Rails core, but authentication absolutely
does not. There’s a fine line between infrastructure and business
logic. i18n sits on the infrastructure side, authentication on the
business logic side.

While I don’t want rails to contain a ton of authentication code, I
would like it to provide a standard mechanism or convention that allows
different components/engines/applications to share their userbase. It
happens all the time: you build a website out of various public packages
(e.g. blog, cms and forum) and then you would like them to use the same
login system so your users are not annoyed when moving from one part of
the site to the other.

I’m not exactly sure how to do that while staying clearly on the
infrastructure side, but I would like something really simple and
lightweight along the lines of:

| class WeblogController < ActionController::Base
| before_filter :authenticate
| def authenticate
| @user = single_sign_on_authentication(WeblogUser)

With single_sign_on_authentication taking care of any magic required to
tie disparate login systems together.

Any thoughts?

This is a bit off topic but…

I’ve been thinking a lot about DHH’s recent comments about engines being
a
sideshow to the Rails main stage[1]. I wonder if Rails is really meant
to do
what you, others and I are hoping it can do without a mess: single
login/multiple apps. I don’t know for sure and could easily be wrong but
I
think DHH is in a position unlike many other Rails web developers. Seems
like he has the luxury of being an idealist: all parts of an application
are
integrated rather than assembled from parts/engines (forum app, cms app,
blog app, etc). I think maybe for him T is closer to zero [2] than it is
for
other developers with less money/time to get a complete application up
and
maintained for, say, a non-profit organization. A developer may want to
make
a few such websites a month. But DHH has only a few main apps (Backpack
etc)
that need maintaining so the work may not seem as time consuming and so
Rails stays single app minded.

So maybe doing what you want to do is tough in the Rails world since
these
assembled apps are not what Rails was originally written for.

  • Peter

[1] Peak Obsession
[2] I don’t know where I read about “T approaches zero” but it meant
that by
the time the application is specified the application is already
finished
being written. Certainly we are not there yet but Rails is closer than
anything else I’ve tried.

Being a long time SSO hater I feel I must weigh in on this. I’ve gone
through the SSO wars on the Java/Portal/WebApp side of things and even
wrote a few little libraries to handle situations where SSO doesn’t help
(search for CryptoWallet). I’ve also used Yale’s CAS system and
investigated other systems. My comments below are entirely my own
though.

For the purpose of this discussion I’ll put things into porn terms since
that’s something we can all understand. See, if I use banks people get
that glazed look when I ask the hard questions. When I talk about porn
though, they listen. You’ll see in a minute. :slight_smile:

The first thing is that when people say “SSO” they generally mean
something entirely different depending on if they are talking about from
the user’s perspective or from the implementer’s perspective. From a
user’s perspective they mean, “Dude logs in to porntastic.com and can
access all 10 of his porn sites without using his other hand to type
another password.” That’s simple enough.

From an implementer’s perspective it gets ugly. Generally, implementers
look at the problem from the user’s perspective, “Dude logs into any
site
on the porntastic.com domain and we BLANK his credentials to the other
sites.” Ok, what is BLANK? Proxied credentials? Passwords from a
database? URL redirect bounces? SAML credentials? CAS requests off
cookies? How do you keep the passwords synced? What if you don’t own
all
the sites? What if the other sites don’t want to play? Huh smart guy?
Huh?!

At some point, BLANK becomes a problem. This is the devilish detail of
SSO that nobody mentions in their product literature:

a) You’ll have to rewrite all of these apps to use foreign and
usually ugly ass technology that never runs right on your flavor of
OS/Language.
b) You’ll need an already consolidated identity management system
such
as LDAP for all the users. SSO and login silos don’t mix.
c) You’ll have to evaluate the impact of SSO for each application.
There are other people in charge of these applications, and they may not
like the idea of letting people log into other apps to get to their’s.
It adds one more attack vector to deal with.
d) It’s a pain in the ass. No solution could be done with just
“single_sign_on_authentication” no matter how slick Rails is. The SSO
tech out there just isn’t that easy to use, and if it is it’s probably
not secure.
e) Beware of people who try to tell you it’s easy. They are liars
trying to sell you something or people who just don’t understand the
problem. You are basically allowing one password to access all your
applications. That’s damn scary.

If you own all the sites and have total control over the authentication
then you have a hope of doing it. Never mind that you’ll have to
retrofit
a completely foreign authentication system onto all of these (probably)
legacy applications. If you’re in this situation, then check Yale’s CAS
and look at OctetString’s Virtual Directory stuff (just bought by
Oracle).
[Disclaimer: I know someone who works there, but it is a good
product.]

If you don’t own the sites, then there’s probably no way you can do
this,
and I’ll use my porn punchline to illustrate the reason why.

Imagine that porntastic.com has 10 sites on SSO because they own all the
sites. Now they want to offer a partnership with getsome.com so that
people can head on over to getsome.com after using a porntastic.com
site.
Here’s why this is stupid:

  1. getsome.com now has to trust you to authenticate users to its site.
    If the deal is bidirectional (that’s DIRECTIONAL dude, read it right)
    then
    you have to trust getsome.com.
  2. The owners of getsome.com now get to see your users yanking habits.
    Hopefully your users don’t find out.
  3. If the users do find out, you have to invalidate them in two
    locations
    at the same time.
  4. You don’t control the getsome.com network, so how do you know they
    aren’t harvesting the passwords? Yes, this is really easy with some
    simple web page changes and pretty hard for you to detect.

In the end, this never happens unless one company buys the other, and
even
then it’s usually too much of a pain to do it. It’s just too risky to
trust another application to not have a security hole.

What this finally comes down to is there’s various strategies for SSO
which makes it much more complex and way outside the realm of Rails.
DHH
is right, it’s not Rails’ job, it’s your job because every SSO
initiative
is just like porn: it’s private and it’s personal.

And that’s just my 0.02. If you’re thinking of doing this, then check
out
CAS and kick the ruby-ldap guys in the ass so that there’s a win32
build.
You’ll never get any attention from serious LDAP vendors if win32 isn’t
supported easily as maybe a rubygem (because yes, occaisionally a
windows
computer has to login using LDAP too).

Zed A. Shaw

There were a lot of very good and true points about SSO in that last
post but mostly they’re about shared trust between sites. That’s not a
vital component of SSO and, frankly, I don’t want to touch it with a
forty-foot pole. I usually view authentication as composed of:

  1. identification: who are you? Bob / unknown
  2. authorization: do you have access? yes / no

It may seem like a trivial thing to point out but it’s important because
I agree that authorization is firmly in the realm of business logic.
Whether someone can be granted access or not is something only the
application can decide. If the app wants to talk to another system and
figure out if the user is trusted, that’s fine, but it doesn’t have
anything to do with the infrastructure.

As for identification, it’s a trivial thing to implement so it’s usually
bundled together with the authorization. But as it was so eloquently
pointed out by Zed, trying to retrofit SSO into an existing app is not
exactly funny. That’s why it has to be part of the infrastructure right
from the start.

rails is all about convention over configuration, right? If there was a
simple convention for authentication, then anyone could use that as a
basis for SSO later on. Unless you want a super-powerful SSO solution
that does everything and the kitchen sink, it is as simple as
“single_sign_on_authentication”. Let’s say that we have something along
the lines of:

| class ActionController::Base
| def authenticate(user_class, username, password)
| session[:rails_user] = user_class.authenticate(username, password)
| end
| def authenticated_user
| session[:rails_user]
| end

This is beyond simplistic; it’s a mere wrapper for the real method doing
the work. But a similar approach built into the infrastructure would
allow an SSO plugin to come in, override those standard methods, and
provide SSO that is gasp not a pain in the ass! My whole point is that
if rails doesn’t leave the door unlocked for SSO, no one can open it.

dd

allow an SSO plugin to come in, override those standard methods, and
provide SSO that is gasp not a pain in the ass! My whole point is that
if rails doesn’t leave the door unlocked for SSO, no one can open it.

If you’ll notice, most of the login generators/plugins/engines derive
somehow from xal’s login generator. Therefore, they all (for the most
part):

store the data (usually the user model which is bad IMO, but anyhow)
in session[:user].

use current_user as a comment method for retrieving the logged in
user.

use login_required as the before filter.

authenticate with a User.authenticate method.

So, there’s already a convention being used. I don’t see why the
framework has ‘support’ anything.

I’ve been able to use these conventions to implement a crude
single-sign on for i2, rforum and typo, three apps with completely
different authentication schemes. I didn’t implement any way to move
freely among the apps, but I could log in with the same
username/password.


rick
http://techno-weenie.net

A new rails i18n mailing list has been created. Please read:
http://www.ruby-forum.com/topic/65659

My .001 . CAS is great (if you have a common auth infrastructure in
place already) and you can use it from rails easy. See http://
wiki.case.edu/Central_Authentication_Service#Ruby_.
2F_Ruby_on_Rails . I converted over a site using the login
generator to this in a hour or so . Also this seems to be more
through https://clearinghouse.ja-sig.org/wiki/display/CAS/Ruby+on
+Rails+CAS+Client .