Microrant on Ruy's Math Skills

Ruby
61

On Thu, Jan 26, 2012 at 7:33 PM, Intransition [email protected]
wrote:

Seriously? You blame the example? Reminds me of that old joke.

Patient: “Doctor, it hurts when I do this.”
Doctor: “Well, don’t do that!”

The problem is you’re looking at in only as an engineer might, dealing with
very small deltas. But as animplementerof such a method as #close_to? or
#approx? I would never assume there are no valid applications that are
concerned with bulkier scales.

We’re talking at cross purposes here, and Yossef’s impoliteness
doesn’t help. Implementing #close_to? or #approx? is trivial, and
yes, it needs to provide for different applications. But talking
about Float#approx?, you may as well be talking about String#approx?,
as in “foobar”.approx?(“foobaz”).

But that has nothing to do with this thread. (Of course, it’s a long
thread and can take in different aspects of the topic, but…) You
started the thread complaining that 1.1 - 1.0 was not equal to 0.1,
that Ruby lacked “math skills”.

The answer to that is: don’t use Float.==. It’s a revealing answer,
because the natural follow-up question is “Why does the language
provide a useless method but not provide a useful replacement?”. When
it comes to floats, everyone has to be aware of their shortcomings and
roll their own comparison method.

If your application demands you use some implementation of #approx?
with a specific delta – e.g. collision detection in a game – then
we’re talking about an application-specific problem that is NOT caused
by the underlying representation of floats.

If your application would like to use “x == 4.75” but can’t, this is
NOT an application-specific problem; it’s a float representation
problem.

“x == 4.75” needs to be replaced by something. Should it be
“x.approx?(4.75)” ? No. That’s using an application-layer approach
to a non-application-layer problem.

“x == 4.75” should be replaced with x.essentially_equals?(4.75), which
tackles the representation problem, not the non-existent application
problem.

Mixing up these ideas makes for a confusing and (unfortunately)
potentially impolite discussion.

The problem is, Ruby doesn’t implement Float#essentially_equals?.
(Yes, a better name would be good.) If it did, there would be an
argument for simply defining Float.== in that way. The current
definition of Float.== is insufficient for its task, and simply
reflects Ruby’s C heritage. I wonder if anybody would be put out if
it were redefined to be more useful and less surprising.

Gavin

62

Taking a step back, I find myself a bit confused. Trying to best
implement
close?/approx? methods I start withe the premise:

|a - b| <= x

Where a and b are the two values being compared and x is the delta or
epsilon. So I am wonder how you came up with your ratio formula. If we
divide both sides by b then:

|a/b - b/b| <= x/b

|a/b - 1| <= x/b

In your formula it is the same except x is not divided by b. What am I
missing?

63

On Thu, Jan 26, 2012 at 2:58 PM, Gavin S. [email protected]
wrote:

The answer to that is: don’t use Float.==. It’s a revealing answer,
because the natural follow-up question is “Why does the language
provide a useless method but not provide a useful replacement?”. When
it comes to floats, everyone has to be aware of their shortcomings and
roll their own comparison method.

+1

If your application demands you use some implementation of #approx?
with a specific delta – e.g. collision detection in a game – then
we’re talking about an application-specific problem that is NOT caused
by the underlying representation of floats.

If your application would like to use “x == 4.75” but can’t, this is
NOT an application-specific problem; it’s a float representation
problem.

But doesn’t this boil down to the question: what algorithm do we want
to use to determine both as equivalent? We know that x is in binary
representation and 4.75 is a decimal representation of a number. But
every literal is converted to binary representation, likely by the
parser or other stage before runtime evaluation. And for reasons of
simplicity, speed and consistency that translation will always be the
same - regardless whether the literal appears in a comparison,
assignment or other expression. So in “x == 4.75” both values have
the same representation. And all comparisons which do not directly
compare the binary representation must apply some algorithm. That’s
the same problem solved by #approx?: input two floats, output true /
false. All of these algorithms I know need a third input (even if it
is hard coded into the algorithm).

“x == 4.75” needs to be replaced by something. Should it be
“x.approx?(4.75)” ? No. That’s using an application-layer approach
to a non-application-layer problem.

“x == 4.75” should be replaced with x.essentially_equals?(4.75), which
tackles the representation problem, not the non-existent application
problem.

There is no common definition of Float#essentially_equals? hence it
cannot be implemented in core library. The reason is that these
questions can only be answered application specifically:

  • What algorithm do we want to use to determine “essentially equality”
    (difference, factor, decimal representation)?
  • What value for the difference or factor do we use? Put informally:
    what is the level of imprecision that we accept to call two floats
    “essentially equal”?

With decimal representation I mean something like this:

class Float
def essentially_equals?(str)
a, b = to_s, str.to_s
a.start_with? b || b.start_with? a
end
end

Mixing up these ideas makes for a confusing and (unfortunately)
potentially impolite discussion.

The problem is, Ruby doesn’t implement Float#essentially_equals?.
(Yes, a better name would be good.) If it did, there would be an
argument for simply defining Float.== in that way. The current
definition of Float.== is insufficient for its task, and simply
reflects Ruby’s C heritage. I wonder if anybody would be put out if
it were redefined to be more useful and less surprising.

For the reasons above there is also no common implementation possible
other than what == does today. We would need at least one additional
input (the level of imprecision) to perform this calculation. A
binary operator always has only two operands so the third value could
only be provided by the environment (constant or variable in Float,
thread local variable). This is error prone since two parts in an
application might need totally different values.

So basically == is the way it is because every object must respond to
this method and there is no better implementation available which
works for the common case.

Kind regards

robert

64

On Thu, Jan 26, 2012 at 15:15, Intransition [email protected] wrote:

|a/b - 1| <= x/b

In your formula it is the same except x is not divided by b. What am I
missing?

a, b, and x are fixed in this case, so let y = x/b, and you have a new
tolerance derived from the old tolerance. That may mean you want to have
a
ratio-based comparison have a smaller delta than a difference-based
comparison, but there’s nothing much else going on.

65

Ah, never mind. You are doing the relative difference, where I was doing
the absolute difference.

66

Gavin S. wrote in post #1042645:

On Thu, Jan 26, 2012 at 7:33 PM, Intransition [email protected]
wrote:
The current
definition of Float.== is insufficient for its task, and simply
reflects Ruby’s C heritage. I wonder if anybody would be put out if
it were redefined to be more useful and less surprising.

Gavin

Well… You lost me here.

Why not just say we don’t compare floats for equality because the
representation of floats in binary is only an approximation. This is
the behavior expected by the computer scientist. It is not a language
specific issue.

This whole thread is based on a faulty premise.

For those who have not had the requisite training try it yourself.

Define a simple floating point format. Super simple. Let’s call it
“8.8”.

8 bits of integer part: 0-255 can be represented. (ignore
negatives/two’s-complement, for this simplistic example).
8 bits of fraction: 1/256ths.

Here is the binary representation for the base 10 float 1.25: 0000 0001
0100 0000
The fractional part is 64. 64/256 = 1/4 = 0.25

Second example. 1.1 in binary: 0000 0001 0001 1001
The fractional part adds up to 25 so this is actually 25/256. Our other
choice would have been 26/256. Neither of these choices are equal to
1/10. They are just an approximation.

Bonus homework: Write pseudo code to display the ascii, human readable,
representation of 8.8

67

On Thu, Jan 26, 2012 at 3:33 AM, Intransition [email protected]
wrote:

The problem is you’re looking at in only as an engineer might, dealing
with very small deltas. But as an implementer of such a method as
#close_to? or #approx? I would never assume there are no valid applications
that are concerned with bulkier scales.

I remember reading a complaint about how programmers think of all
constraint numbers as variables where pretty much any value is valid.
Why
write something that works for an array of 10 elements? It should work
for
ANY array. The array could have 1000 elements. Or 301039384!

…or maybe you really should consider the real problem space and see
that
the array will only ever have up to 10 elements, tops.

There’s an excellent example of this thinking in The Cryptonomicon
(which I
recently re-read) with some characters wondering why the “Pontifex”
algorithm specifically mentioned the number 54 instead of “X, where X
could
be 29, or 54, or 128”. As it turns out, there’s a very good reason for
the
54.

As Gavin clearly pointed out, there are two different concerns at play
here. One is the standard problem with IEEE floating-point
representation.
The other is deciding whether one number is “close enough” to another.
One
of these could be the concern of a programming language, while the other
would be the concern of a particular program.

68

(1) I love this thread! Though every time I open it, I think some poor
guy named Ruy needs a math tutor…

(2) The amount of emotion and defensiveness around this issue is
staggering. It is the very definition of a religious argument. On one
side are people who think “math should work like I learned in grade
school,” on the other are people who think “math does work like I
learned in college” (and the subset of those saying, “if you haven’t
had my level of training then you are stupid”), and then there a bunch
of people in the middle saying, “you’re both right, sort of!”

(3) I spend a lot of time teaching people programming, not writing
statistical algorithms, and I’ve written a unit testing library that
aims to be as unobtrusive as possible, so my main concern is to
minimize surprise while not undermining correctness. So naming my
method “close_to?” allows me to dodge the religious question “what is
equality?” and focus on getting noobies to understand just enough
complicated math to get their tests to pass, which basically amounts
to the lesson “floats are weird sometimes”.

I do think it’s important that Ruby Float == keeps working exactly
like C == and JavaScript == (kinda) and Python == though.

Or, what Yossef and Gavin said: “there are two different concerns at
play
here. One is the standard problem with IEEE floating-point
representation.
The other is deciding whether one number is “close enough” to another.
One
of these could be the concern of a programming language, while the other
would be the concern of a particular program.”

(4) Gavin, I did follow your message to the end, and the code helped
me really get your point. I also like Whitestone (I mean, “I Lk W”)
and I might steal some of your comparators. Allow me to return the
favor by teaching you that Github supports line numbers in its URLs,
e.g.

(5) How tragic is it that C broke a mathematical notation that had
worked just fine for hundreds of years and chose to define “=” as
assignment instead of equality? Amirite? Who’s with me? :slight_smile:

69

Hi all,

Such a large discussion from a simple initial post that was, in all
likelihood, an attempt to troll for as many responses as possible. It
looks like it has been wildly successful.

Perhaps a response like this would have been appropriate:

“Floats in Ruby are represented using the floating point numbers
provided by the underlying system, which are generally base 2 floating
point numbers, as per IEEE 754. They have advantages and disadvantages
that are worth being aware of, including loss of precision for certain
operations and values. This means that you cannot rely on testing for
equality in certain cases, and it may be more appropriate to check to
see if the result is within a certain epsilon. If the disadvantages are
not suitable for your problem domain or use case, you will have to use a
different numeric representation system more suited to your problem. All
representation systems have their own strengths and weaknesses.
BigDecimal may be more suitable than float in this case. A web search
for ‘floating point representation’ will provided much additional useful
information.”

In any form of software development you need to be aware of the
capabilities and limitations of the systems that you use, and in many
cases make an informed decision based on a set of potential systems. To
remain wilfully ignorant of these systems, and then cry “pathetic”, is
disingenuous. To subsequently push for language changes based on such a
shaky foundation is ludicrous. There are far better ways to stimulate
genuine discussion on an genuinely interesting topic than through a
deliberately inflammatory initial posting and comparable followups.

IMHO.

Garth

On 22/01/12 00:36, Intransition wrote:

So simple…

1.1 - 1.to_f == 0.1

false

(rumble) (rumble) Pathetic!

70

1.1 - 1.to_f == 0.1

false

(rumble) (rumble) Pathetic!

Agree.

At least with 1.9.2 it’s possible to tell more easily why this occurs:

1.1-1
=> 0.10000000000000009

0.1
=> 0.1

So some progress has been made.
But I’m with you I’d prefer the default to be rational or bigdecimal,
and you have to use
1.1f

for floats. Then you still have speed available, but not lose precision
in the default case. But that’s just me, and apparently it’s too big of
a change for Matz to swallow I suppose…
-roger-

71

On Thu, Jan 26, 2012 at 6:05 PM, Garthy D <
[email protected]> wrote:

by the underlying system, which are generally base 2 floating point

Garth

I don’t think it was intended to troll, and Trans definitely understands
IEEE 754 floats. He just thinks Ruby should move beyond them.

Personally, I rarely ever use them, because of these issues. They are
unsuitable for more uses than they are suitable for, and they contradict
the idea that abstractions shouldn’t leak implementation. OTOH, they’re
fast and ubiquitous. So I’m not very opinionated either way, but I think
it’s a discussion worth having.

72

On Fri, Jan 27, 2012 at 11:05 AM, Garthy D
[email protected] wrote:

To remain wilfully
ignorant of these systems, and then cry “pathetic”, is disingenuous. To
subsequently push for language changes based on such a shaky foundation is
ludicrous.

Regardless of the problems and advantages of the underlying
representation, and even in full knowledge of it, it is not pathetic
to think a modern language, one that seeks to please humans before
computers, should be able to recognise the truth of (1.1 - 1.0 ==
0.1). I do agree that wilful ignorance is probably never a good
thing, but I give Mr Intransition the benefit of the doubt. It was
advertised as a “rant”, not the sum of his knowledge.

One possible change that has been mentioned at least a couple of times
is:

1.1 - 1.0 == 0.1
# → true, using programmer-friendly representation like
BigDecimal

1.1f - 1.0f == 0.1f
# → false; the programmer has explicitly chosen to use
computer-friendly floats

Of course, with coercion, the second example could simply be:
1.1f - 1.0 == 0.1 # → false

Such a language change could be debated and rejected, but it is not
self-evidently on shaky foundations and it is certainly not ludicrous.

There may be difficulties associated with designing and implementing
that particular idea, but it appear to me to be quite consistent with
Ruby as a whole.

Gavin

73

On Fri, Jan 27, 2012 at 12:40 PM, Gavin S. [email protected]
wrote:

1.1f - 1.0 == 0.1 # → false

not bad.
but somehow, my mind seem to prefer 0f1 over 1f, and i can also do
0f_1

kind regards -botp

74

(2) The amount of emotion and defensiveness around this issue is
staggering. It is the very definition of a religious argument. On one
side are people who think “math should work like I learned in grade
school,” on the other are people who think “math does work like I
learned in college” (and the subset of those saying, “if you haven’t
had my level of training then you are stupid”), and then there a bunch
of people in the middle saying, “you’re both right, sort of!”

Well. I have always been bad in Math.

But when I see something like:

x = 1.1

Then i assume that x will be exactly 1.1

That would even by MY own principle of least surprise.

Now I read that, due to the way how Floats are handled on
computers, this is only an approximation.

And I am sad. Because it shows that computers are not perfect
at all.

Because I want to think that Math is more elegant and more
precise than it should have to deal with a

1.100000000000000088817841970012523233890533447265625000000000

instead.

Where is the elegance in that number? It is as if someone is trying
to add some insignificant numbers to Infinity … but of course
once done, these are no longer insignificant! They are now significant
and just rounded!

There goes a beautiful mind. :frowning:

75

Hi Gavin,

On 27/01/12 15:10, Gavin S. wrote:

On Fri, Jan 27, 2012 at 11:05 AM, Garthy D
[email protected] wrote:

To remain wilfully
ignorant of these systems, and then cry “pathetic”, is disingenuous. To
subsequently push for language changes based on such a shaky foundation is
ludicrous.

…

Such a language change could be debated and rejected, but it is not
self-evidently on shaky foundations and it is certainly not ludicrous.

Indeed. The remainder of my text immediately following the excerpt above
was the following:

There are far better ways to stimulate genuine discussion on an
genuinely interesting topic than through a deliberately inflammatory
initial posting and comparable followups.

Again, IMHO.

My apologies for the further noise.

Garth

76

On Fri, Jan 27, 2012 at 04:13, Marc H. [email protected]
wrote:

at all.

It’s a direct consequence of trying to fit certain numbers into a binary
representation. You can work through this without a computer at all,
just
the mathematics around trying to fit certain numbers into fixed-length
representations in some model; the computer is actually incidental here,
to
an extent.

If it weren’t 1.1 but were instead an irrational, such as sqrt(2), would
you still think it’s a failure of a computer that you can’t store its
infinite, non-recurring digits into a computer?

The mathematics behind it is precise. It’s just that you’re assuming
one
finite mathematical model to be an infinite model catered to matching
symbolic manipulation. Is that mismatch unexpected to absolute
beginners?
Most likely. Once you understand the model, though, there’s nothing
bizarre
happening. You haven’t lost some intrinsic beauty. It’s the model.

77

Am 25.01.2012 15:28, schrieb botp:

if i need to recompile manually.
Well, there is at least float-formats
[http://float-formats.rubyforge.org/] available that some kind of
supports IEEE 754-2008 and its decimal float types, but I’m failing
quickly making it show me some basic tricks.

ruby-1.9.3-p0 :006 > a = 1.1 - 1
=> 0.10000000000000009
ruby-1.9.3-p0 :007 > [a].pack(‘D’).each_char.map {|c|
c.ord.to_s(16).upcase}.join(" ")
=> “A0 99 99 99 99 99 B9 3F”
ruby-1.9.3-p0 :008 > IEEE_binary64(a).to_hex(true)
=> “9A 99 99 99 99 99 B9 3F”

Frankly, I don’t know how long we can maintain the one-size-fits-all
approach to floats – representation and calculation precision– in Ruby.
Fixnums automatically prolong into Bignums, strings can have any size
and encoding, but DOUBLE’s (binary 64 floats) are the nearest and the
furthest we come. How about someone already fancy going for 80 bit
binary or the new 128 bit types? Decimal floats are not gonna make it
easier to be dealt with.

The least –and probably ugliest– we could get would be some kind of
global switch which allows the selection of the wanted floating-point
types.

btw: who is Ruy? :wink:

– Matthias

78

On Fri, Jan 27, 2012 at 2:47 PM, Adam P. [email protected]
wrote:

And I am sad. Because it shows that computers are not perfect
at all.

No smiley here? I think the expectation of perfection in a human
built is a tad too optimistic. :slight_smile: The human imperfection infects
everything we do. But: the good news is that there are systems for
symbolic computations around which in fact do a pretty decent job. I
had the chance to work with Maple a while ago and personally I find
those beasts amazing.

The mathematics behind it is precise. It’s just that you’re assuming one
finite mathematical model to be an infinite model catered to matching
symbolic manipulation. Is that mismatch unexpected to absolute beginners?
Most likely. Once you understand the model, though, there’s nothing bizarre
happening. You haven’t lost some intrinsic beauty. It’s the model.

And thanks to IEEE the model (and thus its limitations) is the same
across a wide range of programming languages. So the knowledge about
it can be reused when learning a new language.

Cheers

robert

79

On Fri, Jan 27, 2012 at 11:02:52PM +0900, Adam P. wrote:

representation within the machine and is subject to manipulations within
some specified system.

No . . . the abstraction is “1.1”, and the literal reality is the ever
so
slightly different value produced by the binary implementation beneath
it. The fact that the binary implementation alters the value of float
“1.1” so that it is not equal to decimal 1.1 any longer, despite the
fact
that’s what someone typed in, is a leak in the abstraction. No amount
of
knowledge of the leak make the leak not exist. It does, however, mean
you can account for its leakiness and avoid getting into trouble with it
by way of some extra effort.

If you think of 1.1 as notation for a much more complex floating point
number, which is not the same as 1.1, that doesn’t mean the abstraction
doesn’t exist: it means you’re unravelling it in your head to
accommodate
the implementation’s divergence from decimal 1.1. In essence, the fact
it looks like 1.1 (but isn’t) is the abstraction itself.

The way abstractions are supposed to help us is by saving us the trouble
of thinking about the more complex reality beneath the abstraction. If
the task of keeping track of which use cases violate the simplicity of
the abstraction is more work than saved by the abstraction, it ends up
being a poor abstraction. This is where the special comparison method
proposals make sense: if such a method can guarantee that it is accurate
up to a known, “standard” precision, it’s easy to think “Floats are as
they appear up to precision X,” and just move on with your life, because
it works; without them, we only have something like == as currently
implemented for Float, whose primary value (as far as I can see) is to
provide a tool for learning about the implementation of the Float type,
because there’s no simple rule of thumb for “accuracy up to precision
X”.

Of course, someone might have some other reason for using IEEE-standard
floating point numbers with Float#== is useful, but I don’t know what
that is off the top of my head, and I’m pretty sure it’s a relatively
rare case. The upshot, then, is that instead of having either a decimal
implementation that has known precision, or a Float type with a
comparison method that is accurate up to a known precision (plus the
literal comparison method, with the ability to add different comparisons
for cases where other types of comparison might be more suitable to a
specific problem domain), what we have is the need to implement a
comparison method of our own individual choosing every single time we
want to be able to rely on accuracy of decimal math.

This is ignoring the case of cumbersome notations for additional decimal
types, because the floating point abstraction has already claimed the
literal decimal ground even though it doesn’t work that way.

Note that a decimal “up to precision X” is also an abstraction, but at
least it is an abstraction that would leak far, far less often, because
of the case of things like rounding. I think the only way around that,
given the fact there are limits to how much RAM we have available, would
be to store rational literals (e.g. 2/3 instead of 0.666 . . .)
somewhere
to provide a back-up method for rounding numbers.

Someone tell me if I’m mistaken about some part of that – preferably
without invective.

80

I think the real path forward here would be to propose that BigDecimal
is
loaded by default in Ruby 2.0, and to propose some type of literal
representation for them, such as 1.1D that was suggested before.

The thing that stops people from using BigDecimal right now is the
inconvenience of doing BigDecimal(“1.1”), and the output of
BigDecimal#to_s
and #inspect is difficult to interpret.