Actions
Bug #4308
closed
Wrong Bignum calculation in some case.
Status:
Rejected
Assignee:
-
Target version:
-
ruby -v:
ruby 1.9.2p136 (2010-12-25 revision 30365) [x86_64-darwin10.5.0]
Backport:
Description
=begin
I tried to calculate variance of an array but calculation returns wrong result
when numbers are big.The problem also happens in 1.8.7-p330.
I attached a file that recreates The problem.
=end
Files
Updated by naruse (Yui NARUSE) over 14 years ago
- Status changed from Open to Rejected
=begin
It seems from floating point's accuracy.
In line 9, it uses to_f and the value exceeds its limit.
=end
Updated by nobu (Nobuyoshi Nakada) over 14 years ago
=begin
#!/usr/bin/env ruby
class Array
def sum
reduce(:+)
end
def var
empty? ? 0.0 : (map {|x| x ** 2}.sum * size - sum ** 2).fdiv(size ** 2)
end
end
All of these should clearly return 0.¶
p [1295218865, 1295218865, 1295218865, 1295218865, 1295218865].var
#=> 0.0
p [1295218865, 1295218865, 1295218865, 1295218865, 1295218865, 1295218865].var
#=> 0.0
p [1295218865, 1295218865, 1295218865, 1295218865, 1295218865, 1295218865, 1295218865].var
#=> 0.0
=end
Actions
Like0
Like0Like0