Ruby Math Precision

[Shabbur]

Is there a way to have Ruby be precise with division?

It seems putting whole numbers in an expanded form (8.00000) increases the significant number places by 5, but is there something that estimates infinite precision?

Edit: Nvm just a .0 makes it precise.

 

[cremnophobia]

I'm not sure what you mean but just use integers. There are also rational numbers. If you need precise decimal numbers, then you're out of luck. Ruby has them but RGSS doesn't.


Oh, you should read: http://floating-point-gui.de/

 

[Tsukihime]

How precise do you need them? Can you just round them off or limit your inputs to "good" numbers?

 

[Sarlecc]

Believed Shabbur wanted to return a floating point with no rounding so instead of getting 0 when dividing a number by a larger number you get a floating point number. (i.e 3 / 9 returns 0 but 3.0 / 9 returns 0.3333333333333333 which is the same as 3.fdiv(9)). Also to round it you can do something like this (2.0 / 9).round(6) returns 0.666667.

If you need precise decimal numbers, then you're out of luck. Ruby has them but RGSS doesn't.

Wait what?? I'm confused by this as I have been able to return the same results in my IDE as in RGSS.

 

[cremnophobia]

Wait what?? I'm confused by this as I have been able to return the same results in my IDE as in RGSS.

Ruby's standard library contains BigDecimal. RGSS lacks it.