809
IEEE 754 (cdn.fosstodon.org)

~~Stolen~~ Cross-posted from here: https://fosstodon.org/@foo/113731569632505985

you are viewing a single comment's thread
view the rest of the comments
[-] addie@feddit.uk 17 points 5 days ago

You can only store rational numbers as a ratio of two numbers, and there's infinitely times more irrational numbers than rational ones - as soon as you took (almost any) root or did (most) trigonometry, then your accurate ratio would count for nothing. Hardcore maths libraries get around this by keeping the "value in progress" as an expression for as long as possible, but working with expressions is exceptionally slow by computer standards - takes quite a long time to keep them in their simplest form whenever you manipulate them.

[-] CanadaPlus@lemmy.sdf.org 9 points 5 days ago

You could choose a subset of fractions, though, and then round it to the nearest one. Maybe you could use powers of two as the denominator for easy hardware implementation. Oh wait, we've just reinvented floats.

this post was submitted on 02 Jan 2025
809 points (99.3% liked)

Programmer Humor

32706 readers
132 users here now

Post funny things about programming here! (Or just rant about your favourite programming language.)

Rules:

founded 5 years ago
MODERATORS