Nicotux
instead of result of math operation we can get
inf or -inf or nan or -nan(ind) or maybe other
is there a way to detect when an over/undeflow appends ?
about integer precision ...
strange leak of precision over 8 digits
11111111 * 10 + 1 = 111111112
111111112 */ 0.1 = 0
this seems to indicate that script are using float (32 bit 4 bytes ±1.18 x 10^-38 up to ±3.4 x 10^38 6-9 significant digits)
this is more than enough for GL textures and operations over sliders 100% or range 0..1 or MIDI byte operations
but 16bit fails sometimes, this is not enought for many other things i.e.: time/date, filesize, bitrate starting 192K
specially 64bit version may not have this limitation anymore
inf or -inf or nan or -nan(ind) or maybe other
is there a way to detect when an over/undeflow appends ?
about integer precision ...
strange leak of precision over 8 digits
11111111 * 10 + 1 = 111111112
111111112 */ 0.1 = 0
this seems to indicate that script are using float (32 bit 4 bytes ±1.18 x 10^-38 up to ±3.4 x 10^38 6-9 significant digits)
this is more than enough for GL textures and operations over sliders 100% or range 0..1 or MIDI byte operations
but 16bit fails sometimes, this is not enought for many other things i.e.: time/date, filesize, bitrate starting 192K
specially 64bit version may not have this limitation anymore
geposted Sat 22 Jun 19 @ 6:59 am
Adion
If the arguments are integers then the result will be integer (32-bit signed)
If either of the arguments is floating point, then the result will indeed also be calculated in 32-bit floating point.
If either of the arguments is floating point, then the result will indeed also be calculated in 32-bit floating point.
geposted Sat 22 Jun 19 @ 9:01 am