Generalized Orders of Magnitude

Hmm, how does this compare to things like

https://github.com/cjdoris/LogarithmicNumbers.jl

or

https://github.com/cjdoris/HugeNumbers.jl

(Apart from the PyTorch impl)

In particular, it feels like storing the complete complex number is a bit silly since we know, a priori, that the number exponentiates to ±1, so, wouldn't this mean that we have wasted 31 bits? (=32-1 since only one bit is needed for the sign.)

That being said, this representation is very useful for certain scenarios, of course, when you know that the dynamic range of your number is very large, but, as far as I can tell, it's not exactly super novel, unless I'm missing something!

This is basically a Pytorch library for executing computations over dynamic ranges that exceed Float64's limits, including on GPUs.

I can see how it could be useful when you really need it. Thank you for sharing it on HN.

I tried the sample code for estimating Lyapunov exponents in parallel. It worked on the first try, and it was much faster than existing methods, as advertised. It's nice to come across something that works as advertised on the first try!

The high-dynamic-range RNN stuff may be interesting to others, but it's not for me. In my book, Transformers have won. Nowadays it's so easy to whip-up a small Transformer with a few lines of Python, and it will work well on anything you throw at it.

repo:

https://github.com/glassroom/generalized_orders_of_magnitude