On 09/10/2025 05:17, Bald Eagle wrote:
> yesbird wrote:
> 
> I guess various packages have "tolerance" and "decimation error" settings...
...
> How to calculate it?
> Rounding and truncation, I would imagine.

Thank you for parsing Google search results, but I do not see a
concrete formula for precision calculation :).

Long things short: Paul Bourke after reviewing the first version of
converter was wondering why the precision of the data was changed
(increased) on conversion and suggested leaving it as it was in the
input file. I guess this makes sense, because we don't recalculate
coordinates, but simply repack them from one format to another.

In the next release I will follow his advice.
--
YB

Post a reply to this message