Perhaps, finally, a good test case for isosurface artefacts I've worked 
around for years! Confirmation Windows / OSX showing similar artefacts 
would be useful. Linux users not on Ubuntu 18.04 too.

Looks like something goes wrong when an isosurface's function has an 
inflection (local min/max) where rays cross the containing box/sphere 
surface. Also looks like the inflection can be well away from any shape 
created inside the container; It need not be near the threshold value.

Move the container sides away from the inflection and things work fine - 
also my long standing hack work-around. In the attached test case, three 
of the four containing box sides are at an inflection for the, adjusted, 
tiling pattern function. The last side isn't - by a tiny amount - and no 
artefacts on that side.

See attached image and scene file for v3.8.

Bill P.

Post a reply to this message

Attachments:
Download 'isoinflectioncontainersurface.png' (149 KB) Download 'isoinflectioncontainersurface.pov.txt' (4 KB)

Preview of image 'isoinflectioncontainersurface.png'

hi,

William F Pokorny <ano### [at] anonymousorg> wrote:
> Perhaps, finally, a good test case for isosurface artefacts I've worked
> around for years! Confirmation Windows / OSX showing similar artefacts
> would be useful. Linux users not on Ubuntu 18.04 too.
> ...

same artefacts (identical looking) on a Slackware box, using
3.8.0-alpha.10013324.unofficial.


regards, jr.

Post a reply to this message

On 10/7/19 1:07 PM, jr wrote:
> hi,
> 
> William F Pokorny <ano### [at] anonymousorg> wrote:
>> Perhaps, finally, a good test case for isosurface artefacts I've worked
>> around for years! Confirmation Windows / OSX showing similar artefacts
>> would be useful. Linux users not on Ubuntu 18.04 too.
>> ...
> 
> same artefacts (identical looking) on a Slackware box, using
> 3.8.0-alpha.10013324.unofficial.
> 
> regards, jr.
> 
Thanks jr.

Well... What's going on is subtle and it's an exposure in all continuous 
patterns used with pigments, isosurfaces, whatever.

The root is the SDL use of mod() or, internal to POV-Ray itself, the use 
of fmod() when executing this bit of code - used by all but the two 
potential patterns currently(2):

     if(waveFrequency != 0.0)
         value = fmod(value * waveFrequency + wavePhase, 1.00001); \
// TODO FIXME - magic number! ....

or this bit when functions are used in pattern{} / virtual pattern blocks:

     DBL value = GenericScalarFunctionInstance(pFn, \ 
pThread).Evaluate(EPoint);
     return ((value > 1.0) ? fmod(value, 1.0) : value);

When the mod()/fmod() numerator value is a harmonic of the denominator, 
the resultant value suddenly jumps to 0.0 - though that's not always the 
right thing to do.

In the example scene image I was normalizing the discontinuities of 
tiling 11 so it acts like a linear displacement function for the 
isosurface sheet. The denominator is therefore 1/3 and harmonics of that 
go to 0. Further, due my container sides being at -1,+1 I'm sitting on a 
harmonic and I get spurious shapes.

However, it's a general pattern problem. Should users specify a pattern 
frequency value >1 or use a similar multiplier for a function's values, 
the multiples of 1, 1.00001 or both, depending, are exposed to harmonics 
where the resultant value jumps to zero.

Most often it will be seen with isosurface box containers given they can 
somewhat easily have sides exactly on the mod() harmonics. When the 
artefact otherwise shows up, it will usually be a line of noise perhaps 
the wrong color from a pigment map or similar somewhere is the image. 
I've seen and seen reports of such artefacts over the years. Expect this 
issue sometimes the cause.

Aside: Yes, the pattern { function { } } treatment is different than 
what pattern { } or pigment { agate/etc .... } does, which is not good. 
The former does a better job of of deciding when to apply fmod and does 
it correctly with a denominator of 1.0 - but it ignores the negative 
value space. The latter has that odd 1.00001 denominator(2); the odd 
skipping of calculations when the frequency is 0.0 - and negative values 
are inverted by magnitude. The last something OK with defined patterns 
(gradient x etc), but potentially confusing and sometimes at odds with 
functions(1). Weird stuff can happen with a function's negative values 
due pattern use.

I've used the trick of specifying a frequency 0 when going after a 
little more performance, but I've come up with no reason to not let a 
user take the resultant values to 0 using frequency 0. In fact, that 
with phase provides a nice way of walking a *_map.

Unsure what all to do here, but certainly going to fix what I can in my 
POV-Ray version.

---
Generally we want - on the mod(numerator/denominator) harmonics to 
return the actual value divided by div(numerator/denominator) - instead 
of 0.0. On a 5 day driving trip from LA to NYC across America, it makes 
no sense to fly back to sleep in LA every night. :-)

We need updates for the two bits of code above in pattern.cpp and we 
need a new inbuilt VM function which implements the smarter mod()/fmod() 
to zero - only when we really want zero - functionality. Currently 
calling it f_npmod() - normalized pattern mod(). Also passing an extra 
multiplier to allow normalization back to a 0-1 range.

I have a first pass f_npmod() and pattern.cpp updates. Tested creating 
isosurface 'sheets' from all 27 tiling patterns. 15 of those originally 
hit the mod()/fmod() jump to zero issue. My initial code cleaned up all 
but 6 of those. 4 of those 6 had other numerical issues in the tiling 
code like not clamping to 1.0 where the old 1.00001 fmod allowed the 
tiny overrun.

I'm left with two tiling patterns still an issue in tilings 19 and 20.

I'm attaching an image for 20 showing the initial pattern.cpp 'mod' 
fixes and new f_mpmod function. These partly fixed issues in 19 and 20 
as seen left to right - but there is still something wrong. It looks 
like an incorrect count in x or similar at the isosurface z sides is 
part of it, but I've not found the problem(s).

----
Aside: All through the tiling code there are bits and comments like:

   answer = max(answer, 2.000001); // TODO FIXME - magic number! Should 
use nextafter()

In the tilings changed, I checked nextafter() and never did it work as 
an alternative. Thus far, the magic numbers look to be addressing real 
numerical issues. I've changed these lines to read:

   answer = max(answer, 2.0+1e-7); // +1e-7 add & magnitude due 
numerical fuzziness.

or similar.

It's been the case the original extra amounts has ranged between 1e-6 
and 1e-8, but in all cases 1e-7 worked fine. Interestingly, the value 
adjustments are of a similar order of magnitude to the min intersection 
depth I settled on with the solver accuracy work (4.4e-8).

---

Expect I'll play for quite a while with my changes before opening a 
github issue and publishing a code branch. All patterns are touched by 
my changes. In the meantime, you have my ramblings - for what they're worth.

Bill P.

(1) - Perhaps allow too fabs() with pattern { functions {} } over 
if(value<0) value -= floor(value);. I don't know. Whatever might come 
would be an option so as to not break scenes built on the current 
inverting of the negative (0-1) magnitudes. I've done nothing with this 
secondary concern around using other than >=0 valued functions as patterns.

(2) - The odd 1.00001 fmod denominator has largely hidden the base issue 
- forever. Why someone did it I'd bet. Not very often will usage land 
exactly on a harmonic of 1.00001.

Post a reply to this message

Attachments:
Download 'tiling20_story.png' (64 KB)

Preview of image 'tiling20_story.png'

hi,

William F Pokorny <ano### [at] anonymousorg> wrote:
> On 10/7/19 1:07 PM, jr wrote:
> > William F Pokorny <ano### [at] anonymousorg> wrote:
> >> Perhaps, finally, a good test case for isosurface artefacts I've worked
> >> around for years! Confirmation Windows / OSX showing similar artefacts
> >> would be useful. Linux users not on Ubuntu 18.04 too.
> >> ...
> >
> > same artefacts (identical looking) on a Slackware box, using
> > 3.8.0-alpha.10013324.unofficial.
> >
> Thanks jr.

pleasure.

(am dismayed -- riled, actually -- that no Windows/Mac users found the few
minutes it took, apparently)

> ...


regards, jr.

Post a reply to this message

On 10/13/19 11:19 AM, jr wrote:
> hi,
> 
> William F Pokorny <ano### [at] anonymousorg> wrote:
>> On 10/7/19 1:07 PM, jr wrote:
>>> William F Pokorny <ano### [at] anonymousorg> wrote:
>>>> Perhaps, finally, a good test case for isosurface artefacts I've worked
>>>> around for years! Confirmation Windows / OSX showing similar artefacts
>>>> would be useful. Linux users not on Ubuntu 18.04 too.
>>>> ...
>>>
>>> same artefacts (identical looking) on a Slackware box, using
>>> 3.8.0-alpha.10013324.unofficial.
>>>
>> Thanks jr.
> 
> pleasure.
> 
> (am dismayed -- riled, actually -- that no Windows/Mac users found the few
> minutes it took, apparently)
> 
>> ...
> 
> regards, jr.
> 

:-) Ah, everyone is busy and focused upon what they're focused upon. I'm 
bad at even minor task switching/juggling. Plus! Could be they see the 
man off wandering in the weeds - as a man off wandering in the weeds.

While I think I understand what's happening, results from POV-Ray 
compiled with other compilers still of some interest should anyone be 
motivated to test.

---
The code involved makes use of abs/fabs, min, max and the like. 
Behaviors of which - especially when you don't get the expected C++ 
standard library version(1) - can be different.

(1) - As happened with vector.h abs() use when linux users initially 
tried the v3.8 user defined camera. The immediate fix was to use fabs(), 
but on my to-do, someday, list to look at vector.h more closely. I 
suspect it still the case we are getting the c99 math functions in 
vector.h and not the standard library versions due cyclic includes, but 
maybe not, and maybe I should stop looking for weeds to whack.

Related to (1): Had the thought to perhaps enforce std::abs, std::min, 
std::max use over bare abs/fabs ?/..., min, max names enabled with 
'using...' statements in coretypes.h & types.h. These in particular are 
known to have MS VS to other compiler implementation differences.

Bill P.

Post a reply to this message

William F Pokorny <ano### [at] anonymousorg> wrote:

I'm not at all sure what deep C++ functions and libraries you're dealing with -
that gets down to a deeper level than I have ever gone.

I think I understand what you're struggling with - it's akin to the coincident
surface problem - so let me ask - would adding something like crand / jitter /
some other random noise to the function make it better or worse?
Say, something with a range from 0.99999 to 1.00001?   Or 1+/- E-7?

Not that it solves the underlying problem, but since it went undiscovered / was
well enough hidden for _this_ long, maybe the thing to do is just build a
deeper, plusher rug to sweep it under, with the humblest of apologies and a most
heartfelt promise to come back and fix it "later"   ;)


It's always the edge cases that take 90% of the time...   :|

Perhaps there's something along the lines of this that might spark some ideas?
http://news.povray.org/povray.general/thread/%3Cweb.5c85aaecd4241a93765e06870%40news.povray.org%3E/

Post a reply to this message

hi,

William F Pokorny <ano### [at] anonymousorg> wrote:
> On 10/13/19 11:19 AM, jr wrote:
> > William F Pokorny <ano### [at] anonymousorg> wrote:
> >> On 10/7/19 1:07 PM, jr wrote:
> >>> William F Pokorny <ano### [at] anonymousorg> wrote:
> >>>> Perhaps, finally, a good test case for isosurface artefacts I've worked
> >>>> around for years! Confirmation Windows / OSX showing similar artefacts
> >>>> would be useful. Linux users not on Ubuntu 18.04 too.
> >>>> ...
> >>>
> >>> same artefacts (identical looking) on a Slackware box, using
> >>> 3.8.0-alpha.10013324.unofficial.
> >>>
> >> Thanks jr.
> >
> > pleasure.
> >
> > (am dismayed -- riled, actually -- that no Windows/Mac users found the few
> > minutes it took, apparently)
> >
>
> :-) Ah, everyone is busy and focused upon what they're focused upon. I'm
> bad at even minor task switching/juggling. Plus! Could be they see the
> man off wandering in the weeds - as a man off wandering in the weeds.

quite the image..  </grin>


> ...
> The code involved makes use of abs/fabs, min, max and the like.
> Behaviors of which - especially when you don't get the expected C++
> standard library version(1) - can be different.
>
> (1) - As happened with vector.h abs() use when linux users initially
> tried the v3.8 user defined camera. The immediate fix was to use fabs(),
> but on my to-do, someday, list to look at vector.h more closely. I
> suspect it still the case we are getting the c99 math functions in
> vector.h and not the standard library versions due cyclic includes, but
> maybe not, and maybe I should stop looking for weeds to whack.
>
> Related to (1): Had the thought to perhaps enforce std::abs, std::min,
> std::max use over bare abs/fabs ?/..., min, max names enabled with
> 'using...' statements in coretypes.h & types.h. These in particular are
> known to have MS VS to other compiler implementation differences.

in a way I'm surprised that POV-Ray does not contain its own low-level maths
library, I'd have thought that an advantage.  so, I wonder (not knowing the code
base + off the top of my head) whether collecting various low-level routines
into a "convenience" library included with POV-Ray would not help with platform
independence/"implementation differences".  libraries like the GSL (see
attached) are highly optimised and easily wrapped/"plundered".


regards ,jr.

Post a reply to this message

On 10/13/19 3:15 PM, Bald Eagle wrote:
> William F Pokorny <ano### [at] anonymousorg> wrote:
> 
> I'm not at all sure what deep C++ functions and libraries you're dealing with -
> that gets down to a deeper level than I have ever gone.
> 
> I think I understand what you're struggling with - it's akin to the coincident
> surface problem - so let me ask - would adding something like crand / jitter /
> some other random noise to the function make it better or worse?
> Say, something with a range from 0.99999 to 1.00001?   Or 1+/- E-7?
> 
> Not that it solves the underlying problem, but since it went undiscovered / was
> well enough hidden for _this_ long, maybe the thing to do is just build a
> deeper, plusher rug to sweep it under, with the humblest of apologies and a most
> heartfelt promise to come back and fix it "later"   ;)
> 

:-)

Often I bump the isosurface boundary a little, hide or avoid parts not 
right - ten plus years of it. Often, I walk away from ideas in 
frustration because POV-Ray doesn't work as intended.

I hit the issues here more than most because I play with isosurfaces. It 
should be the tiling patterns and others provide a rapid path to complex 
shapes with low gradients. Shapes where we can use all the usual pattern 
modifiers and extensions. The larger mechanisms are there, but they 
don't completely work. It's frustrating.

Your recent posts in part motivated me push again on tiling patterns 
with isosurfaces. I can these days often determine the core problems in 
the source code. I uttered the words, "let's figure this out." (He talks 
to himself too...)

Started expecting a problem or two. Found a half a dozen bugs / 
shortcomings. I've in hand code addressing most. A worry the changes 
ripple into parts of POV-Ray in ways I don't presently see. The 
abs/min/max concerns are secondary and general.

Many of my wild ideas require warps - bending assemblies rather than 
blobbing assemblies. I often cannot avoid the mod/fmod harmonics without 
introducing visible seams due the avoidance. I'm stuck between a rock 
and a hard place, off the beaten path - in the weeds.

Speaking of rugs - and now completely drifting - I believe a secrete 
attraction of physically-based / stochastic rendering methods is the 
approach covers for train loads of numerical sin. Rugs and trains - 
where's Ton... ;-)

> 
> It's always the edge cases that take 90% of the time...   :|

Yep!

> 
> Perhaps there's something along the lines of this that might spark some ideas?
>
http://news.povray.org/povray.general/thread/%3Cweb.5c85aaecd4241a93765e06870%40news.povray.org%3E/
> 
Yes, I remember. A similar discussion. :-)

Bill P.

Post a reply to this message

On 10/13/19 5:14 PM, jr wrote:
> 
> in a way I'm surprised that POV-Ray does not contain its own low-level maths
> library, I'd have thought that an advantage.  

If I'm following your meaning. There is some. Some even in the shipped 
SDL includes.

> so, I wonder (not knowing the code
> base + off the top of my head) whether collecting various low-level routines
> into a "convenience" library included with POV-Ray would not help with platform
> independence/"implementation differences". 

Again improvement possible in this vein though I don't expect it to help 
much with platform issues - if we have many.

Truth is POV-Ray is really good at compiling and running the same way on 
any platform with a standards compliant C++ compiler. The core 
developers have worked hard at to make this so.

I was speaking of a few concerns - those mostly given recent past 
vector.h behavior in not getting the C++ standard include for abs. It 
just happened the code involved here touches on those 'concerns.' Had 
OSX and Windows users tested, it was as much to confirm no differences 
as anything. I think we 'might' have a soft spot - but maybe not, it's a 
worry.

> libraries like the GSL (see
> attached) are highly optimised and easily wrapped/"plundered".
> 

Yeah, such suggestions always sound good - to me too on first glance. 
The devils are in the details...

In looking at the solvers I've looked on and off at gsl, eigen, Intel's 
embree(sp?) package and others. I borrowed ideas for updates to our 
common quadratic solver from a CERN math package. The julia language 
itself is interesting in that it might eventually be a path to an llvm 
jit implementation POV-Ray could use - but I think it too unstable to 
consider seriously at this point. Julia is in the shiny, new, gazillion 
commits a day phase. Aside: I see julia as an implementation of try this 
math on the fly ideas CERN has been doing themselves for a long time by 
hacking C or C++. So many languages, packages, and papers out there... 
One just drowns in the noise.

I'm looking again at eigen's nD vector support of late as an alternative 
to vector.h which - it appears - is tangled in a negative performance 
impact 3.7 to 3.8. But more interested due my immediate solver work 
needs for flexibility with respect to the 'n' part of nD. I'd like our 
vector.h to be just what I need and it isn't. :-)

Anyway. What can I say somewhat quickly. With my solver work one aim has 
been moving my code in a direction which makes plugging in a common 
library more doable. Still, lots of details would hang us up. Just 
trying to plug in standard solvers where it could be done technically 
(you'd get roots) is iffy because ours are often ray tracer tweaked 
solvers. A lot of the code you find, in sage for example, has the 
solvers set up for more than direct on floating point hardware reals too 
- meaning the code is often built upon largish software packages 
themselves - which makes extracting just what we want harder.

Making parts and pieces of gsl et al available as functions in the 
POV-Ray VM could be interesting. There, not be being tangled in the 
larger POV-Ray code base might make it easier to do.

Bill P.

Post a reply to this message

On 10/12/19 1:57 PM, William F Pokorny wrote:
...
> I have a first pass f_npmod() and pattern.cpp updates. Tested creating 
> isosurface 'sheets' from all 27 tiling patterns. 15 of those originally 
> hit the mod()/fmod() jump to zero issue. My initial code cleaned up all 
> but 6 of those. 4 of those 6 had other numerical issues in the tiling 
> code like not clamping to 1.0 where the old 1.00001 fmod allowed the 
> tiny overrun.
> 
> I'm left with two tiling patterns still an issue in tilings 19 and 20.
> 
> I'm attaching an image for 20 showing the initial pattern.cpp 'mod' 
> fixes and new f_mpmod function. These partly fixed issues in 19 and 20 
> as seen left to right - but there is still something wrong. It looks 
> like an incorrect count in x or similar at the isosurface z sides is 
> part of it, but I've not found the problem(s).
...

Following up. The last issues with 19 and 20 were me not getting the 
scaling for repetition right. I'd cut those two patterns in half on one 
axis.

All tiling patterns now clean for me in isosurface use.

Bill P.

Post a reply to this message