We have established that the `datetime(now)` combo is both currently 
broken (on some systems), and has some shortcomings in its design (on 
all systems, both broken and otherwise).


I think the key issues in finding any satisfactory solution are the 
following:


- The current broken variant of the behavior has been around for quite a 
while, and people have been using it. From their perspective, the only 
thing wrong about the broken behavior is that there's a trailing "Z" 
behind the default date/time strings generated.

- Probably the most common use case for the `datetime(now)` combo would 
ideally have it report local time (matching the broken behavior), rather 
than UTC as originally intended and documented.

- Aside from this combo, the use cases for the `now` pseudo-variable may 
actually give more weight for UTC-based behavior.

- It would be excessively complex to make the `now` pseudo-variable 
convey timezone information to `datetime` (especially considering that 
one might want to do arithmetics on it first, or store it in another 
variable, or any other scenario where it would have to have numeric 
value semantics).

- We can't have both at the same time.


I hereby present the following alternative approaches as possible solutions:


(A) Timezone Global Setting
---------------------------

- A `timezone` global setting is added, with possible values being 
either `local` or `utc`.

- In `local` mode, `now` returns a local time based value on all 
systems, and the `datetime` format string defaults to `%Y-%m-%d 
%H:%M:%S` (note the absence of any time zone specifier whatsoever; see 
notes below for reasons).

- In `utc` mode, `now` returns a UTC based value on all systems, and the 
`datetime` format string defaults to `%Y-%m-%d %H:%M:%SZ` (as it 
currently does; note the presence of a literal "Z" time zone specifier 
to explicitly indicate UTC).

- For now, choosing `timezone local` will prompt a warning that the 
behavior with respect to daylight savings time needs further testing, 
may be subject to change, and is to be regarded as experimental.

- If `timezone` is not explicitly specified, it will default to `utc` 
(being the originally intended and documented behavior), with the first 
use of `now` or `datetime` prompting a warning that the behavior may not 
be as expected.

- In deviation of the above, if `#version 3.7` is specified, and in the 
absence of an explicit `timezone` setting, the behavior of `now` and 
`datetime` will match the current behavior (i.e. `now` will exhibit 
`utc` or `local` mode behavior depending on platform, while `datetime` 
will invariably exhibit `utc` mode behavior), and the warning will be 
somewhat different, in that it will indicate that the behavior is 
specifically for backward compatibility and may lead to bogus results on 
some systems and/or unexpected results on others.

- Switching between modes will be supported, but since values obtained 
from `now` in one mode would lead to bogus results when used in 
`datetime` (with default format string) in the other mode, a warning 
will be issued if there is any switch between a use of `now` and a 
subsequent use of `datetime` without an explicit format string.


NOTES:

- The most common `local` mode use cases presumably don't need an 
explicit time zone specifier, so including a trailing "%z" or "%Z" in 
the default format string would only unnecessarily add another instance 
of time zone handling that could turn out to have pitfalls of its own. 
Users who need such information are still free to include such suffixes 
by specifying a custom format string.

PROS:

- The default `datetime` format string is automatically adapted to 
whatever mode the user chooses.

CONS:

- It might turn out that `local` mode can't be made to work properly due 
to issues related to daylight savings time.

VARIANTS:

(A.1) Alternatively, `now` could be implemented to always return UTC 
based values, with `datetime` instead automatically adding a timezone 
offset when in `local` mode. This variant should be safe from potential 
daylight saving times issues, and the mode could be switched at will 
without exercising any extra caution. However, it would also mean the 
user couldn't easily probe for the time zone offset.


(B) Timezone Pseudo-Variable
----------------------------

- A `timezone` pseudo-variable is added, evaluating to the timezone 
offset to UTC in fractions of a day.

- `now` will be fixed to return a UTC based value on all systems (unless 
`#version 3.7` is specified, in which case behavior will be as currently 
implemented, while prompting a warning if the result differs from fixed 
behavior),

- `datetime` will continue to work as currently implemented, i.e. using 
a string that implies an UTC-based timestamp. A warning might be issued 
if no explicit format string specified and the timestamp parameter is 
anything other than pure `now`.

- Local date/time strings can thus be generated using 
`datetime(now+timezone,FORMAT)`


PROS:

- No switching between modes needed.
- The solution should be immune to daylight savings time issues.

CONS:

- The value of the timezone variable would have unexpected units (days 
instead of hours).
- Getting local date/time strings would always require am explicit 
format string.

VARIANTS:

(B.1) Have `datetime` default to specifying no timezone information 
whatsoever. This variant would avoid the need to specify an explicit 
format string to produce proper local date/time strings. However, it 
would produce misleading results for `datetime(now)` as the absence of a 
timezone specifier would erroneously imply local time when it would 
actually be UTC.

(B.2) Have `now` return local time instead of UTC, and use 
`now-timezone` to get UTC. This variant would avoid the need to specify 
an explicit format string to produce proper local date/time strings, 
while still getting sane (albeit local time) results from 
`datetime(now)`. However, this approach might turn out to not work 
properly due to potential daylight savings time issues.


(C) Datetime mode parameter
---------------------------

This approach is somewhat similar to (A.1):

- `now` will be fixed to return a UTC based value on all systems (unless 
`#version 3.7` is specified, in which case behavior will be as currently 
implemented, while prompting a warning if the result differs from fixed 
behavior),

- An optional timezone mode parameter is added to `datetime`, with 
possible values being either `local` or `utc`, the default being `utc`.

- In `local` mode, `datetime` automatically adds a timezone offset to 
the timestamp parameter (which is always presumed to be based on UTC), 
and the format string defaults to `%Y-%m-%d %H:%M:%S` (note the absence 
of any time zone specifier whatsoever).

- In `utc` mode, `datetime` takes the timestamp parameter as-is, and the 
format string defaults to `%Y-%m-%d %H:%M:%SZ` (as it currently does; 
note the presence of a "Z" time zone specifier to explicitly indicate UTC).


PROS:

- The default `datetime` format string is automatically adapted to 
whatever mode the user chooses.
- The solution should be immune to daylight savings time issues.

CONS:

- The presumably more frequent use case, `local` mode, would always 
require to specify the mode parameter.

VARIANTS:

(C.1) The default of the mode parameter could be specified to be `local` 
rather than `utc`. This would avoid the need to explicitly specify the 
parameter in the presumably more common `local` use case. However, for 
backward compatibility it would require that `datetime` behave 
differently when `#version 3.7` is specified, and would still require 
specifying the mode parameter each time `utc` mode is desired.

(C.2) This approach could be combined with (A.1), using the global 
setting as the default when `datetime` is used without an explicit mode 
parameter. This would avoid the need to explicitly specify the parameter 
each time `local` mode is desired, while at the same also avoiding the 
need to make `datetime` behavior subject to `#version` setting.

Post a reply to this message

Am 09.08.2021 um 00:54 schrieb clipka:

> I hereby present the following alternative approaches as possible 
> solutions:

My personal preferences lean heavily towards (A), with a perspective to 
change over to (A.1) later if daylight savings time does indeed turn out 
to be a problem.

(A) explicitly includes provisions to let people know that the `local` 
mode may still be subject to change, for this exact reason.


The reason for any DST issues would be that, in order for stuff to work 
nicely, the time zone of the timestamp "zero point" (2000-1-1 00:00) 
probably needs to match the time zone applicable for (a) the point in 
time represented by the timestamp, or (b) the point in time at which the 
scene is parsed (and I'm not sure which of the two). Either way, since 
2000-1-1 is outside the DST period for most time zones, there is reason 
to expect things to break down when invoking `datetime(now)` during the 
DST period, since the DST time zone typically doesn't match the regular 
time zone.

Since UTC does not have a DST variant, this is a non-issue if our 
numeric timestamps are always UTC-based.

Post a reply to this message

hi,

clipka <ano### [at] anonymousorg> wrote:
> We have established that the `datetime(now)` combo is both currently
> broken (on some systems), and has some shortcomings in its design (on
> all systems, both broken and otherwise).
>
>
> I think the key issues in finding any satisfactory solution are the
> following:
> ...

could make a "fine" setting for 'povray.conf', eg:

  [timezone]
  ; local
  utc


regards, jr.

Post a reply to this message

Am 09.08.2021 um 09:07 schrieb jr:
> hi,
> 
> clipka <ano### [at] anonymousorg> wrote:
>> We have established that the `datetime(now)` combo is both currently
>> broken (on some systems), and has some shortcomings in its design (on
>> all systems, both broken and otherwise).
>>
>>
>> I think the key issues in finding any satisfactory solution are the
>> following:
>> ...
> 
> could make a "fine" setting for 'povray.conf', eg:
> 
>    [timezone]
>    ; local
>    utc

I disagree.

`povray.conf` is there to tie the *NIX front-end(!) of your POV-Ray 
installation in to your *NIX run-time environment, and nothing more. 
Mainly to let the binary know where to find its auxiliary files 
(includes, primarily), and what directories never to access even if a 
scene says pretty please with a cherry on top.

There's also no such file in POV-Ray for Windows; such information is 
handled entirely differently there.


Also, it is definitely a setting that individual scenes should be able 
to override. Because in the end it will usually be the scene author who 
knows best whether they want some date or time displayed in their scene 
in local or UTC format - and it might change from scene to scene, or 
even between different portions of the scene.


If anything, one could argue that there should be an INI setting to 
specify the default, which in turn would make it configurable globally 
via the master `povray.ini`.

But the more places we want to add to tweak the setting, the more work 
it is to implement, and for the time being I'd like to keep it simple, 
and leave the implementation of such an INI default as an exercise to 
the reader, so to speak.

Post a reply to this message

hi,

clipka <ano### [at] anonymousorg> wrote:
> Am 09.08.2021 um 09:07 schrieb jr:
> > hi,
> >
> > clipka <ano### [at] anonymousorg> wrote:
> >> We have established that the `datetime(now)` combo is both currently
> >> broken (on some systems), and has some shortcomings in its design (on
> >> all systems, both broken and otherwise).
> >>
> >>
> >> I think the key issues in finding any satisfactory solution are the
> >> following:
> >> ...
> >
> > could make a "fine" setting for 'povray.conf', eg:
> >
> >    [timezone]
> >    ; local
> >    utc
>
> I disagree.

I do too, now.  :-)

> `povray.conf` is there to tie the *NIX front-end(!) of your POV-Ray
> installation in to your *NIX run-time environment, and nothing more.
> Mainly to let the binary know where to find its auxiliary files
> (includes, primarily), and what directories never to access even if a
> scene says pretty please with a cherry on top.

a better suggestion would have been:

  [locale]
  tz = {local|utc}
  lang = en_GB.utf8

as you say, conf provides environment type info.  given that 'strftime()' uses
locale, a setting to provide a preference/default could be useful.  (POV-Ray
appears to use the locale already, anyway)


> There's also no such file in POV-Ray for Windows; such information is
> handled entirely differently there.

ok, did not know that.


> Also, it is definitely a setting that individual scenes should be able
> to override. ...

sure, thought setting a convenient default, and override in 'datetime()'s second
arg, if required.


regards, jr.

Post a reply to this message

> - The current broken variant of the behavior has been around for quite a
> while, and people have been using it. From their perspective, the only
> thing wrong about the broken behavior is that there's a trailing "Z"
> behind the default date/time strings generated.

Why not keep it the way it is an have an option/flag for removing the Z?

Gotta head out in a few, so only briefly:
A lot of this seems complicated, and I personally dislike too many options and
settings.

Would it be a bad thing to have localtime (), utctime (), and timezone () as
completely separate functions?  No idea what to do about daylight savings time
since it's such a bad idea that there are people organizing and petitioning to
get rid of it all together.

Post a reply to this message

Am 09.08.2021 um 12:53 schrieb jr:

> a better suggestion would have been:
> 
>    [locale]
>    tz = {local|utc}
>    lang = en_GB.utf8
> 
> as you say, conf provides environment type info.  given that 'strftime()' uses
> locale, a setting to provide a preference/default could be useful.  (POV-Ray
> appears to use the locale already, anyway)

No, that's not really it.

You don't need to tell POV-Ray whether your system HAS a time zone or 
not. Because your system DOES, period. Even if that timezone happens to 
be UTC.

In the context of the issue, you have to tell POV-Ray whether you want 
to operate in the context of that time zone, or rather prefer to operate 
in a context that is universal across time zones.

And that doesn't differ based on what system you're running POV-Ray on. 
It differs based on what you want to do with POV-Ray.


As for specifying _details_ about the locale, such as the actual time 
zone, language or whatnot, that isn't much of a thing either. Your 
system has a default locale, and there's a standard interface for making 
use of that. And there's little to no point in using a different one for 
POV-Ray (except possibly on a per-scene basis, but in that case 
`povray.conf` wouldn't be the right place anyway).


>> Also, it is definitely a setting that individual scenes should be able
>> to override. ...
> 
> sure, thought setting a convenient default, and override in 'datetime()'s second
> arg, if required.

As I said, the INI file mechanism would be the way to go for such purposes.

Post a reply to this message

Am 09.08.2021 um 13:17 schrieb Bald Eagle:

>> - The current broken variant of the behavior has been around for quite a
>> while, and people have been using it. From their perspective, the only
>> thing wrong about the broken behavior is that there's a trailing "Z"
>> behind the default date/time strings generated.
> 
> Why not keep it the way it is an have an option/flag for removing the Z?

Because the current status is inconsistent. It currently does different 
things on different systems.

Just adding a flag to suppress the trailing "Z" seems a bit daft to me, 
as it just dumps all the responsibility into the user's lap to get 
things working properly, when we could just as well implement a 
mechanism that does everything right out of the box, while not being any 
more complex - from the user's point of view.


> Gotta head out in a few, so only briefly:
> A lot of this seems complicated, and I personally dislike too many options and
> settings.

Maybe you're confuzzled by the fact that I've described impementation 
details, not just what the user sees in a standard use case.


> Would it be a bad thing to have localtime (), utctime (), and timezone () as
> completely separate functions?

If you can describe a system of such functions that works consistently, 
and is reasonably foolproof to use, then I'd be happy to take it into 
consideration.

As it currently stands, this sounds like a system with a couple of 
building blocks but no mortar to properly tie them together, so the user 
has to mix their own and hope that it's suited for the task.

Pitfalls are plenty in the world of time and timezones, so I'd prefer 
users not have to worry about it themselves (unless they know what they 
are doing and consciously decide to do their own worrying).


> No idea what to do about daylight savings time
> since it's such a bad idea that there are people organizing and petitioning to
> get rid of it all together.

For the time being (no pun intended), daylight savings time does exist 
as an undeniable fact of life we have to put up with. If we try to 
ignore it, we will produce a program that breaks at least half of the 
time in half of the world.

Post a reply to this message

Am 09.08.2021 um 13:56 schrieb clipka:

>> Would it be a bad thing to have localtime (), utctime (), and timezone 
>> () as
>> completely separate functions?
> 
> If you can describe a system of such functions that works consistently, 
> and is reasonably foolproof to use, then I'd be happy to take it into 
> consideration.

(Also, remember that `now` and `datetime` already exist, and we should 
make at least some good faith effort to make them consistent - both with 
respect to themselves, as well as each other, and any additional time 
related stuff we may want to add - and rid them from any undesired 
quirks if possible.)

Post a reply to this message

hi,

clipka <ano### [at] anonymousorg> wrote:
> Am 09.08.2021 um 12:53 schrieb jr:
>
> > a better suggestion would have been:
> >
> >    [locale]
> >    tz = {local|utc}
> >    lang = en_GB.utf8
> >
> > as you say, conf provides environment type info.  given that 'strftime()' uses
> > locale, a setting to provide a preference/default could be useful.  (POV-Ray
> > appears to use the locale already, anyway)
>
> No, that's not really it.
>
> You don't need to tell POV-Ray whether your system HAS a time zone or
> not. Because your system DOES, period. Even if that timezone happens to
> be UTC.

:-)  ("divided by a common language")


> In the context of the issue, you have to tell POV-Ray whether you want
> to operate in the context of that time zone, or rather prefer to operate
> in a context that is universal across time zones.

exactly.  hence tell POV-Ray, in .conf ideally, what the system/user preference
is, ie whether 'now' ought to mean "local" or "utc", unless overridden.


> ...
> As I said, the INI file mechanism would be the way to go for such purposes.

perhaps.


regards, jr.

Post a reply to this message

On 8/8/21 6:54 PM, clipka wrote:
> We have established that the `datetime(now)` combo is both currently 
> broken (on some systems), and has some shortcomings in its design (on 
> all systems, both broken and otherwise).
> 
> 
> I think the key issues in finding any satisfactory solution are the 
> following:
...

Suppose I'd lean toward (C.2) with (A.1) for what you've proposed.

FWIW.
----
In June I worked some on datetime() to support an internal always local 
clock now, but as importantly to make work the %z and %Z formatting 
options whether the fractional days were coming from now or being 
generated on the fly. The last bit, I failed initially to recall on 
mentioning the trailing 'Z' with a non utc time result.

Beyond the initial changes, I had plans for a new f_clock() internal 
function. Admittedly, my aim was in part to insulate povr from whatever 
'now' was doing, but there were additional reasons. Due your post, I've 
worked up an initial version of f_clock() much earlier than planned to 
test how crazy my plans were and to help clarify my thoughts.

---
In your preamble, you didn't mention 'now' being used as an SDL timing 
mechanism. It's something people do today and it carries risk with 
respect to any particular timing result. The reasons is we are using the 
system_clock for 'now' which can be updated at any time moving forward 
or backward - synchronization with time servers, etc.

We don't warn against timing use of 'now' in the documentation as far as 
I know. Perhaps we should?

In povr's f_clock() the up front returned time comes from steady_clock 
instead - which on my Ubuntu 20.04 machine appears to be a monotonic 
clock since the last reboot/boot.

Suppose the following povr SDL:

//---
#declare Fn_TimePt00 = function (sec_in_day) { now*sec_in_day }
#declare TimePt00 = f_clock(0,0,0);
#debug concat("f_clock(0,0,0) ",str(f_clock(0,0,0),9,9),"\n")
#for (I,0,3000000,1)
     #declare Dummy = 1.0;
#end
#declare Fn_TimePt01 = function (sec_in_day) { now*sec_in_day }
#declare TimePt01 = f_clock(0,0,0);
#debug concat("f_clock(0,0,0) ",str(f_clock(0,0,0),9,9),"\n")

#debug "\n"
#declare SecInDay = (24*60*60);
#debug concat("Fn_TimePt00 ",str(Fn_TimePt00(SecInDay),9,9),"\n")
#debug concat("Fn_TimePt01 ",str(Fn_TimePt01(SecInDay),9,9),"\n")
#debug concat("Fn_TimePt01 - Fn_TimePt00 ",
     str(Fn_TimePt01(SecInDay)-Fn_TimePt00(SecInDay),9,9),"\n")

#debug "\n"

#debug concat("TimePt00 ",str(TimePt00,9,9),"\n")
#debug concat("TimePt01 ",str(TimePt01,9,9),"\n")
#debug concat("TimePt01 - TimePt00 ",
     str(TimePt01-TimePt00,9,9),"\n")
#debug "\n"

#error "Parsing test. Stop early."
//---

The povr branch generates:

f_clock(0,0,0) 234835.462146713
Parsing 17764K tokens
f_clock(0,0,0) 234838.496408219

Fn_TimePt00 681978977.077232242
Fn_TimePt01 681978980.111495137
Fn_TimePt01 - Fn_TimePt00 3.034262896

TimePt00 234835.462139445
TimePt01 234838.496401535
TimePt01 - TimePt00 3.034262090

Of note for POV-Ray v3.8 and onward is that the Fn_TimePt00, Fn_TimePt00 
method for capturing time points should work in v3.8 beta 1, but it 
doesn't because when 'now' was added, the support for it was not added 
to parser_strings.cpp alongside pi, tau, etc. This a stand alone 'now' 
issue.

The other current return types from f_clock() are:

1 - Whole seconds to system wall clock since Y2K.
2 - Fractional days to system wall clock since Y2K (The povr 'now').
3 - Difference between machine epoch and y2k epoch in seconds.
4 - Convert local seconds since machine epoch to utc.

The second argument is whether to account for dst, or not, for types 1,2,3.

The third and last argument is the local seconds argument for return 
type utc. The method for this turned out uglier than hoped, though it 
appears to work.

I don't consider the form of f_clock final - in povr, as often as not, 
just trying outs ideas. A type to directly return seconds since machine 
epoch might be nice. Maybe other stuff. We'll see.

---
Putting more of the whole plan/implementation on the table because I see 
secondary 'concerns' such as POV-Ray's Y2K epoch not lining up with any 
unix like system I've ever used - all have been at Jan 1, 1970. This 
makes understanding the values POV-Ray is generating and using harder to 
work with.

Unix commands like date have some nice options:

date --date='TZ="America/Los_Angeles" 09:00 next Mon' +"%s"

but we cannot use the results directly in POV-Ray. Hmm, maybe an 
explicit unix time to y2k seconds type for f_clock...

---
I'll end with my view our 'now' is trying to be both a timer and a 
time/date input and I'm not sure these are completely compatible. Most 
of the time/date stuff which works for dst uses only seconds resolution 
as far as I know(1).

---
Aside:
Supposing datetime fixed to support %z %Z, and 'now' returning always a 
no dst utc or local time you can fairly easily adjust the fractional day 
value to account for dst of say an hour with:

#declare DSToffset = (1/24);

#debug concat("now + (1/24) ",
datetime(now+DSToffset,"%Y-%m-%d %H:%M:%S %z"),"\n")

#debug concat("now + (1/24) ",
datetime(now+DSToffset,"%Y-%m-%d %H:%M:%S %Z"),"\n")

FWIW.

Bill P.

(1) - The C++/Boost (plus time.h etc) date/time options go on forever. I 
don't begin to understand them all and it looks like c++20 is aiming to 
revamp/extend a lot of it. Coding wise. I tried to stick mostly with c.

Post a reply to this message

Am 11.08.2021 um 14:57 schrieb William F Pokorny:

> In your preamble, you didn't mention 'now' being used as an SDL timing 
> mechanism. It's something people do today and it carries risk with 
> respect to any particular timing result. The reasons is we are using the 
> system_clock for 'now' which can be updated at any time moving forward 
> or backward - synchronization with time servers, etc.
> 
> We don't warn against timing use of 'now' in the documentation as far as 
> I know. Perhaps we should?

I don't think there's much sense to warn of using `now` in this capacity 
in particular.

If people want a rough idea of how long the individual pieces of their 
scene take to parse, for example to decide what portions of their scene 
to try and optimize, `now` is probably good enough. It's not like system 
clocks are typically corrected umpteen times a day, or changed by huge 
values.

Also, given that `now` has days as its unit of measure, and the docs 
make no promises about its precision other than that it is "seconds or 
better", I think that's enough of a warning to not use it for 
microseconds precision measurements if anything serious depends on it.

If people want precise performance metrics, anything based on a real 
time clock - even `std::chrono::steady_clock` - is a poor choice of tool 
anyway. In general, I would guess that the task/thread scheduling jitter 
introduces far more noise into the measurements than the occasional 
system clock adjustment. CPU time would be a far better candidate for 
such things.


Also note that for the purpose of performance metrics, we already 
provide measurements of the total parsing time, both real time and CPU time.

If you think we should provide more tools for performance measurements, 
I would argue that some `#split_time` directive to print out the current 
parser stats would make more sense than some basic building block that 
users would have to build additional SDL code around, that in turn would 
potentially skew the results, to actually put to use.

But I don't see any dire need for such a feature.


> In povr's f_clock() the up front returned time comes from steady_clock 
> instead - which on my Ubuntu 20.04 machine appears to be a monotonic 
> clock since the last reboot/boot.

That's what `steady_clock` is for. If it wasn't monotonic, it would 
violate the C++ specs.

But being monotonic and steady, it is obviously no good as a building 
block for getting the time of day as a string (and conversion to an 
absolute date may also be a pain).


> Of note for POV-Ray v3.8 and onward is that the Fn_TimePt00, Fn_TimePt00 
> method for capturing time points should work in v3.8 beta 1, but it 
> doesn't because when 'now' was added, the support for it was not added 
> to parser_strings.cpp alongside pi, tau, etc. This a stand alone 'now' 
> issue.

Then that sounds like a bug to me, or at least an issue.


> Putting more of the whole plan/implementation on the table because I see 
> secondary 'concerns' such as POV-Ray's Y2K epoch not lining up with any 
> unix like system I've ever used - all have been at Jan 1, 1970. This 
> makes understanding the values POV-Ray is generating and using harder to 
> work with.

This was a deliberate choice, based on various considerations:


* The value was to be exposed to the user, allowing them to assign it to 
other variables, as well as perform arithmetics on it (such as adding a 
day or an hour before converting to a date/time string, or computing the 
number of minutes the parsing took and inserting that value as text in 
the image; or computing a value from scratch to be later represented as 
a date/time string).

This inevitably meant that the format would have to be based on a 
floating-point representation.

This in turn meant that the maximum possible absolute precision would 
not be constant, but vary depending on the distance from whatever "zero 
point" we would choose. So it would be advantageous to choose a "zero 
point" close to the time we live in.


* The value should have a precision of seconds in the worst case, but 
higher precision if possible.


* With POV-Ray being a platform-agnostic project at heart, I did not 
want to give preference for any existing platform-specific time 
representation format - neither POSIX' native seconds-since-1970 
timestamp format, nor Windows' native 100ns-intervals-since-1601 format. 
The only format I would have considered would have been a format defined 
as part of the C/C++ standard. Had I followed that route, I would have 
ended up using 1900 as the basis, as that's the only epoch year that had 
been explicitly specified by the official C/C++ standard at the time 
(see the `std::tm` structure). The POSIX epoch hasn't made it into the 
official C/C++ standard until C++20, and even then only for non-C 
features for now.

Also remember that back then (2013 or earlier), POSIX time wasn't nearly 
as ubiquitous as it is today.


* I did want a format that would be easy for users to understand, and

* I specificially did want a format that would be easily convertible to 
POSIX time.

While the latter would seem to clearly speak in favor of POSIX time 
itself, in conjunction with the previous point it actually speaks 
AGAINST it; here's why:

POSIX time is often cited as being the number of seconds that have 
elapsed since 1970-01-01 00:00 UTC.
That is NOT the case.
As a matter of fact, POSIX time is currently about 30 seconds short of 
that value.

Instead, POSIX time is the number of _days_ that have elapsed since that 
point in time, plus the number of seconds that have elapsed since 00:00 
UTC of the current (UTC) day.
(Modulo one or two other minor quirks.)
The difference being that this method does not include leap seconds.

By specifying the format of `now` to give a number of days, rather than 
seconds, we're avoiding this subtle difference between what peopole 
_think_ POSIX time is, and what it _actually_ is.
(As a minor snag, on days with leap seconds we're actually off by up to 
1 second in the value reported, but those days are rare.)


Now with using days instead of seconds for the `now` format, some math 
would be required to convert to or from POSIX time anyway; so 
introducing a constant offset in addition to the seconds-per-day factor 
was anything but a big deal.

Year 2000 made for a far nicer epoch than 1970, or 1961, or even 1900. 
And it was closer to "home", meaning we'd have more headroom in terms of 
precision for longer time in the future.

(Year 2000 also had the benefit that UTC was already defined in its 
current form then. In 1970 it wasn't, so the POSIX time value 0 is 
somewhat ambiguous. Not that it matters, as current dates are still 
well-defined.)


* Speaking of conversion, If it would turn out that users wanted to 
convert to or from POSIX time frequently, enough so that it would 
warrant support from our side, the format choisen would have made it 
trivial to provide an official include file to define a function or 
macro to convert between the two formats. And other then-competing 
formats besides.


> Unix commands like date have some nice options:
> 
> date --date='TZ="America/Los_Angeles" 09:00 next Mon' +"%s"
> 
> but we cannot use the results directly in POV-Ray. Hmm, maybe an 
> explicit unix time to y2k seconds type for f_clock...

What's wrong with a simple conversion function?

Why put all the burden on the building block that has the primary job of 
getting the current date and time in _some_ format?

That's a difficult enough job as it is. Leave conversion stuff to 
separate dedicated functions. It's also a difficult enough job.

There is _no_ reason whatsoever to mix them. Just as there is _no_ 
reason whatsoever to cram `now` and `datetime` in a single monolithic 
function, as the original design would have had it.


> Aside:
> Supposing datetime fixed to support %z %Z, and 'now' returning always a 
> no dst utc or local time you can fairly easily adjust the fractional day 
> value to account for dst of say an hour with:
> 
> #declare DSToffset = (1/24);

Yeah, no.
I'm already working on a solution that is far simpler for the user.


> (1) - The C++/Boost (plus time.h etc) date/time options go on forever. I 
> don't begin to understand them all and it looks like c++20 is aiming to 
> revamp/extend a lot of it. Coding wise. I tried to stick mostly with c.

Yeah, it makes for fun reading. Especially the parts where you go, 
"yeah, you got THIS functionality here, and THAT functionality there, 
but all of this doesn't give me THAT THING I REALLY WANT, which should 
be far more trivial to implement than THIS and THAT?! And probably just 
as common a use case."

Case in point:

There are two functions to convert `std::time_t` to a `std::tm` 
structure: One giving you the UTC representation, the other giving you 
the local time representation.

There is also a function to convert a `std::tm` in local time 
representation to `time_t`.

But where is the **** function that will allow me to populate a 
`std::tm` with a UTC representation and convert *THAT* to `time_t`? That 
would have to be even easier to implement!
So how else do I get a `time_t` representation of 2000-01-01 00:00 UTC?!

And no, just casting 30 years' worth of seconds to `time_t` is NOT the 
proper way to do it folks, because `time_t` is NOT guaranteed to use 
POSIX representation. Not even in C++20.


I've now decided that the cleanest way of doing it (in pure C++11 
without boost anyway) may be to convert 2000-01-01 00:00 local time to 
`time_t`, convert that into UTC, convert that back again to `time_t` 
pretending it to be local time, compute the difference to the first 
`time_t` obtained, and subtract the result from that first `time_t`.

Behold - 2000-01-01 00:00 UTC as `time_t`. Easy as pi. Or half a tau. 
Anyway, irrational. Transcendental even.

Post a reply to this message

Am 11.08.2021 um 14:57 schrieb William F Pokorny:

> Of note for POV-Ray v3.8 and onward is that the Fn_TimePt00, Fn_TimePt00 
> method for capturing time points should work in v3.8 beta 1, but it 
> doesn't because when 'now' was added, the support for it was not added 
> to parser_strings.cpp alongside pi, tau, etc. This a stand alone 'now' 
> issue.

I think you mean `parser_functions.cpp`?

There's a point to be made in keeping it that way: If we allow `now` in 
a function, what value should it evaluate to? The point of time when the 
function is parsed and compiled, or the one when it is invoked?

Since the answer to both alternatives is probably "yes" (depending on 
the use case), allowing it at all in functions _will_ cause confusion.

For e.g. `pi` and `tau`, by contrast, the answer is easy: "Who cares, 
it's the same either way."


Also, if you design a function to return current date and time encoded 
in a different time format (say, POSIX timestamp), it's easy to just add 
another parameter, and pass `now` when invoking that function. (*)

And while a function that evaluates to a different value upon each 
invocation during render _may_ sound like a fun idea, I'm quite sure its 
applicability is pretty much limited to being a novelty.


(*Speaking of extra parameters: Passing the number of seconds in a day 
as a _parameter_...??!)

Post a reply to this message

On 8/11/21 2:07 PM, clipka wrote:
> If people want precise performance metrics, anything based on a real 
> time clock - even `std::chrono::steady_clock` - is a poor choice of tool 
> anyway. In general, I would guess that the task/thread scheduling jitter 
> introduces far more noise into the measurements than the occasional 
> system clock adjustment. CPU time would be a far better candidate for 
> such things.
> 
> 
> Also note that for the purpose of performance metrics, we already 
> provide measurements of the total parsing time, both real time and CPU 
> time.
> 
> If you think we should provide more tools for performance measurements, 
> I would argue that some `#split_time` directive to print out the current 
> parser stats would make more sense than some basic building block that 
> users would have to build additional SDL code around, that in turn would 
> potentially skew the results, to actually put to use.
> 
> But I don't see any dire need for such a feature.

First, thanks for the detailed response.

I remain of the opinion POV-Ray would be better positioned with two 
'now's with a 'now_for_datetime' and a 'now_for_timing'.

A now_for_datetime can be sloppy to the order most concerns we've 
pointed out are OK. It could always use the local machine's epoch and 
local setting - aligning with other local tools in the machine/OS 
environment. Something easier for users to use date time wise.

As to timing things inside the parser. My initial reaction was similar 
to yours. The idea rattled around a while inside the cavity behind my 
eyes, and, I started to think that certain sorts of timing 
characterizations are likely simpler/better-done within the parser.

The parser, in being single threaded, is at some advantage timing wise 
for using only a one thread.

---
Something which gets asked every now and again, is whether POV-Ray could 
provide some idea as to the relative expense of certain features over 
others.

Instead of timing some block of user SDL code, imagine instead a parser 
only job which first characterizes the cost of an empty loop and - to 
the degree it can - other "wrapper" costs. We then start inserting 
things to characterize in that loop/wrapper. Single patterns in turn as 
called via functions. Functions themselves one at a time. Or trace() 
operations against base shapes/surfaces one at a time. Or particular 
parser functions one a time. Or whatever else. In many respects the 
parser can more easy isolate particular functionality than can be 
accomplished via other methods.

Imagine a large, long running, parser characterization job for all the 
'things' for which we want to track relative performance. The 
information from which we, perhaps, stick in a big table via jr's new 
macro and publish. Regular timing results useful to users and developers.

This the sort of parser timing application which I believe requires a 
more stable and dedicated 'now_for_timing'. But, yep, it's just another 
idea at this point - except for now having povr's timer specific 
f_clock() feature in hand.

Bill P.

Post a reply to this message

On 8/12/21 7:59 AM, clipka wrote:
> Am 11.08.2021 um 14:57 schrieb William F Pokorny:
> 
>> Of note for POV-Ray v3.8 and onward is that the Fn_TimePt00, 
>> Fn_TimePt00 method for capturing time points should work in v3.8 beta 
>> 1, but it doesn't because when 'now' was added, the support for it was 
>> not added to parser_strings.cpp alongside pi, tau, etc. This a stand 
>> alone 'now' issue.
> 
> I think you mean `parser_functions.cpp`?

I did, sorry.

As for what you say below... We're not going to agree. Plus, I need to 
run to other stuff for the day.

The token 'now' is what it is when the parse sees it. It's no different 
than 'pi' in that regard. Why create inconsistent behavior for users?

> 
> There's a point to be made in keeping it that way: If we allow `now` in 
> a function, what value should it evaluate to? The point of time when the 
> function is parsed and compiled, or the one when it is invoked?
> 
> Since the answer to both alternatives is probably "yes" (depending on 
> the use case), allowing it at all in functions _will_ cause confusion.
> 
> For e.g. `pi` and `tau`, by contrast, the answer is easy: "Who cares, 
> it's the same either way."
> 
> 
> Also, if you design a function to return current date and time encoded 
> in a different time format (say, POSIX timestamp), it's easy to just add 
> another parameter, and pass `now` when invoking that function. (*)
> 
> And while a function that evaluates to a different value upon each 
> invocation during render _may_ sound like a fun idea, I'm quite sure its 
> applicability is pretty much limited to being a novelty.
> 
> 
> (*Speaking of extra parameters: Passing the number of seconds in a day 
> as a _parameter_...??!)

Post a reply to this message

Am 12.08.2021 um 14:16 schrieb William F Pokorny:

> A now_for_datetime can be sloppy to the order most concerns we've 
> pointed out are OK. It could always use the local machine's epoch and 
> local setting - aligning with other local tools in the machine/OS 
> environment. Something easier for users to use date time wise.

Um - nope.

Juging from my experience with trying to get the `datetime` and `now` 
stuff sorted out, a `now` based on local time will only be a source of 
utter confusion.

What would that value even represent?

The only sensible formulation is a "time units since D-Day H-Hour".

Such a D-Day H-Hour would have well-defined DST semantics, either 
explicitly by definition or implicitly by user expectation. If D-Day was 
2000-01-01, H-Hour was 00:00, and the value was advertised as local 
time, then users would expect H-Hour to imply local non-DST in the 
northern hemisphere, local DST in the southern hemisphre, or non-DST if 
their time zone does not observe DST at all.

So if some user were to compute a point in time of, say, 2000-07-01 
00:00 local time - would that be D-Day H-Hour plus exactly 182*24 hours, 
or 1 hour more or less due to DST?


That won't fly. Not in the "easier for the user" sort of way, at any rate.

And certainly not in the "easier for the programmer" sort of way either, 
that much I can guarantee already.


> Imagine a large, long running, parser characterization job for all the 
> 'things' for which we want to track relative performance. The 
> information from which we, perhaps, stick in a big table via jr's new 
> macro and publish. Regular timing results useful to users and developers.

And that will be utterly useless if the functionality to actually 
implement the timing measurements takes considerable time compared to 
what we actually want to measure.

For such an endeavor, things need to have the lightest footprint in 
parser itself that we can possibly create. Ideally, it would just be a 
single (and possibly even very short) token. What that statement would 
trigger can be a bit more complex, because it will be compiled, rather 
than having to be parsed.

Possibly a job for a dedicated build, that adds a single-character token 
just for the purpose, and which isn't functional in release builds. "@" 
isn't used anywhere, for example.

This could trigger one or more performance metrics to be logged whenever 
"@" is encountered, and the result dumped once parsing (or rendering) 
has completed. Maybe a fixed number of characters to follow (say, 4), to 
be logged along with it. The resulting dump could then be correlated 
with the scene later.

If you want precise results, that would be the road to give you the most 
reliable data.

Post a reply to this message