Add high resolution time support (nanoseconds) to Time::HiRes via a new method: hrtime()

[scottchiefbaker]

I would like to get fast, integer based, timing from Perl. Time::HiRes returns a float, and is not ideal for this kind of thing.

Using CLOCK_GETTIME on Posix systems this PR adds the hrtime() method to Time::HiRes to return a UV of nanosecods. Care was taken to make this similar in style to the other hires functions of this module. It will fail gracefully with a croak() error if CLOCK_GETTIME is not available.

POD documentation has been included with explanations of the epoch, and the counter roll-over.

It has been tested out-of-tree as a standalone module and passes except for one test:

#   Failed test 'Time::HiRes::sleep's prototype matches CORE::sleep's'
#   at ./t/sleep.t line 13.
#          got: ';@'
#     expected: ';$'
# Looks like you failed 1 test of 5.

I didn't touch any of this code, so I suspect this is a side-effect of building out of tree? Not sure what this error is.

See also: PHP's hrtime()

[scottchiefbaker]

For posterity I'm writing down the method I use currently:

sub micros {
	require Time::HiRes;

	my @p   = Time::HiRes::gettimeofday();
	my $ret = ($p[0] * 1000000) + $p[1];

	return $ret;
}

This gets us pretty accurate timing, but we can do a even better.

[Grinnz]

Rather than have it do a multiplication, would it be better to have a function that simply returns the sec and nanosec components in the way gettimeofday does, so no more nanoseconds are spent than necessary?

[scottchiefbaker]

@Grinnz I did consider that as an option. However, every time I need nanoseconds, I need it as an integer. I'd rather do the math in XS than in Perl.

We could add an option to hrtime() function to have it return an array of (seconds, ns) if required. FWIW this is what PHP does.

[Grinnz]

I'm failing to think of a use case where you care about the floating point precision of Time::HiRes::time, but not about doing extra work. (For example if you're using this for comparison, you can compare seconds and then only compare nanoseconds if those are equal.) Whatever the use cases, this should provide the building blocks for the most useful data.

[Grinnz]

Secondary question to that regard: does it need to be limited to CLOCK_MONOTONIC? That restricts its usefulness to timing comparisons. Could it take a clock option like clock_gettime?

[tonycoz]

It has been tested out-of-tree as a standalone module and passes except for one test:

#   Failed test 'Time::HiRes::sleep's prototype matches CORE::sleep's'
#   at ./t/sleep.t line 13.
#          got: ';@'
#     expected: ';$'
# Looks like you failed 1 test of 5.

It's not happening in CI, which tests dist/ modules against a few older perls. Do you see this failure reproducibly?

There is one failure though:

porting/copyright.t ....... ok
# not ok 4 - dist/Time-HiRes/HiRes.pm version 1.9780
porting/cmp_version.t ..... 
Failed 1/5 subtests 
porting/corelist.t ........ ok

which is easily correctable.

Rather than have it do a multiplication, would it be better to have a function that simply returns the sec and nanosec
components in the way gettimeofday does, so no more nanoseconds are spent than necessary?

For a single return value hrtime() can use dXSTARG, which uses the entersub OPs TARG, to return a list we'd need to create a new SV, which would be a little slower.

Integer multiplication is pretty quick (at the C level ) on modern hardware, perl's arithmetic tends to be slower due to all the tests done to preserve integers where possible..

As to the floating point result, I did a simple C test, the floating point result for CLOCK_REALTIME does drop some precision:

# left is integer, right is double
tony@venus:.../perl/git$ ./clock
1: 1781482194.731588567 1781482194.731588602
2: 1781482194.731588734 1781482194.731588840
tony@venus:.../perl/git$ ./clock
1: 1781482197.005435100 1781482197.005434990
2: 1781482197.005435244 1781482197.005435228
tony@venus:.../perl/git$ ./clock
1: 1781482197.777433880 1781482197.777433872
2: 1781482197.777433913 1781482197.777433872
tony@venus:.../perl/git$ ./clock
1: 1781482204.114995376 1781482204.114995480
2: 1781482204.114995511 1781482204.114995480

I don't think this is a bad idea, but:

• I don't know if the name is the best
• like PHP hrtime() I think it should return an NV if IVs are 32-bit
• returning a scalar in scalar context and a list (secs, nsecs) in list context seems reasonable and perlish, though that can bite people.
• I do think it should accept a clock type

[Grinnz]

• returning a scalar in scalar context and a list (secs, nsecs) in list context seems reasonable and perlish, though that can bite people.

I would object to this as it would cause the classic vulnerability when stuck in a hash constructor.

[guest20]

I would object to [returning (secs,nsecs) in list context] as it would cause the classic vulnerability when stuck in a hash constructor.

If it's doing a gettimeofday with microseconds it kinda has to do that, since that's the API.

Also, it's really only a "vulnerability" if the badguy gets flow control. This would just be a regular bug.

[Grinnz]

I would object to [returning (secs,nsecs) in list context] as it would cause the classic vulnerability when stuck in a hash constructor.

If it's doing a gettimeofday with microseconds it kinda has to do that, since that's the API.

We're not talking about gettimeofday, it can have whatever API is appropriate.

Also, it's really only a "vulnerability" if the badguy gets flow control. This would just be a regular bug.

Tell that to bugzilla: https://www.cve.org/CVERecord?id=CVE-2014-1572

[Grinnz]

It causes a switch of keys and values in the list, so the unintended attack vectors could take any number of natures.

[scottchiefbaker]

I modified the function to take an optional boolean param:

my $ns         = hrtime();
my ($sec, $ns) = hrtime(1);

This mimics the behavior of the PHP version.

I did not implement CLOCK_REALTIME which returns time since the Unix epoch. CLOCK_REALTIME is susceptible to changes in the system clock by the user or NTP. Using CLOCK_REALTIME would mean that it's possible for time to go backwards.

For the purposes of "I need nanoseconds for timing some code" we probably want to stick with CLOCK_MONOTONIC so the clock never goes backwards.

[Grinnz]

That presumes that is the only use case for this proposed function. Which is fine, but the purpose of making it configurable would be to make it useful for other use cases.

[chansen]

I'd prefer clock_gettime_ns($which). It's self-explanatory, mirrors clock_gettime(), and naturally extends to clocks beyond wall and monotonic.

[Grinnz]

The sanity test failure is because Time::HiRes needs a version bump, BTW

[chansen-sfr]

If a shorter monotonic-time API is wanted as well, consider adding monotonic_ns, documented as an shortcut for clock_gettime_ns(CLOCK_MONOTONIC).

[haarg]

Looking for prior art, darwin systems have a C function clock_gettime_nsec_np. _np meaning non-portable.

I like the names clock_gettime_nsec($which) or clock_gettime_ns($which) more than hrtime. hr on its own doesn't indicate what units it will return.

[chansen-sfr]

I find clock_gettime_nsec misleading because it can be read as "return the nsec part of a timespec", which is not the behavior of the function.

[guest20]

@chansen-sfr Right, that's why normally when functions are added the authors will include a bit of descriptive text along with it to explain how it works, what it does and what you pass in, what it returns and which units the apply to the returned value.

it returns the number of nanoseconds... that's the nsec "part", no?

[chansen-sfr]

@chansen-sfr Right, that's why normally when functions are added the authors will include a bit of descriptive text along with it to explain how it works, what it does and what you pass in, what it returns and which units the apply to the returned value.

it returns the number of nanoseconds... that's the nsec "part", no?

This is somewhat unrelated to the issue being discussed. POSIX explicitly defines tv_nsec as the nanosecond portion of a second, with values in the range [0, 999999999]. See the <time.h> specification: https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/time.h.html

[scottchiefbaker]

Functionality available on CPAN via Time::Nanos. I'd still like to see it in Core.

[chansen-sfr]

It would be good to agree on the API and return value semantics when UV is not 64-bit. Would it make sense to return an NV?

At current epoch timestamps, nanoseconds since the epoch are on the order of 2^60. For a double, the spacing between adjacent representable values at that magnitude is approximately:

2^(60 - 52) = 256

As a result, an NV used to represent nanoseconds since the epoch would have a resolution of about 256 ns.

[Leont]

Functionality available on CPAN via Time::Nanos.

Or the latest version of POSIX::RT::Clock ;-)

[chansen-sfr]

Updated: added monotonic_ns() and refactored common logic to avoid duplication.

Perhaps something along these lines: returns undef on failure, since 0 is a valid clock value.

diff --git a/dist/Time-HiRes/HiRes.pm b/dist/Time-HiRes/HiRes.pm
index f1fb463582..0420a182aa 100644
--- a/dist/Time-HiRes/HiRes.pm
+++ b/dist/Time-HiRes/HiRes.pm
@@ -382,6 +382,25 @@ C<CLOCK_MONOTONIC>, which guarantees a monotonically increasing time
 value (unlike time() or gettimeofday(), which can be adjusted).
 See your system documentation for other possibly supported values.
 
+=item clock_gettime_ns ( $which )
+
+Like C<clock_gettime()>, but returns the current value of the specified
+POSIX high resolution timer in nanoseconds.
+
+Returns C<undef> on failure.
+
+On systems where Perl's native integer type is 64 bits, the result is
+returned as an integer.  On systems where Perl's native integer type is
+32 bits, the result is returned as a floating-point value because the
+number of nanoseconds cannot be represented in a native integer.
+
+At current epoch timestamps, the number of nanoseconds since the epoch
+is on the order of 2^60.  Since a double-precision floating-point value
+has 53 bits of precision, values represented this way have a granularity
+of approximately 2^(60 - 52), or about 256 nanoseconds.  Therefore,
+C<clock_gettime_ns()> on a 32-bit Perl cannot represent every individual
+nanosecond value.
+
 =item clock_getres ( $which )
 
 Return as seconds the resolution of the POSIX high resolution timer
@@ -429,6 +448,21 @@ but not necessarily identical.  Note that due to backward
 compatibility limitations the returned value may wrap around at about
 2147 seconds or at about 36 minutes.
 
+=item monotonic_ns()
+
+Returns the current value of the POSIX monotonic clock in nanoseconds.
+
+This is equivalent to:
+
+    clock_gettime_ns(CLOCK_MONOTONIC)
+
+The monotonic clock is guaranteed to be monotonically increasing and is
+suitable for measuring elapsed time intervals.
+
+Returns C<undef> on failure.
+
+This function is available only on platforms that support C<CLOCK_MONOTONIC>.
+
 =item stat
 
 =item stat FH
diff --git a/dist/Time-HiRes/HiRes.xs b/dist/Time-HiRes/HiRes.xs
index 0c4fde4258..4cebfeb714 100644
--- a/dist/Time-HiRes/HiRes.xs
+++ b/dist/Time-HiRes/HiRes.xs
@@ -896,6 +896,33 @@ nsec_without_unslept(struct timespec *sleepfor,
 
 #endif
 
+#if defined(TIME_HIRES_CLOCK_GETTIME)
+
+static void _clock_gettime_sv(pTHX_ clockid_t clock_id, bool fractional, SV *dsv) {
+    struct timespec ts;
+    int status;
+
+#   ifdef TIME_HIRES_CLOCK_GETTIME_SYSCALL
+        status = syscall(SYS_clock_gettime, clock_id, &ts);
+#   else
+        status = clock_gettime(clock_id, &ts);
+#   endif
+
+    if (fractional)
+        sv_setnv(dsv, status == 0 ? (NV) ts.tv_sec + (NV) ts.tv_nsec / NV_1E9 : -1);
+    else if (status != 0)
+        sv_setsv(dsv, &PL_sv_undef);
+    else if (UVSIZE == 4)
+        sv_setnv(dsv, (NV) ts.tv_sec * NV_1E9 + (NV) ts.tv_nsec);
+    else
+        sv_setuv(dsv, (UV) ts.tv_sec * (UV) IV_1E9 + (UV) ts.tv_nsec);
+}
+
+#define clock_gettime_sv(clock_id, fractional, dsv) \
+  _clock_gettime_sv(aTHX_ clock_id, fractional, dsv)
+
+#endif
+
 /* In case Perl and/or Devel::PPPort are too old, minimally emulate
  * IS_SAFE_PATHNAME() (which looks for zero bytes in the pathname). */
 #ifndef IS_SAFE_PATHNAME
@@ -1378,22 +1405,19 @@ utime(accessed, modified, ...)
 
 #if defined(TIME_HIRES_CLOCK_GETTIME)
 
-NV
+void
 clock_gettime(clock_id = CLOCK_REALTIME)
     clockid_t clock_id
+    ALIAS:
+        Time::HiRes::clock_gettime = 0
+        Time::HiRes::clock_gettime_ns = 1
     PREINIT:
-        struct timespec ts;
-        int status = -1;
-    CODE:
-#  ifdef TIME_HIRES_CLOCK_GETTIME_SYSCALL
-        status = syscall(SYS_clock_gettime, clock_id, &ts);
-#  else
-        status = clock_gettime(clock_id, &ts);
-#  endif
-        RETVAL = status == 0 ? ts.tv_sec + (NV) ts.tv_nsec / NV_1E9 : -1;
-
-    OUTPUT:
-        RETVAL
+        dXSTARG;
+        bool fractional = ix == 0;
+    PPCODE:
+        clock_gettime_sv(clock_id, fractional, TARG);
+        ST(0) = TARG;
+        XSRETURN(1);
 
 #else  /* if defined(TIME_HIRES_CLOCK_GETTIME) */
 
@@ -1507,6 +1531,26 @@ clock()
 
 #endif /*  #if defined(TIME_HIRES_CLOCK) && defined(CLOCKS_PER_SEC) */
 
+#if defined(TIME_HIRES_CLOCK_GETTIME) && defined(CLOCK_MONOTONIC)
+
+void
+monotonic_ns()
+    PREINIT:
+        dXSTARG;
+    PPCODE:
+        clock_gettime_sv(CLOCK_MONOTONIC, FALSE, TARG);
+        ST(0) = TARG;
+        XSRETURN(1);
+
+#else
+
+void
+monotonic_ns()
+    PPCODE:
+      croak("Time::HiRes::monotonic_ns(): unimplemented in this platform");
+
+#endif /*  #if defined(TIME_HIRES_CLOCK_GETTIME) && defined(CLOCK_MONOTONIC) */
+
 void
 stat(...)
 PROTOTYPE: ;$

[chansen-sfr]

@scottchiefbaker would this ^^^ API work for you?

[scottchiefbaker]

Yeah I think this is fine. I just need nanoseconds as an integer, the rest is all gravy.

[bulk88]

This argument is illogical in the Perl 5 language and shows zero knowledge of Perl coding.

Either call PP wantarray or C GIMME_V to learn this information the PP Grammar correct way.

[bulk88]

XSRETURN_EMPTY should not be used in a PPCODE: block. XSRETURN_FOOs are intended for CODE: blocks where local nor global SP has never been rewound to zero, or the validity/non-SEGV Ness of local var SP is unknown, such as in EU::PXS codegen blocks that can not determine if call_sv() indirectly ever got executed in the body of a XSUB thus causing a possible Perl stack realloc grow event, and the end dev was sloppy or careless in writing SPAGAIN in the correct locations. PPCODE: puts a XSprePuSH: in the codegen at the top, rewinding local SP to 0. "PUTBACK; return;" is all that is needed to return an empty list in a PPCODE block.

[bulk88]

     croak("Time::HiRes::hrtime(): unimplemented in this platform");

Replace with "%s" and Split off string "Time::HiRes::hrtime" into arg 1 of croak(), nobody needs ASCII string "Time::HiRes::hrtime" to appear twice in the .so/.dll file.

[bulk88]

Eliminate EXTEND 1, it's not needed, all XSUBs are guaranteed 1 slot on PL_stack_sp bc pp_entersub removes 1 SV* from the stack to invoke your CV*. The alternative is to do EXTEND 2 on all paths, the Perl stack is incapable of shrinking for the life of the process. If 8 bytes of malloc() space is the difference between the kernel OOM killing Perl and a successful app run, there r much bigger problems going on. It's not worth the code space or CPU wall Time to bother branching between bounds check of 1 and 2. just do 2 always, or check for 2 on the list return path and no check on scalar return path.

[bulk88]

What if UV is size 32bits such as on i386 Perl? You might want type U64 math here or throw a C syntax error. Truncate to 32 bits 3 times silently is a bad idea here if all 3 inputs are size 64b. Even Visual C 6.0 can be beaten with a heavy stick to cough up support for types U64 and I64 .

[scottchiefbaker]

I've been pondering best strategies for how to implement this, the ended up with the methodology in Time::Nanos. When I need ns I use that for simplicity. Perhaps having it on CPAN trumps this? I don't know. Commence bike-shedding.

Alternately I would use the micros() function above that works in CORE today.