[teqc] some sort of sub-second bug for '-st' option

Lou Estey lou at unavco.org
Thu Oct 6 14:30:10 MDT 2016


Ok, I think I have a generalized fix for this bug, which should work for windowing using
'-st', '-e', and/or '-dX' or '+dX'.  The fix was tested on 10-, 25-, 50-, and 100-Hz data
using '-st' and '-e', all starting and ending on non-integer second epochs.  (And using a
few different formats, depending on the rate.)

The fix, of course, will be in the next official version of teqc.

cheers,
--lou

On 06-Oct-16 07:42 AM, Lou Estey wrote:
> This bug has been tracked down to floating point noise, not a logic error.  There are
> two values to be compared, the second's value for the window start and that for the
> epoch: the window value is created from reading the input string from '-st' using a C
> sscanf() and using a '%lf' for the second's value; whereas the epoch values here depend
> on numerical manipulation, often (as in the case of SBF below and other formats)
> using integer week values and seconds/milliseconds/microseconds (into the week)
> integer values, all of which are then converted to floating point.  In the case below,
> here's a closer look at the actual floating point values of the second's value, printed
> out to higher precision:
>
> window start = 59.159999999999997 seconds
> first epoch =  59.159999999974389 seconds
>
> ... both annoyingly close to exactly 59.16, but not quite, and they are not equal:
> the first epoch is less than the window start, so, from a floating-point perspective,
> the first epoch is before the window start and needs to be skipped -- which of course
> is wrong.  As a test, rounding both to the nearest 0.1 nanosecond (1e-10 second) does
> the trick in this particular case:
>
> rounded window start = 59.160000000000004 seconds
> rounded first epoch  = 59.160000000000004 seconds
>
> ... so now they are both equal and all is well.
>
> Obviously a numerical tolerance has to be introduced, but this will take some thought.
>
> Not that most of you need to know all this, but maybe this gives you some idea of the
> nit-picking details that have to be considered in numerical code. :)
>
> cheers,
> --lou
>
> On 05-Oct-16 01:28 PM, Lou Estey wrote:
>> All,
>>
>> Discovered: There is some sort of sub-second bug for the '-st' option
>> which skips one or more initial epochs if the starting epoch of the input
>> is not on an integer second and the '-st' option specifies the exact starting
>> epoch.  Example using a 25-Hz SBF data file:
>>
>> [12357] teqc +quiet +meta *.sbf | grep ^start
>> start date & time:       2012-06-29 10:04:59.160
>>
>> Now:
>>
>> [12358] teqc +quiet -st 2012-06-29T10:04:59.160 *.sbf > tmp.obs; teqc +meta tmp.obs | grep ^start
>> start date & time:       2012-06-29 10:04:59.200
>>
>> ... which misses the first epoch.  But using slightly earlier '-st' times works aok:
>>
>> [12359] teqc +quiet -st 2012-06-29T10:04:59.120 *.sbf > tmp.obs; teqc +meta tmp.obs | grep ^start
>> start date & time:       2012-06-29 10:04:59.160
>> [12360] teqc +quiet -st 2012-06-29T10:04:59.080 *.sbf > tmp.obs; teqc +meta tmp.obs | grep ^start
>> start date & time:       2012-06-29 10:04:59.160
>> [12361] teqc +quiet -st 2012-06-29T10:04:59.040 *.sbf > tmp.obs; teqc +meta tmp.obs | grep ^start
>> start date & time:       2012-06-29 10:04:59.160
>> [12362] teqc +quiet -st 2012-06-29T10:04:59.000 *.sbf > tmp.obs; teqc +meta tmp.obs | grep ^start
>> start date & time:       2012-06-29 10:04:59.160
>>
>> For now the workaround seems to be to not include the start time with '-st' at all:
>>
>> [12363] teqc +quiet *.sbf > tmp.obs; teqc +meta tmp.obs | grep ^start
>> start date & time:       2012-06-29 10:04:59.160
>>
>> ... or use a start time with '-st' which truncates the sub-second portion.
>>
>> This bug not only occurs when translating, but when doing other operations
>> such as using '-O.sum .'.
>>
>> cheers,
>> --lou
>>
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> Louis H. Estey, Ph.D.              office:  [+001] 303-381-7456
>> UNAVCO, 6350 Nautilus Drive           FAX:  [+001] 303-381-7451
>> Boulder, CO  80301-5554            e-mail:  lou  unavco.org
>>      WWW:  http://www.unavco.org   http://jules.unavco.org
>>
>> "If the universe is the answer, what is the question?"
>>                                                -- Leon Lederman
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



More information about the teqc mailing list