[teqc] helpful tip of week 1925

Lou Estey lou at unavco.org
Wed Nov 30 08:04:32 MST 2016


This week's tip: explicitly time-windowed qc and the SWN line

This week's tip involves understanding what extra output you will see when you do
am explicitly time-windowed quality check (qc).  If you were paying close attention
to some of the output from last week's tip, then you would have noticed part of
the extra output: the SWN line.  But first, some background:

Before today's era of permanent stations with (more or less) continuous GNSS data,
data collection was episodic.  Any given data file might have an arbitrary start and
end time, depending on when the operator could set up for data collection. The goal
of the data quality check was to qc the data collected, meaning that the start and
the end of the data served as the bounds of the qc analysis. In other words, there
was no point in worrying about (or providing any quality check statistics on) data
that was not collected outside of that time window.

With the arrival of many permanent station sites that are supposed to have continuous
GNSS data, there arose a need for something more: the qc analysis of data that was
not collected but could have been collected under ideal conditions.  If I recall
correctly, the seed for this modification in teqc was planted some time in 2006 or
early 2007 by Giovanni Sella, who helps manage the CORS network for the US National
Geodetic Survey (NGS); I've had something in place since August 2008.

The standard qc in teqc has and still does fulfill part of this latter requirement
in that it accounts for any gaps in data that might occur between the start and end
of the collected data.  However, it does not account for missing data beyond the
start or end of the collected data.  But an accounting of missing data beyond the
start or end of the collected data is done if you explicitly extend the time window.

The basic idea of a time-windowed qc is easily demonstrated.  First, let's look
at some lines from a qc of 20 hours of collected data from a site without any explicit
time-windowing:

[4593] teqc +mds tmp.obs
2016-11-15 00:00:00  2016-11-15 19:59:30   3009492  tmp.obs

[4595] teqc +quiet +qcq tmp.obs
...
  SV+----------|----------|----------|----------|----------|---------|-------+ SV
   5|o+_                          _L+oooooooooooooooo+_                      |  5
  26|o++_                        _Lmoooooooooo+_                         _L+o| 26
  16|oooo--_                        _L+oooooooooo+L_                         | 16
  21|ooooo++_                            _L+oooooooooooo+_                   | 21
  20|oooooom_                           _Loooooooooooooooo++_                | 20
...
Clk|                                                                        |Clk
    +----------|----------|----------|----------|----------|---------|-------+
00:00:00.000                                                        19:59:30.000
2016 Nov 15                                                          2016 Nov 15
...
Time of start of window : 2016 Nov 15  00:00:00.000
Time of  end  of window : 2016 Nov 15  19:59:30.000
Time line window length : 19.99 hour(s), ticked every 3.0 hour(s)
...
Poss. # of obs epochs   :   2400
Epochs w/ observations  :   2400
...
Possible obs >   0.0 deg:  29132
Possible obs >  10.0 deg:  24096
...
       first epoch    last epoch    hrs   dt  #expt  #have   %   mp1   mp2 o/slps
SUM 16 11 15 00:00 16 11 15 19:59 20.00  30  24096  24039 100  0.28  0.25   6010

i.e. all the usual output for good data.  Now, let's tell teqc that we want a full
24-hour qc, starting at the same time, but now including the missing 4 hours at the
end of the day:

[4596] teqc +quiet +qcq -e 23:59:30 tmp.obs | m
...
  SV+---------|--------|--------|--------|--------|--------|--------|--------+ SV
  26|o+_                    _Lmoooooooo+_                     _Lo------------| 26
   5|o+_                     _Loooooooooooooo+_                    _---------|  5
  16|oooo-_                    _LoooooooooL_                     _-----------| 16
  21|oooo++_                       _Loooooooooo++_                  _--------| 21
  20|ooooom_                      _Looooooooooooo++                   __-----| 20
...
Clk|                                                            ^^^^^^^^^^^^|Clk
    +---------|--------|--------|--------|--------|--------|--------|--------+
00:00:00.000                                                        23:59:30.000
2016 Nov 15                                                          2016 Nov 15
...
Time of start of window : 2016 Nov 15  00:00:00.000
    Last epoch in window : 2016 Nov 15  19:59:30.000
Time of  end  of window : 2016 Nov 15  23:59:30.000
Time line window length : 23.99 hour(s), ticked every 3.0 hour(s)
...
Poss. # of obs epochs   :   2880
Epochs w/ observations  :   2400
...
Poss win obs >   0.0 deg:  34735
Poss win obs >  10.0 deg:  28607
Possible obs >   0.0 deg:  29132
Possible obs >  10.0 deg:  24096
...
       first epoch    last epoch    hrs   dt  #expt  #have   %   mp1   mp2 o/slps
SUM 16 11 15 00:00 16 11 15 19:59 20.00  30  24096  24039 100  0.28  0.25   6010
SWN 16 11 15 00:00 16 11 15 23:59 24.00  30  28607  24039  84  0.28  0.25   6010

The differences and the extra information inform you of what data is missing from (in
this case) the 24-hour window.  Notice that you get both the original SUM line -- bounded
by the data that is present -- and a SWN line -- bounded by your specified time window.

What happens if your specified time window exactly matches that of the data?  In
that case, you'll still get the extra lines like the SWN line, but the information
should match the original qc information, e.g.:

[4599] teqc +quiet +qcq -e 19:59:30 tmp.obs
...
  SV+----------|----------|----------|----------|----------|---------|-------+ SV
   5|o+_                          _L+oooooooooooooooo+_                      |  5
  26|o++_                        _Lmoooooooooo+_                         _L+o| 26
  16|oooo--_                        _L+oooooooooo+L_                         | 16
  21|ooooo++_                            _L+oooooooooooo+_                   | 21
  20|oooooom_                           _Loooooooooooooooo++_                | 20
...
Clk|                                                                        |Clk
    +----------|----------|----------|----------|----------|---------|-------+
00:00:00.000                                                        19:59:30.000
2016 Nov 15                                                          2016 Nov 15
...
Time of start of window : 2016 Nov 15  00:00:00.000
Time of  end  of window : 2016 Nov 15  19:59:30.000
Time line window length : 19.99 hour(s), ticked every 3.0 hour(s)
...
Poss. # of obs epochs   :   2400
Epochs w/ observations  :   2400
...
Poss win obs >   0.0 deg:  29132
Poss win obs >  10.0 deg:  24096
Possible obs >   0.0 deg:  29132
Possible obs >  10.0 deg:  24096
...
       first epoch    last epoch    hrs   dt  #expt  #have   %   mp1   mp2 o/slps
SUM 16 11 15 00:00 16 11 15 19:59 20.00  30  24096  24039 100  0.28  0.25   6010
SWN 16 11 15 00:00 16 11 15 19:59 20.00  30  24096  24039 100  0.28  0.25   6010

If the specified time window is longer than the collected data, as done in the middle qc
above, which qc result should you use?  That depends entirely on what you are trying
to elucidate.  Is it the data that are actually there, as in an episodic data file?
Or is it data that could have been there in an ideal situation, as from a continuous
permanent station?  The choice is yours.

Happy teqc-ing!

cheers,
--lou

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Louis H. Estey, Ph.D.              office:  [+001] 303-381-7456
UNAVCO, 6350 Nautilus Drive           FAX:  [+001] 303-381-7451
Boulder, CO  80301-5554            e-mail:  lou  unavco.org
       WWW:http://www.unavco.org    http://jules.unavco.org

"If the universe is the answer, what is the question?"
                                                -- Leon Lederman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Past helpful tips:

week 1894: using teqc config files - http://postal.unavco.org/pipermail/teqc/2016/002067.html
week 1895: qc of high-rate data - http://postal.unavco.org/pipermail/teqc/2016/002071.html
week 1896: UNIX/Linux shells for Windows - http://postal.unavco.org/pipermail/teqc/2016/002072.html
week 1897: '-' vs. '+' teqc options - http://postal.unavco.org/pipermail/teqc/2016/002076.html
week 1898: auto-identification of formats - http://postal.unavco.org/pipermail/teqc/2016/002092.html
week 1899: auto-identification vs. format flags - http://postal.unavco.org/pipermail/teqc/2016/002096.html
week 1900: square brackets in options - http://postal.unavco.org/pipermail/teqc/2016/002105.html
week 1901: using option '+mds' - http://postal.unavco.org/pipermail/teqc/2016/002108.html
week 1902: qc results w/ problematic nav messages - http://postal.unavco.org/pipermail/teqc/2016/002113.html
week 1903: '-no_orb[it]' and '-no_pos[ition]' options - http://postal.unavco.org/pipermail/teqc/2016/002115.html
week 1904: '-week' option - http://postal.unavco.org/pipermail/teqc/2016/002117.html
week 1905: using '+bcf' for XYZ/geodetic conversion - http://postal.unavco.org/pipermail/teqc/2016/002126.html
week 1906: the '+v[erify]' option - http://postal.unavco.org/pipermail/teqc/2016/002128.html
week 1907: '+C2', '+L5', "+L6', '+L7', '+L8', and '+all' options - http://postal.unavco.org/pipermail/teqc/2016/002130.html
week 1908: no doppler shortcut; RINEX L2 - http://postal.unavco.org/pipermail/teqc/2016/002131.html
week 1909: using paths w/ file names - http://postal.unavco.org/pipermail/teqc/2016/002132.html
week 1910: the (un)importance of file names - http://postal.unavco.org/pipermail/teqc/2016/002133.html
week 1911: notices, warnings, and errors - http://postal.unavco.org/pipermail/teqc/2016/002134.html
week 1912: the '-max_rx_SVs' option - http://postal.unavco.org/pipermail/teqc/2016/002137.html
week 1913: the end of '++igs' and '+igs' - http://postal.unavco.org/pipermail/teqc/2016/002140.html
week 1914: splicing together RINEX files - http://postal.unavco.org/pipermail/teqc/2016/002144.html
week 1915: using '-O.int' and '-O.dec' - http://postal.unavco.org/pipermail/teqc/2016/002145.html
week 1916: '+doy' option - http://postal.unavco.org/pipermail/teqc/2016/002146.html
week 1917: '-tbin' and '-ast' options - http://postal.unavco.org/pipermail/teqc/2016/002152.html
week 1918: mp12 RMS before/after Oct 2013 - http://postal.unavco.org/pipermail/teqc/2016/002158.html
week 1919: the global windowing options - http://postal.unavco.org/pipermail/teqc/2016/002159.html
week 1920: '-M.dec' and '-N.dec' options - http://postal.unavco.org/pipermail/teqc/2016/002163.html
week 1921: combining time filtering options - http://postal.unavco.org/pipermail/teqc/2016/002176.html
week 1922: helping me (or someone else on the list) help you - http://postal.unavco.org/pipermail/teqc/2016/002187.html
week 1923: the "build" line - http://postal.unavco.org/pipermail/teqc/2016/002190.html
week 1924: the qc '-w[idth]' option - http://postal.unavco.org/pipermail/teqc/2016/002193.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://postal.unavco.org/pipermail/teqc/attachments/20161130/c6a1541b/attachment-0001.html>


More information about the teqc mailing list