[teqc] helpful tip of week 1965

Lou Estey lou at unavco.org
Thu Sep 7 10:24:49 MDT 2017

This week's tip: handling RINEX comment lines

A RINEX comment line, no matter where it is, has the following format, as described
in https://igscb.jpl.nasa.gov/igscb/data/format/rinex211.txt for RINEX 2.11
and for earlier RINEX specs:

  |    HEADER LABEL    |               DESCRIPTION                |   FORMAT   |
  |  (Columns 61-80)   |                                          |            |
*|COMMENT             | Comment line(s)                          |     A60    |*

(The asterisk means this is an optional line.)  A comment line might occur in the
header of any RINEX file and it is also allowed to occur after the header in a RINEX
observation file along with, possibly, other lines usually found in the header,
as long as these "special record" lines are all immediately proceeded by another
line with an epoch flag value of 2-5 followed by the number of lines to follow,
up to a maximum of 999 lines:

  |             | If epoch flag  2-5:                             |            |
  |             |                                                 |            |
  |             |   - Event flag:                                 |  [2X,I1,]  |
  |             |     2: start moving antenna                     |            |
  |             |     3: new site occupation (end of kinem. data) |            |
  |             |        (at least MARKER NAME record follows)    |            |
  |             |     4: header information follows               |            |
  |             |     5: external event (epoch is significant,    |            |
  |             |        same time frame as observation time tags)|            |
  |             |                                                 |            |
  |             |   - "Number of satellites" contains number of   |    [I3]    |
  |             |     special records to follow.                  |            |
  |             |     Maximum number of records: 999              |            |

Note that there is a limit of 60 characters of comment text per comment line, or only 59
or fewer characters if you want at least one character of whitespace before 'COMMENT',
which itself must start at the 61st character in the line.

Without teqc, one could just use the UNIX/Linux utility `grep` to find if there are
any lines in a RINEX file with 'COMMENT'.  With teqc, you can read a RINEX file
using the '++config' option, e.g. from `teqc +help`:

    ++config             output all known parameters as a superset configuration to stdout

and then look for all "teqc configuration lines" that start with '-O.c' (e.g. see
more about teqc configs in the first tip of week
https://postal.unavco.org/pipermail/teqc/2016/002067.html).  The following shows how
comment lines are specified with this teqc option:

[6585] teqc ++config tmp.obs | grep -- -O.c
-O.c[omment] "Solaris 5.10|UltraSparc IIIi|cc -m64 SS12.1|=+|*Sparc"
-O.c[omment] "teqc  2017Aug17     Lou Estey           20170817 13:38:26UTC"
-O.c[omment] "Solaris 5.10|UltraSparc IIIi|cc -m64 SS12.1|=+|*Sparc"
-O.c[omment] " SNR is mapped to RINEX snr flag value [0-9]"
-O.c[omment] "  L1 & L2: min(max(int(snr_dBHz/6), 0), 9)"

In this case, I'm reading a RINEX observation file and teqc is using '-O.c' to specify
that the text shown was found in a 'COMMENT' line in the observation file(s) being read.
If this had been a RINEX navigation file, '-N.c' would have been used for its comments
and if this had been a RINEX meteorological file, '-M.c' would have been used for its
comments.  Notice that double-quotes are used to define the text for each line, including

To include additional comment lines, either to an existing RINEX file being read or during
translation, use one or more of:

+O.c "<text>" -- for a RINEX observation file
+N.c "<text>" -- for a RINEX navigation file
+M.c "<text>" -- for a RINEX meteorological file

All '+O.c', '+N.c', '+M.c' lines that are specified will be written to the respective
RINEX file header and will follow any existing comment lines or any that teqc will
include automatically.

Note: You are able to include a double-quote character in the text, if desired,
by using a back-slash '\', e.g.

[6587] teqc +O.c "now including \" in comment text" tmp.obs | grep COMMENT
Solaris 5.10|UltraSparc IIIi|cc -m64 SS12.1|=+|*Sparc       COMMENT
teqc  2017Aug17     Lou Estey           20170817 13:38:26UTCCOMMENT
  SNR is mapped to RINEX snr flag value [0-9]                COMMENT
   L1 & L2: min(max(int(snr_dBHz/6), 0), 9)                  COMMENT
now including " in comment text                             COMMENT


Now, what about merging or splicing RINEX files, say, hourly RINEX into daily RINEX?
Each RINEX file that is being merged has its own header which might have its own
comments.  What happens to those comments in the final merged file?

For RINEX navigation and meteorological files, such lines are ignored by teqc for
the output because these RINEX file types do not allow for "post-header" lines after
the header.

For RINEX observation files, any header information that has changed between files
(e.g. antenna position) will be output as post-header lines in the final RINEX file.
Also included will be all the comment lines that were in each header of the files
being merged.  This is because the default operation of teqc is:

    +phc                 output all RINEX post-header comments (default)

You can eliminate these comments in post-header fields by using '-phc':

    -phc                 suppress RINEX post-header comments

However, even when using '-phc', even though post-header comment lines from the
RINEX files being merged will not appear, there will be a set of two lines at each
splice point, one of which is a comment line added by teqc:

                             4  1
other post-header comments skipped                          COMMENT

These two lines are completely valid RINEX formatting, as per the specification noted
at the beginning of this email.  There is no teqc option for removing these two lines.
The reason for these two lines is to identify, for all time, that a splice between two
RINEX observation files took place here.

This is because the general structure of RINEX observation files is sufficiently complex,
given epoch times and phase/pseudorange variables (of which there are two styles of presentation,
to be covered in a future 'tip of the week') and the variety of "epoch" flags (values 0-6)
that could show up in all sorts of combinations.  Considering all the possibilities,
errors in the final merged file are a distinct possibility, even though teqc has been
tuned through the years to hopefully minimize such errors.

Therefore, the two lines shown above at each splice point when merging RINEX obs files
with '-phc' stay.  (So please don't ask me again about these two lines when using the
'-phc' option; I'll just remind you to reread this tip of the week.)

Happy teqc-ing!


Louis H. Estey, Ph.D.              office:  [+001] 303-381-7456
UNAVCO, 6350 Nautilus Drive           FAX:  [+001] 303-381-7451
Boulder, CO  80301-5554            e-mail:  lou  unavco.org

"If the universe is the answer, what is the question?"
                                                -- Leon Lederman

Past helpful tips:

week 1894: using teqc config files - http://postal.unavco.org/pipermail/teqc/2016/002067.html
week 1895: qc of high-rate data - http://postal.unavco.org/pipermail/teqc/2016/002071.html
week 1896: UNIX/Linux shells for Windows - http://postal.unavco.org/pipermail/teqc/2016/002072.html
week 1897: '-' vs. '+' teqc options - http://postal.unavco.org/pipermail/teqc/2016/002076.html
week 1898: auto-identification of formats - http://postal.unavco.org/pipermail/teqc/2016/002092.html
week 1899: auto-identification vs. format flags - http://postal.unavco.org/pipermail/teqc/2016/002096.html
week 1900: square brackets in options - http://postal.unavco.org/pipermail/teqc/2016/002105.html
week 1901: using option '+mds' - http://postal.unavco.org/pipermail/teqc/2016/002108.html
week 1902: qc results w/ problematic nav messages - http://postal.unavco.org/pipermail/teqc/2016/002113.html
week 1903: '-no_orb[it]' and '-no_pos[ition]' options - http://postal.unavco.org/pipermail/teqc/2016/002115.html
week 1904: '-week' option - http://postal.unavco.org/pipermail/teqc/2016/002117.html
week 1905: using '+bcf' for XYZ/geodetic conversion - http://postal.unavco.org/pipermail/teqc/2016/002126.html
week 1906: the '+v[erify]' option - http://postal.unavco.org/pipermail/teqc/2016/002128.html
week 1907: '+C2', '+L5', "+L6', '+L7', '+L8', '+all' options - http://postal.unavco.org/pipermail/teqc/2016/002130.html
week 1908: getting RINEX doppler and L2 - http://postal.unavco.org/pipermail/teqc/2016/002131.html
week 1909: using paths w/ file names - http://postal.unavco.org/pipermail/teqc/2016/002132.html
week 1910: the (un)importance of file names - http://postal.unavco.org/pipermail/teqc/2016/002133.html
week 1911: notices, warnings, and errors - http://postal.unavco.org/pipermail/teqc/2016/002134.html
week 1912: the '-max_rx_SVs' option - http://postal.unavco.org/pipermail/teqc/2016/002137.html
week 1913: the end of '++igs' and '+igs' - http://postal.unavco.org/pipermail/teqc/2016/002140.html
week 1914: splicing together RINEX files - http://postal.unavco.org/pipermail/teqc/2016/002144.html
week 1915: using '-O.int' and '-O.dec' - http://postal.unavco.org/pipermail/teqc/2016/002145.html
week 1916: '+doy' option - http://postal.unavco.org/pipermail/teqc/2016/002146.html
week 1917: '-tbin' and '-ast' options - http://postal.unavco.org/pipermail/teqc/2016/002152.html
week 1918: mp12 RMS before/after Oct 2013 - http://postal.unavco.org/pipermail/teqc/2016/002158.html
week 1919: the global windowing options - http://postal.unavco.org/pipermail/teqc/2016/002159.html
week 1920: '-M.dec' and '-N.dec' options - http://postal.unavco.org/pipermail/teqc/2016/002163.html
week 1921: combining time filtering options - http://postal.unavco.org/pipermail/teqc/2016/002176.html
week 1922: helping me (or someone else on the list) help you - http://postal.unavco.org/pipermail/teqc/2016/002187.html
week 1923: the "build" line - http://postal.unavco.org/pipermail/teqc/2016/002190.html
week 1924: the qc '-w[idth]' option - http://postal.unavco.org/pipermail/teqc/2016/002193.html
week 1925: qc with explicit time windowing - http://postal.unavco.org/pipermail/teqc/2016/002194.html
week 1926: the '+rx_state' option - http://postal.unavco.org/pipermail/teqc/2016/002200.html
week 1927: the '-O.sum' option - http://postal.unavco.org/pipermail/teqc/2016/002204.html
week 1928: the '+meta' and '+mds' options - http://postal.unavco.org/pipermail/teqc/2016/002206.html
week 1930: more on '-O.sum' - http://postal.unavco.org/pipermail/teqc/2017/002207.html
week 1931: the '-O.s[ystem]' option - http://postal.unavco.org/pipermail/teqc/2017/002208.html
week 1932: leap seconds - http://postal.unavco.org/pipermail/teqc/2017/002215.html
week 1936: what you can (and shouldn't) do in a RINEX obs file - http://postal.unavco.org/pipermail/teqc/2017/002229.html
week 1938: the '+psp' option - http://postal.unavco.org/pipermail/teqc/2017/002231.html
week 1939: the '+diag' option - http://postal.unavco.org/pipermail/teqc/2017/002235.html
week 1951: '-n_<system>' and SV filtering options - http://postal.unavco.org/pipermail/teqc/2017/002277.html
week 1953: more with '+diag' option - http://postal.unavco.org/pipermail/teqc/2017/002287.html
week 1954: using '+diag' output to split raw files - http://postal.unavco.org/pipermail/teqc/2017/002290.html
week 1955: current qc notation - http://postal.unavco.org/pipermail/teqc/2017/002302.html
week 1956: the '+qcq' option - http://postal.unavco.org/pipermail/teqc/2017/002304.html
week 1957: using Trimble formats - http://postal.unavco.org/pipermail/teqc/2017/002305.html
week 1958: ToC != ToE messages - http://postal.unavco.org/pipermail/teqc/2017/002310.html
week 1959: receivers vs. formats - http://postal.unavco.org/pipermail/teqc/2017/002311.html
week 1960: when the '-week' option is very wrong to use - https://postal.unavco.org/pipermail/teqc/2017/002314.html
week 1961: "less" is usually best - https://postal.unavco.org/pipermail/teqc/2017/002315.html
week 1962: using GPS L2C with teqc - https://postal.unavco.org/pipermail/teqc/2017/002316.html
week 1964: the '+eds' options - https://postal.unavco.org/pipermail/teqc/2017/002317.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://postal.unavco.org/pipermail/teqc/attachments/20170907/7edf5196/attachment-0001.html>

More information about the teqc mailing list