[teqc] helpful tip of week 1953

Lou Estey lou at unavco.org
Fri Jun 16 13:28:25 MDT 2017


This week's tip: more on using the '+diag' output

Back in the tip for week 1939 http://postal.unavco.org/pipermail/teqc/2017/002235.html
I said I'd cover a case or two where the '+diag' option was showing something unexpected.
If this occurs, it's usually when reading binary data (i.e. not RINEX).  Here's a quick example:

[753] teqc +diag . +mds av39201706140000a.t00
? Error ? 'av39201706140000a.t00' rejection due to id != xor id
? Error ? 'av39201706140000a.t00' rejection due to id != xor id
? Error ? 'av39201706140000a.t00' rejection due to id != xor id
<a whole bunch more lines like this>
? Error ? 'av39201706140000a.t00' rejection due to id != xor id
? Error ? 'av39201706140000a.t00' rejection due to id != xor id
CMC Binary frame 0x01 @ 0o00000014 = 0x0000000c = 00000012   type= 0x64 = 100
CMC Binary frame 0x01 @ 0o00000022 = 0x00000012 = 00000018   type= 0x09 = 9
CMC Binary frame 0x01 @ 0o00000065 = 0x00000035 = 00000053   type= 0x04 = 4
CMC Binary frame 0x01 @ 0o00000074 = 0x0000003c = 00000060   type= 0x13 = 19
CMC Binary frame 0x01 @ 0o00000102 = 0x00000042 = 00000066   type= 0x02 = 2
CMC Binary frame 0x01 @ 0o00000106 = 0x00000046 = 00000070   type= 0x00 = 0
<a whole bunch more lines like this>

Ok, so this is not what you expected.  First, av39201706140000a.t00 is
a Trimble .t00 file, and teqc can't read Trimble .r00/.t00/.t01/.t02 files.
So even if you had used '-tr d':

[754] teqc -tr d +diag . +mds av39201706140000a.t00
! Notice ! 'av39201706140000a.t00' has Trimble .dat records 0 or 7; no support in teqc ... skipping
ksh: 12201 Bus Error

In the first command, you left it up to teqc to auto-identify the format
(see tip of week 1898: http://postal.unavco.org/pipermail/teqc/2016/002092.html)
... and teqc can't, in this case ... so it identified the format as CMC binary
(which it isn't) and you get a bunch of nonsense.

In the second command, you told teqc that the format was Trimble .dat/.tgd --
which it isn't -- and you get the resultant (in this case) reading disaster.

Now, if I convert av39201706140000a.t00 (which teqc can't read) into av39201706140000a.tgd
(which teqc can read) using Trimble's runpkr00 program and then use 'teqc +diag ...'
on the resultant .dat or .tgd file:

[755] runpkr00 -g -d av39201706140000a.t00
[756] teqc -notice +diag . +mds av39201706140000a.tgd
Trimble .dat frame t @ 0o00000000 = 0x00000000 = 00000000   type= 0x05 = 5
Trimble .dat frame t @ 0o00000030 = 0x00000018 = 00000024   type= 0x05 = 5
Trimble .dat frame t @ 0o00000172 = 0x0000007a = 00000122   type= 0x05 = 5
Trimble .dat frame t @ 0o00000273 = 0x000000bb = 00000187   type= 0x05 = 5
Trimble .dat frame t @ 0o00000373 = 0x000000fb = 00000251   type= 0x05 = 5
Trimble .dat frame t @ 0o00000551 = 0x00000169 = 00000361   type= 0x0f = 15
Trimble .dat frame t @ 0o00000745 = 0x000001e5 = 00000485   type= 0x1e = 30
Trimble .dat frame t @ 0o00001036 = 0x0000021e = 00000542   type= 0x1e = 30
Trimble .dat frame t @ 0o00001074 = 0x0000023c = 00000572   type= 0x1e = 30
Trimble .dat frame t @ 0o00001127 = 0x00000257 = 00000599   type= 0x0c = 12
Trimble .dat frame t @ 0o00002013 = 0x0000040b = 00001035   type= 0x13 = 19
Trimble .dat frame t @ 0o00002031 = 0x00000419 = 00001049   type= 0x10 = 16
...

and now we are back to a normal situation with the expected output from '+diag'.

Ok, what about this example, where the input file is (or should be) the Ashtech stream
format:

[787] teqc +diag . +mds ac12raw.str 2>&1 | m
! Warning ! Ashtech MBEN/DBEN id 0x12: file 'ac12raw.str' checksum= 0x00ff vs. in record= 0x0070 ... skipping record
? Error ? 'ac12raw.str' rejection due to bad checksum
! Warning ! Ashtech MBEN/DBEN id 0x12: file 'ac12raw.str' checksum= 0x007b vs. in record= 0x0050 ... skipping record
? Error ? 'ac12raw.str' rejection due to bad checksum
! Warning ! Ashtech MBEN/DBEN id 0x12: file 'ac12raw.str' checksum= 0x00ce vs. in record= 0x00e6 ... skipping record
? Error ? 'ac12raw.str' rejection due to bad checksum
! Warning ! Ashtech MBEN/DBEN id 0x12: file 'ac12raw.str' checksum= 0x008c vs. in record= 0x001d ... skipping record
? Error ? 'ac12raw.str' rejection due to bad checksum
! Warning ! Ashtech MBEN/DBEN id 0x12: file 'ac12raw.str' checksum= 0x00cd vs. in record= 0x0000 ... skipping record
? Error ? 'ac12raw.str' rejection due to bad checksum
<and more of the same>

(If I had left out the '+diag .', there would just be the Warning messages.)
Here teqc is informing us that the data file is corrupted in some way.  How?  Why?
I don't know.  But basically at least part of the file is useless.  In this case,
not all of it though:

[790] teqc +quiet +mds ac12raw.str
2008-07-02 10:24:30  2008-07-02 10:24:30     36074  ac12raw.str

... indicating that there is one epoch that has data that can be retrieved, but
precious little as it turns out, so removing '+mds':

[791] teqc +quiet ac12raw.str
      2.11           OBSERVATION DATA    G (GPS)             RINEX VERSION / TYPE
teqc  2017Jun16     Lou Estey           20170616 18:07:29UTCPGM / RUN BY / DATE
...
      7    L1    L2    C1    P1    P2    S1    S2            # / TYPES OF OBSERV
...
   2008     7     2    10    24   30.0000000     GPS         TIME OF FIRST OBS
                                                             END OF HEADER
  08  7  2 10 24 30.0000000  0  1G27

          5.000


But that's it.  One measurement of S1 for G27.  Probably not too helpful.

I sometimes get an email from a user who has a file that teqc says is corrupted.
Once we determine that there's no format issue (overriding teqc's auto-identification,
say), and teqc still says the file is corrupt, I inevitably get the next question
from the user, "Well, how can teqc fix the file?"  Well, sorry, but nothing can be done.
This is a bit like handing me a scrambled egg that's been run over by a truck --
yes, I can tell it probably once used to be an egg, but we can't reconstitute it
back to a raw egg.  It is what is at that point.  Trash.

Sometimes the corruption occurs later in the file, with the initial part of the file
in good shape.  Let's say this is a raw binary data file you want to save the good
part of the file so that it can be used in other software that can't cope with the
corrupted part.  Can teqc help with that?   The answer is yes ... for some formats.
This involves using the '+diag' output to identify where the file goes "bad" and
then using a tool that can slice up the binary file to save the "good" part.  I
discuss that in a future tip.

Happy teqc-ing!

cheers,
--lou

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Louis H. Estey, Ph.D.              office:  [+001] 303-381-7456
UNAVCO, 6350 Nautilus Drive           FAX:  [+001] 303-381-7451
Boulder, CO  80301-5554            e-mail:  lou  unavco.org

"If the universe is the answer, what is the question?"
                                                -- Leon Lederman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Past helpful tips:

week 1894: using teqc config files - http://postal.unavco.org/pipermail/teqc/2016/002067.html
week 1895: qc of high-rate data - http://postal.unavco.org/pipermail/teqc/2016/002071.html
week 1896: UNIX/Linux shells for Windows - http://postal.unavco.org/pipermail/teqc/2016/002072.html
week 1897: '-' vs. '+' teqc options - http://postal.unavco.org/pipermail/teqc/2016/002076.html
week 1898: auto-identification of formats - http://postal.unavco.org/pipermail/teqc/2016/002092.html
week 1899: auto-identification vs. format flags - http://postal.unavco.org/pipermail/teqc/2016/002096.html
week 1900: square brackets in options - http://postal.unavco.org/pipermail/teqc/2016/002105.html
week 1901: using option '+mds' - http://postal.unavco.org/pipermail/teqc/2016/002108.html
week 1902: qc results w/ problematic nav messages - http://postal.unavco.org/pipermail/teqc/2016/002113.html
week 1903: '-no_orb[it]' and '-no_pos[ition]' options - http://postal.unavco.org/pipermail/teqc/2016/002115.html
week 1904: '-week' option - http://postal.unavco.org/pipermail/teqc/2016/002117.html
week 1905: using '+bcf' for XYZ/geodetic conversion - http://postal.unavco.org/pipermail/teqc/2016/002126.html
week 1906: the '+v[erify]' option - http://postal.unavco.org/pipermail/teqc/2016/002128.html
week 1907: '+C2', '+L5', "+L6', '+L7', '+L8', '+all' options - http://postal.unavco.org/pipermail/teqc/2016/002130.html
week 1908: getting RINEX doppler and L2 - http://postal.unavco.org/pipermail/teqc/2016/002131.html
week 1909: using paths w/ file names - http://postal.unavco.org/pipermail/teqc/2016/002132.html
week 1910: the (un)importance of file names - http://postal.unavco.org/pipermail/teqc/2016/002133.html
week 1911: notices, warnings, and errors - http://postal.unavco.org/pipermail/teqc/2016/002134.html
week 1912: the '-max_rx_SVs' option - http://postal.unavco.org/pipermail/teqc/2016/002137.html
week 1913: the end of '++igs' and '+igs' - http://postal.unavco.org/pipermail/teqc/2016/002140.html
week 1914: splicing together RINEX files - http://postal.unavco.org/pipermail/teqc/2016/002144.html
week 1915: using '-O.int' and '-O.dec' - http://postal.unavco.org/pipermail/teqc/2016/002145.html
week 1916: '+doy' option - http://postal.unavco.org/pipermail/teqc/2016/002146.html
week 1917: '-tbin' and '-ast' options - http://postal.unavco.org/pipermail/teqc/2016/002152.html
week 1918: mp12 RMS before/after Oct 2013 - http://postal.unavco.org/pipermail/teqc/2016/002158.html
week 1919: the global windowing options - http://postal.unavco.org/pipermail/teqc/2016/002159.html
week 1920: '-M.dec' and '-N.dec' options - http://postal.unavco.org/pipermail/teqc/2016/002163.html
week 1921: combining time filtering options - http://postal.unavco.org/pipermail/teqc/2016/002176.html
week 1922: helping me (or someone else on the list) help you - http://postal.unavco.org/pipermail/teqc/2016/002187.html
week 1923: the "build" line - http://postal.unavco.org/pipermail/teqc/2016/002190.html
week 1924: the qc '-w[idth]' option - http://postal.unavco.org/pipermail/teqc/2016/002193.html
week 1925: qc with explicit time windowing - http://postal.unavco.org/pipermail/teqc/2016/002194.html
week 1926: the '+rx_state' option - http://postal.unavco.org/pipermail/teqc/2016/002200.html
week 1927: the '-O.sum' option - http://postal.unavco.org/pipermail/teqc/2016/002204.html
week 1928: the '+meta' and '+mds' options - http://postal.unavco.org/pipermail/teqc/2016/002206.html
week 1930: more on '-O.sum' - http://postal.unavco.org/pipermail/teqc/2017/002207.html
week 1931: the '-O.s[ystem]' option - http://postal.unavco.org/pipermail/teqc/2017/002208.html
week 1932: leap seconds - http://postal.unavco.org/pipermail/teqc/2017/002215.html
week 1936: what you can (and shouldn't) do in a RINEX obs file - http://postal.unavco.org/pipermail/teqc/2017/002229.html
week 1938: the '+psp' option - http://postal.unavco.org/pipermail/teqc/2017/002231.html
week 1939: the '+diag' option - http://postal.unavco.org/pipermail/teqc/2017/002235.html
week 1951: '-n_<system>' and SV filtering options - http://postal.unavco.org/pipermail/teqc/2017/002277.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://postal.unavco.org/pipermail/teqc/attachments/20170616/2ae36485/attachment-0001.html>


More information about the teqc mailing list