Decoding logic to match the proper category desctiption

[dongsin-kim]

Hi @dsalantic, I would like to express my sincere gratitude for your effort to maintain and support this wonderful works.

I am trying to decode an actual CAT 20 data file (multilateration data of each flight on an airport) but the decode output says that the file contains a mix of category descriptions. (CAT 56, 1, 0, 254, 167, ..., etc.) Please find a relevant github issue and steps as follows:

---

Step1. write main.py

import asterix
print(asterix.__version__)

sample_filename = asterix.get_sample_file('tp1_20191221_150000_Cat20.ast')

with open(sample_filename, "rb") as f:
    data = f.read()

    parsed = asterix.parse(data)

    formatted = asterix.describe(parsed)
    print(formatted)

Step2. execute main.py
>> python main.py > output.txt

Step3. open output.txt
(Please find the attached output.txt file)

output.txt

---

I was hoping that I could get an output (an aircraft position, heading, callsign, velocity, ... etc. with a delimeter ,) as follows:
0,37.424707,127.126923,150, ...
1,37.424807,127.127923,151, ...
2,37.424907,127.128923,152, ...
3,37.425007,127.129923,153, ...
...

Should I specify a category description as CAT 20 before the command parsed = asterix.parse(data)?

[dsalantic]

Hi @dongsin-kim

It looks like your binary data is not raw asterix.
Can you check what is the real format of your input file ? Sometimes there are some headers before asterix data and you need to parse them first.
If you can send your input file (or at least first 100 bytes) I could look inside.

[dongsin-kim]

Hi @dsalantic, I really appreciate for your reply. Since I already signed up for a confidentiality agreement, I am not sure that I have a permission to upload (at least first 100 bytes of) a file on Github... Please let me ask this issue to an organization..

In the meantime, I am planning to replace an existing category definition (CAT 20 v1.7) to a newer version (CAT 20 v1.10 or v1.09). Maybe this issue was caused by a version inconsistency..?

[dsalantic]

No, I don't think it will change anything. Please check the first 3 bytes of your data. First byte represents Asterix category. If you expect that it contains CAT020 then first byte must be 0x14. Next 2 bytes represent the asterix block length, so this value should be something between 10 and 1000.

[ifsnop]

Reading the name of the file, maybe this files are from an Era Mlat system? If so, you could try skipping first 8 bytes of each data record.

…

Hi @dsalantic <https://github.com/dsalantic>, I really appreciate for your reply. Since I already signed up for a confidentiality agreement, I am not sure that I have a permission to upload at least first 100 bytes of a file on Github... Please let me ask this issue to an organization..

[dongsin-kim]

Hi @dsalantic and @ifsnop, thank you for the comments.

Using the AsterixInspector, I could see a table as follows:

Block Offset	Length	Category
0x0	99	56
0x63	40	1
0x8b	60675	0
0xed8e	16429	69
...	...	...

Could I interpret this table that the first byte is 0x0 and the Asterix block length is 99..?

Regarding skipping the first 8 bytes, how could I specify 8 bytes for skipping before the parsed = asterix.parse(data) in main.py..?

import asterix
print(asterix.__version__)

sample_filename = asterix.get_sample_file('tp1_20191221_150000_Cat20.ast')

with open(sample_filename, "rb") as f:
    data = f.read()

    parsed = asterix.parse(data)

    formatted = asterix.describe(parsed)
    print(formatted)

[dsalantic]

Unfortunately, there is no option in asterix decoder to skip the header, but
if you want to skip 8 bytes manually do the following:

header_bytes = f.read(8)
data = f.read()
parsed = asterix.parse(data)

Problem is that you will skip only header of first record, so you need to calculate the record length (from second 2 bytes) and then read only record length of data.
See the example from read_final_file.py how you can skip header if you now its structure.