BDAV .m2ts extraction support

[uplusplus]

ref: https://en.wikipedia.org/wiki/MPEG_transport_stream

m2ts file have a 4-byte header:
4-byte Transport Stream Header

TsExtractor cannot recognize it.

video data clip:
000000000 01 95 1D 2E 47 40 00 10 00 00 B0 11 00 00 C1 00 00 00 00 E0 1F 00 01 E1 00 23 5A AB 82 FF FF FF ....G@....°...Á....à...á.#Z«.ÿÿÿ
000000032 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000064 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000096 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000128 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000160 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000192 01 95 22 24 47 41 00 30 15 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 00 02 .."$GA.0..ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ..
000000224 B0 37 00 01 C1 00 00 F0 01 F0 0C 05 04 48 44 4D 56 88 04 0F FF 84 FC 1B F0 11 F0 14 05 08 48 44 °7..Á..ð.ð...HDMV...ÿ.ü.ð.ð...HD
000000256 4D 56 FF 1B 57 3F 31 08 00 00 00 00 F0 00 00 01 82 F1 00 F0 00 A4 6E A1 2F FF FF FF FF FF FF FF MVÿ.W?1.....ð....ñ.ð.¤n¡/ÿÿÿÿÿÿÿ
000000288 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000320 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000352 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000384 01 95 25 72 47 40 1F 10 00 7F F0 19 FF FF C1 00 00 F0 0A 63 08 C1 D4 C0 FF FF FF FF FF 00 01 80 ..%rG@....ð.ÿÿÁ..ð.c.ÁÔÀÿÿÿÿÿ...
000000416 00 03 00 38 6D FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ...8mÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000448 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000480 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000512 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000544 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000576 01 95 28 C0 47 10 01 20 B7 10 00 00 AC DE 7E 70 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ..(ÀG.. ·...¬Þpÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000608 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000640 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000672 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000704 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000736 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
000000768 01 95 31 03 47 50 11 10 00 00 01 E0 00 00 80 C0 0A 31 00 0B 7E ..1.GP.....à...À.1..

[AquilesCanta]

Unfortunately we cannot work with a hexdump of your file. We will need the file itself. You can either post it here or send it to [email protected]. However, I can see that the sync byte is not at the beginning of the file, so if you remove the first four bytes (so that the file starts with 0x47), it should work. If this solves your issue, let me know through this issue. If it doesn't, open a new issue including all required information in the issue template.

[Avetri]

It is M2TS format. It has four additional bytes at start of each common DVB packet. As I know a timestamp is stored there. For fast access on the low level.
You may (as I had made it) create a wrapper around TsExtractror and realize consumption that cuts the first four bytes and than send DVB part into TsExtractor.

[AquilesCanta]

Well, considering this addition breaks the 188 packet size restriction, I wouldn't say it is M2TS format. If these four extra bytes appear for every TS packet then no "actual TS parser" will be able to sync, and therefore, to read.

I am wondering though where you got this file from (considering @Avetri knows about it). On top of that, the timestamp included there is to facilitate access according to what? If it was playback time, I would have expected the first four bytes to include a value near 0.

[uplusplus]

another m2ts file:http://download1655.mediafire.com/x9vxw78itlbg/muj20yyjdqi/Babylon+A.D.M2TS
packet structure: 4 bytes header + 188 body

[ojw28]

As per the response above, if you're adding data between transport stream packets then it's not a valid transport stream. I'm not sure why you expect this to work?

[uplusplus]

https://en.wikipedia.org/wiki/MPEG_transport_stream

Partial Transport Stream Packet Format

Name	Number of bits	32-bit BE mask	Description
4-byte Transport Stream Header
Sync byte	8	0xff000000	Bit pattern of 0x47 (ASCII char 'G')
Transport Error Indicator (TEI)	1	0x800000	Set when a demodulator can't correct errors from FEC data; indicating the packet is corrupt.[7]
Payload Unit Start Indicator	1	0x400000	Set when a PES, PSI, or DVB-MIP packet begins immediately following the header.
Transport Priority	1	0x200000	Set when the current packet has a higher priority than other packets with the same PID.
PID	13	0x1fff00	Packet Identifier, describing the payload data.
Scrambling control	2	0xc0	'00' = Not scrambled. For DVB-CSA and ATSC DES only:[8] '01' (0x40) = Reserved for future use '10' (0x80) = Scrambled with even key '11' (0xC0) = Scrambled with odd key
Adaptation field flag	1	0x20
Payload flag	1	0x10
Continuity counter	4	0xf	Sequence number of payload packets (0x00 to 0x0F) within each stream (except PID 8191) Incremented per-PID, only when a payload flag is set.
Optional fields
Adaptation field	variable		If Adaptation field flag is set, see below.
Payload Data	variable		If Payload flag is set. Payload may be PES packets, program specific information (below), or other data.

[ojw28]

I'm not sure what you're trying to show by pasting part of a Wikipedia article into this issue with no accompanying comment. Please clarify?

Also, I think you're misreading the specification. The packet structure is not 4-random-bytes-to-do-what-you-like-with followed by a 188-byte-body. The packet structure is a 4-byte-packet-header followed by 184 bytes of payload. The 4 byte header consists of the fields starting from the sync byte and ending at the continuity counter, as defined in the table above (note that these fields total to 4 bytes in length). The remaining 184 bytes is the payload data (optionally preceded by an adaptation field).

[Avetri]

@ojw28 @uplusplus A small patch to add the M2TS support.
AddM2TS.patch.zip

[ojw28]

A link to https://en.wikipedia.org/wiki/.m2ts would have cleared up a lot of confusion on this issue a long time ago... Please can you provide some working sample content in this format to [email protected]. The link to sample content above no longer works.

[Avetri]

It (four byte header) is better to skip.
Better_to_skip.patch.zip

[ojw28]

We need proper sample content to look at this. The patch above is incomplete in that it doesn't implement seeking or duration parsing, both of which should be possible using the extra four bytes in this type of media.

[Avetri]

"We need proper sample content to look at this."
@ojw28 Unfortunately i am not able to give such content sample.
You may look it for in sample packs as DVD5, DivX Test or other that is compatible with ExoPlayer. The player has not been flexible enough yet to work with any TS or M2TS (BDAV as You named it) file.

"implement seeking or duration parsing"
It will be gorgeous if somebody writes such code.

[ojw28]

Until someone provides sample content for this enhancement, we wont be looking at it.

[Avetri]

@ojw28
https://www.dropbox.com/sh/poubu1hb6f1lzu8/AADl9XHl3EKFq2QAnsYWHBFOa?dl=0

A few sample files there are in the folder. Many of them are audio related.

[Avetri]

Diff from adding the M2TsExtractor to ExoPlayer 2.

Add M2TsExtractor to ExoPlayer 2.diff.zip

[NanyangTaiji]

In case that any one interested in MTS file, I provide two examples
https://we.tl/t-RM6TKf1JN2
https://we.tl/t-0FrOmwzgXe

It seems that there are some differences between MTS video and M2TS video, as indicated in the following links
https://repairit.wondershare.com/video-repair/mts-vs-m2ts.html

[NanyangTaiji]

@ojw28 @uplusplus A small patch to add the M2TS support. AddM2TS.patch.zip

I applied this patch to r1.5.7, the video can be played smoothly, but there is no sound.

[Avetri]

@ojw28 @uplusplus A small patch to add the M2TS support. AddM2TS.patch.zip

I applied this patch to r1.5.7, the video can be played smoothly, but there is no sound.

I suppose You should use something like an FFmpeg extension. Or device with AC3/DTS hardware support.

Please look for audio-related messages in the Logcat: audio tracks list, track switching, decoder errors, etc.

[NanyangTaiji]

@ojw28 @uplusplus A small patch to add the M2TS support. AddM2TS.patch.zip

I applied this patch to r1.5.7, the video can be played smoothly, but there is no sound.

I suppose You should use something like an FFmpeg extension. Or device with AC3/DTS hardware support.

Please look for audio-related messages in the Logcat: audio tracks list, track switching, decoder errors, etc.

Dear Avetri

Thanks for you quick reply, which I did not wish to have one.
r.1.5.7 does not come with an FFMpeg extension, so I applied the patch to r.2.0.4, and use the demo directly, which iitself includes the FFMpeg extension.
It turned out, for the m2ts clips, video is played without sound, with a prompted message: "Media includes video tracks, but none are playable by this device".
For mts clips, only the above message is prompted with no video.

I also tried with device with AC3 hardware support, in which Mediaplayer are able to play these clips with sound. But the exoplayer I complied did not succeed.

[NanyangTaiji]

ExoPlayer-r2.0.4.zip

[NanyangTaiji]

Can anyone make this into high priority? I don't believe it; an issue has been kept low priority for 6 years. The developer must be poor guys who never used professional video recording equipment. Our university has accumulated thousands of T of video recordings for online lecture, they are all recorded by Sonny camcorder, they are in mts/m2ts formats.

[marcbaechinger]

Reassigning. Thanks for your kind words.

[NanyangTaiji]

Any progress???

[NanyangTaiji]

@ojw28 @uplusplus A small patch to add the M2TS support. AddM2TS.patch.zip

I applied this patch to r1.5.7, the video can be played smoothly, but there is no sound.

I suppose You should use something like an FFmpeg extension. Or device with AC3/DTS hardware support.

Please look for audio-related messages in the Logcat: audio tracks list, track switching, decoder errors, etc.

Dear Avetri

Just wonder for mts/m2ts video format, the AC-3 audio track needs to skip the four additional bytes synchronically as well to get the sound?

Thanks and God bless you and your country!

[Avetri]

Dear Avetri

Just wonder for mts/m2ts video format, the AC-3 audio track needs to skip the four additional bytes synchronically as well to get the sound?

Thanks and God bless you and your country!

Hello NanyangTaiji.

I am not entirely sure what the four bytes You mentioned exactly are.
All the audio streams are broadcasted as all the other ones in DVB MPEG TS. They are encapsulated inside audio or private PESs (Packetized Elementary Stream). And should be defined in an appropriate PMT (Program Map Table).

You may look inside the MPEG TS file or stream. I recommend Wireshark for it.

Thank You for your kind words.

[NanyangTaiji]

It is M2TS format. It has four additional bytes at start of each common DVB packet. As I know a timestamp is stored there. For fast access on the low level.
You may (as I had made it) create a wrapper around TsExtractror and realize consumption that cuts the first four bytes and than send DVB part into TsExtractor.

Dear Avetri

Thanks for your quick reply. The four additional bytes was what you mentioned in your early post at the top:

"It is M2TS format. It has four additional bytes at start of each common DVB packet. As I know a timestamp is stored there. For fast access on the low level.
You may (as I had made it) create a wrapper around TsExtractror and realize consumption that cuts the first four bytes and than send DVB part into TsExtractor."

In your https://github.com/google/ExoPlayer/files/267909/AddM2TS.patch.zip, you skip these bytes in video track. I wonder whether Audio track needs to be skipped as well.

All the best!

[Avetri]

In your https://github.com/google/ExoPlayer/files/267909/AddM2TS.patch.zip, you skip these bytes in video track. I wonder whether Audio track needs to be skipped as well.

I skip those four bytes for every DVB (M2ts/MTS) packet. There are no tracks on that level.

Audio track content is on the last level of the chain: M2TS->MTS->PAT->PMT->PES->AudioElemetaryStream

[NanyangTaiji]

In your https://github.com/google/ExoPlayer/files/267909/AddM2TS.patch.zip, you skip these bytes in video track. I wonder whether Audio track needs to be skipped as well.

I skip those four bytes for every DVB (M2ts/MTS) packet. There are no tracks on that level.

Audio track content is on the last level of the chain: M2TS->MTS->PAT->PMT->PES->AudioElemetaryStream

Thanks. This is beyond my knowledge. I think you are one of the most suitable persons to make a good pull request for BDAV .m2ts extraction support.

[NanyangTaiji]

In your https://github.com/google/ExoPlayer/files/267909/AddM2TS.patch.zip, you skip these bytes in video track. I wonder whether Audio track needs to be skipped as well.

I skip those four bytes for every DVB (M2ts/MTS) packet. There are no tracks on that level.

Audio track content is on the last level of the chain: M2TS->MTS->PAT->PMT->PES->AudioElemetaryStream

The sound issue is solved after adding the FFmpeg extension. But the information of video duration is not available, so the video is not seekable.