FFmpeg QSV embedded EIA-608 caption decoding/transcoding

PCs, Processors & Graphics Graphics & Gaming
1

I have the following situation.  I have H.264 files in .ts container that contain in-band EIA-608 (A/53 / DTVCC Transport) captions.  I need to transcode the files with QSV into MPEG-2, retaining the captions.

Per official transcode examples:

Using FFmpeg 5.0 stable:

ffmpeg -hwaccel qsv -qsv_device 1 -codec:v h264_qsv -i in1.ts -codec:v mpeg2_qsv out1.mpg

Stream #0:0[0x44c]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn

(no separate subtitle/captions stream)

Resulting output file does not contain captions.  I am trying to utilize QSV decoding and encoding.

I also tried with -a53cc 1 flag, but also no captions in the output file.

ffmpeg -hwaccel qsv -qsv_device 1 -codec:v h264_qsv -i in1.ts -codec:v mpeg2_qsv -a53cc 1 out1.mpg

For the reference, software transcode without Intel QSV:

ffmpeg -i in1.ts -c:v mpeg2video out1.mpg

Results in an output file with captions.

My QSV hardware is DG1.

Help is appreciated.  Thank you.

2

r00tb33r, Thank you for posting in the Intel® Communities Support.

In order for us to provide the most accurate assistance on this matter, we just wanted to confirm a few details about your system:

What is the model of the Intel® processor?

What is the model of the motherboard?

If this is a laptop, what is the model of it?

Is this a new computer?

When did you purchase it?

Were you able to retain the captions previously on this specific machine?

If so, when did the issue start?

Did you make any recent hardware/software changes that might cause this problem?

Which Windows* version are you using?

Does the problem happen at home or in the work environment?

Please attach the SSU report so we can verify further details about the components in your platform, check all the options in the report including the one that says “3rd party software logs”:

https://downloadcenter.intel.com/download/25293/Intel-System-Support-Utility-for-Windows-?product=91600

Any questions, please let me know.

Regards,

Albert R.

Intel Customer Support Technician

3

What is the model of the Intel® processor?

i5-11400F, discrete DG1 “Iris Xe MAX”

What is the model of the motherboard?

Gigabyte Z590 Aorus Master

If this is a laptop, what is the model of it?

Not a laptop.

Is this a new computer?

Yes.

When did you purchase it?

March 2022

Were you able to retain the captions previously on this specific machine?

No, it was built for testing of Intel QSV transcodes.  It has not worked as expected yet.

If so, when did the issue start?

From the beginning.

Did you make any recent hardware/software changes that might cause this problem?

Can’t think of any.

Which Windows* version are you using?

Windows 10 Pro 21H2

Does the problem happen at home or in the work environment?

Work environment.  This is a broadcast facility.  We are evaluating Intel graphics products for fitness for production workflow.  FFmpeg is one of the software products Intel is offering support for with the new discrete graphics products.  We are evaluating as potential adopters of the current and upcoming Intel discrete graphics products.

SSU report attached.

Moderator:

Please unlock the following thread, I have additional information to share as I have an up-to-date supported product:

https://community.intel.com/t5/Graphics/ffmpeg-5-0-breaks-QSV-syntax-for-hardware-aceceleration/m-p/1368941

ssu.txt (117 KB)

4

r00tb33r, Thank you very much for providing that information and the SSU report.

We will do further research on this matter, as soon as I get any updates, I will post all the details on this thread.

Regards,

Albert R.

Intel Customer Support Technician

5

Here is some additional information you might find useful:

This is the applicable standard.

Each of the applicable decoders and encoders in FFmpeg should support it, including Intel’s QSV implementation, and if missing should be fixed with a patch.

6

Hello r00tb33r, Thank you very much for sharing those details.

After reviewing the case and the SSU, we can confirm that this is a Xe MAX graphics controller, however, these cards were not available for desktop end-users, so, just to confirm, is it possible to get some pictures may be of the card?

Did it come integrated with the system you bought/build?

Do you know by any chance if what you received was an engineering sample?

Do you have the option to provide the file you are using for the streams (video file, captions/ subtitles) and instructions on how to run it?

Regards,

Albert R.

Intel Customer Support Technician

7

is it possible to get some pictures may be of the card?

Attached a picture of one of my DG1 cards.

Did it come integrated with the system you bought/build?

These came from pre-built system in retail, like this one:

https://www.bestbuy.com/site/cyberpowerpc-gaming-desktop-intel-core-i5-11400f-8gb-memory-intel-iris-xe-500gb-ssd-black/6462676.p?skuId=6462676

Do you know by any chance if what you received was an engineering sample?

These are not engineering samples.  Considering I have run the same tests on integrated UHD 630 with the same results it will probably be of more use to focus on the software side of the QSV implementation.

Do you have the option to provide the file you are using for the streams (video file, captions/ subtitles) and instructions on how to run it?

Yes, here’s a test file with embedded captions:

https://drive.google.com/file/d/1jF-rCBuLTli4v9P37rAS4yIf8KQ71AOx/view?usp=sharing

Get FFmpeg:

https://www.gyan.dev/ffmpeg/builds/

https://www.gyan.dev/ffmpeg/builds/packages/ffmpeg-2022-04-18-git-d5687236ab-full_build.7z

Get MediaInfo (optional):

https://mediaarea.net/en/MediaInfo/Download/Windows

https://mediaarea.net/download/binary/mediainfo-gui/22.03/MediaInfo_GUI_22.03_Windows.exe

Run the following tests from command line:

QSV-accelerated transcode (test.ts is the input file, qsv.mpg is the output):

ffmpeg -hwaccel qsv -codec:v h264_qsv -i test.ts -codec:v mpeg2_qsv qsv.mpg

Examine the output using MediaInfo or VLC (captions are in the Subtitle menu).

Test expectation: captions present

Test result: no captions present in the output file

Software-only transcode (no QSV acceleration) (output file software.mpg):

ffmpeg -codec:v h264 -i test.ts -codec:v mpeg2video software.mpg

Examine the output using MediaInfo or VLC.

Test expectation: captions present

Test result: captions present in the output file

QSV decoder test (output file is qsv.yuv):

ffmpeg -hwaccel qsv -c:v h264_qsv -i test.ts -vf hwdownload,format=nv12 -pix_fmt yuv420p qsv.yuv

Examine the output using VLC (captions are in the Subtitle menu).

Test expectation: captions present

Test result: no captions present in the output file

Software-only decoder test (no QSV acceleration) (output file software.yuv):

ffmpeg -codec:v h264 -i test.ts -vf format=nv12 -pix_fmt yuv420p software.yuv

Examine the output using VLC (captions are in the Subtitle menu).

Test expectation: captions present

Test result: captions present in the output file

dg1.jpg
dg1.jpg1024×768 292 KB

8

r00tb33r, Thank you very much for letting us know those aspects and for providing the picture.

We will continue with our research on this matter, as soon as I get more details, I will post all the updates on this thread.

Regards,

Albert R.

Intel Customer Support Technician

9

Hello r00tb33r, I just received another update on this matter.

While still working on this case, we just wanted to confirm, if you do the same transcoding without QuickSync do the captions remain?

Regards,

Albert R.

Intel Customer Support Technician

10

Yes, the software-only transcode without QSV acceleration results in captions remaining in the output, which is the desired outcome.

With QSV they are missing.

11

r00tb33r, Thank you very much for confirming that information.

We will continue with our research on this matter, once I get more updates I will post all of them on this thread.

Regards,

Albert R.

Intel Customer Support Technician

12

Hi  @r00tb33r  ,

Would you be able to provide a sample of the .TS file with EIA-608 closed captions you are using when seing the issue?

Regards,

Esteban R.

13

@r00tb33r you can obtain “correct” ffmpeg-commandline(s) easily through (open source) FastFlix GUI, which also let you to take advantage of rigaya’s QSVEnc

14

Yes, the sample .TS file has already been provided earlier in this thread:

community.intel.com/t5/Graphics/FFmpeg-QSV-embedded-EIA-608-caption-decoding-transcoding/m-p/1378933#M107364

15

Hi  @r00tb33r  ,

Intel QSV encode has the capability to support subtitles, but developers using it need to write the code to add that support.

In your specific case, we advise you to contact the FFMpeg developer for support. Since they need to add support for Closed Caption in their h264_qsv and mpeg2_qsv codecs code.

Regards,

Esteban R.