JPEG Decoder in HW mode delivers incomplete image when using Rotation

[Riedl__Christian]

I implemented a still JPEG Transcoder (Resizer) similar to the transcoder sample : 

sts = _mfxDECODE->DecodeFrameAsync(&mfxIBS, &_decSurface, &outSurface, &syncDecode);
sts = _mfxVPP->RunFrameVPPAsync(outSurface, &_encSurface[0], NULL, &syncVpp);
sts = _mfxENCODE->EncodeFrameAsync(&encodeCtrl, &_encSurface[0], &mfxOBS, &syncEncode);
sts = _mfxSession.SyncOperation(syncEncode, MSDK_WAIT_INTERVAL);

It works quite good and fast on my 8th gen i7. Now I added the Rotation feature to the JPEG decoder.

In SW mode it works as expected, but in HW mode (without d3d) it delivers an incomplete image. When I rotate a 3000 * 2000 image it delivers a result where the upper 2000 * 2000 is correct while the bottom part is solid green. 

I tried to reproduce the problem with sample_decode using

"decode jpeg -hw -nv12 -jpeg_rotate 90 -i c:\temp\w.jpg -o c:\temp\w.dmp"  

When I inspected the dump file w.dmp I noticed that (different than in SW mode result) there were large areas (1/3) filled with zero bytes.

The size of the dump file is the same like a file produced without rotation.

[Mark_L_Intel1]

Hi Christian,

I will try sample_decode and see if it can be reproduced when I have a chance, but could you try following things on your side?

1. Rotate 180, 270.

2. A low resolution image like 300x300

3. Different color format, I think your input is NV12, right?

I also check the developer reference and find no explanation on rotation parameters.

Mark

[Mark_L_Intel1]

Hi Christian,

I can reproduce the issue with sample_decode. I think the result is correct because the sample_decode is designed to meed every condition, it assumes the input aspect ratio 1:1, it can be seen clearly at the beginning of the following output message:

sample_decode.exe jpeg -hw -nv12 -jpeg_rotate 90 -i forest2.jpg -o out90.yuv
pretending that aspect ratio is 1:1
Decoding Sample Version 8.3.26.0

Input video     JPEG
Output format   NV12
Input:
  Resolution    672x384
  Crop X,Y,W,H  0,0,660,372
Output:
  Resolution    672x384
Frame rate      30.00
Memory type             system
MediaSDK impl           hw
MediaSDK version        1.27

Decoding started
Frame number:    1, fps: 3.251, fread_fps: 5538.836, fwrite_fps: 662.977
Decoding finished
 

Mark

[Riedl__Christian]

Hi Mark,

thank you for your response.

I performed the proposed tests :

1. Rotation 180 is ok, 270 shows the same error like 90

2. A small and big square jpeg (2000*2000, 300*300) is rotated correct.

3. I was not able to make my transcoder work with a different color format than NV12

I also made some tries with different values for ApectRatioW/H. It had no influence to the result.

So I do not believe that decoding sample only supports aspect ratio 1:1.  In the "SDK Developer Reference for JPEG" you cannot find the term "aspect ratio". A restriction to square pictures would make the rotate feature useless. 

As I said in the first post : Rotate works correct in SOFTWARE mode. So it seems to be a problem in the HW decoder.

I need this functionality to rotate pictures with portrait orientation. They contain the EXIF/Orientation which is responsible that the (original) pictures are show correctly in most browsers. But during JPEG decode => resize => JPEG encode all metadata is lost.
Adding JPEG metadata is time consuming when done as seperate step. Is it possible to set metadata during JPEG encode ?

In my opinion it is a bug in the HW JPEG decoder.

Christian

[Mark_L_Intel1]

Thanks so much,

I basically agree with you, but still curious if JPEG hardware decoding in sample code is designed to show the full functions; if not, there should be some document on how to use it. 

About your comment #1 and #2, it looks like the code only samples a squared area(aspect ratio 1:1), I tried a lower resolution and it is the same; if I modified the same picture to 1:1, it can rotate successfully.

I will submit an investigation request first and do some research myself.

Mark

[Mark_L_Intel1]

Hi Christian,

Just an update, the dev team confirmed the issue and starts looking at it.

We found an update of the Media SDK open source code at the following location but it didn't solve the problem completely. So we are still investigating.

https://github.com/Intel-Media-SDK/MediaSDK/commit/297a1eb2f7e7bbfd68a6803f0a0bc4567b383814 ;

Mark

[Riedl__Christian]

Hi Mark,

Thank you for the info. So I am quite optimistic that I get my problem solved.

Christian

[Mark_L_Intel1]

Thanks so much,

This is good and sorry for not being able to help you in time.

Could you give us a hint so we could fix it quickly?

Mark.

[Riedl__Christian]

Mark, 

you wrote some time ago that you can reproduce my problem. 

Maybe I forgot to tell you that I run my software on Windows Server 2016. The Opensource Intel-Media-SDK is only for Linux.

With all sources available it would be not too hard for me to fix a reproducable problem (I solve such kind of problems since 40 years).

But for the Windows version I have only the binaries. I told you that the problem does not occure in "software mode", it occurs only with hardware mode (without d3d).  So the bug must be somewhere in the "hardware specific" part.

Christian

[Mark_L_Intel1]

Thanks for the feedback,

Yes, I understand and this is the issue I submitted.

The develop team can reproduce this issue and they understand this is in hardware part of the library. I will double check.

For now, it is still investigation but they made some progress to allocate where the problem is. I will keep you updated.

Mark

[Riedl__Christian]

Mark,

in the meanwhile Media SDK 2019 R1 has been published. In the release notes I did not find a hint that my problem has been solved.

Do you have any information ?

Christian

[Mark_L_Intel1]

Hi Christian,

Sorry for the late response, I checked the status, it is unresolved yet.

The last time I check was 3 month ago and I didn't get dev team response yet. Let me ping them again.

Mark

[Riedl__Christian]

Mark,

after installing latest Media SDK 2019 R.1 and updating the UHD 630 drivers to latest version, the problem was solved.

Christian

[Mark_L_Intel1]

Thanks so much for letting me know this,

It looks like the dev team had fixed the issue and applied it to Windows release, the Windows release might adopt the Linux code but they might forgot to update the release notes.

But feel free to let me know if this issue happens again.

Mark