Hi John,

John_L_9 · ‎07-06-2015

Hi,

I have been trying unsuccessfully to encode a video and keep its original orientation. Below is the code I am using with no luck. I have pulled this together from your samples and a few posts from this forum. Any guidance you could provide would be greatly appreciated.

I have also attached a sample video file I have been testing with that was recorded on a Moto X (2nd gen) running Android 5.1 and what the output looks like.

import android.annotation.TargetApi;
import android.content.Context;
import android.media.MediaCodecInfo;
import android.media.MediaFormat;
import android.media.MediaMetadataRetriever;
import android.os.Build;

import com.intel.inde.mp.AudioFormat;
import com.intel.inde.mp.IProgressListener;
import com.intel.inde.mp.MediaComposer;
import com.intel.inde.mp.MediaFile;
import com.intel.inde.mp.MediaFileInfo;
import com.intel.inde.mp.Uri;
import com.intel.inde.mp.VideoFormat;
import com.intel.inde.mp.android.AndroidMediaObjectFactory;
import com.intel.inde.mp.android.AudioFormatAndroid;
import com.intel.inde.mp.android.VideoFormatAndroid;
import com.intel.inde.mp.android.graphics.EglUtil;
import com.intel.inde.mp.android.graphics.VideoEffect;
import com.intel.inde.mp.domain.Pair;
import com.intel.inde.mp.domain.Resolution;

import java.io.File;
import java.io.IOException;

@TargetApi(Build.VERSION_CODES.JELLY_BEAN_MR1)
public class VideoTranscoder {

    private final Context mContext;
    private MediaComposer mMediaComposer;
    private AndroidMediaObjectFactory mFactory;
    private MediaFileInfo mMediaFileInfo;

    private final Uri mSourceUri;
    private final File mOutputFile;

    private long mTrimStartTime;
    private long mTrimEndTime;

    // Video Params
    private final String mVideoMimeType = "video/avc";
    private int mVideoBitRateInKBytes = 2 * 1024;
    private int mVideoFrameRate = 30;
    private int mVideoIFrameInterval = 10;
    private int mVideoRotationOut = 0;
    private Resolution mFrameSize;

    // Audio Params
    private final String mAudioMimeType = "audio/mp4a-latm";
    private int mAudioSampleRate = 48000;
    private int mAudioChannelCount = 2;
    private int mAudioBitRate = 96 * 1024;
    private int mAudioProfile = MediaCodecInfo.CodecProfileLevel.AACObjectLC;

    public VideoTranscoder(Context context, String sourceUri, File outputFile) {
        mContext = context.getApplicationContext();
        mSourceUri = new Uri(sourceUri);
        mOutputFile = outputFile;
    }

    public void startTranscode(IProgressListener listener) {
        try {
            mFactory = new AndroidMediaObjectFactory(mContext);

            setOutputVideoOrientation();
            setEncodingParameters();
            transcode(listener);
        } catch (Exception e) {
            LogUtils.e(e);
        }
    }

    public void destroy() {
        if (mMediaComposer != null) {
            mMediaComposer.stop();
        }
    }

    public void setTrim(long start, long end) {
        mTrimStartTime = start * 1000;
        mTrimEndTime = end * 1000;
    }

    public void stop() {
        if (mMediaComposer != null) {
            mMediaComposer.stop();
        }
    }

    protected void setEncodingParameters() {
        try {
            mMediaFileInfo = new MediaFileInfo(mFactory);
            mMediaFileInfo.setUri(mSourceUri);

            setVideoEncodingParameters(mMediaFileInfo);
            setAudioEncodingParameters(mMediaFileInfo);
        } catch (Exception e) {
            String message = (e.getMessage() != null) ? e.getMessage() : e.toString();
            LogUtils.e(message, e);
        }
    }

    /**
     * *******************************************************
     * Private Methods
     * *******************************************************
     */

    private void setAudioEncodingParameters(MediaFileInfo mediaFileInfo) {
        AudioFormat audioFormat = (AudioFormat) mediaFileInfo.getAudioFormat();
        if (audioFormat != null) {
            // TODO : These may throw exceptions if the values are not available in the
            // TODO : MediaFileInfo class, so should we try catch each one individually?
            mAudioSampleRate = Math.min(audioFormat.getAudioSampleRateInHz(), mAudioSampleRate);
            mAudioChannelCount = Math.min(audioFormat.getAudioChannelCount(), mAudioChannelCount);
            mAudioBitRate = Math.min(audioFormat.getAudioBitrateInBytes(), mAudioBitRate);
            mAudioProfile = audioFormat.getAudioProfile();
        } else {
            LogUtils.e("Audio format info unavailable");
        }
    }

    private void setVideoEncodingParameters(MediaFileInfo mediaFileInfo) {
        VideoFormat formatToCopy = (VideoFormat) mediaFileInfo.getVideoFormat();
        mFrameSize = formatToCopy.getVideoFrameSize();

        if(mVideoRotationOut == 90 || mVideoRotationOut == 270 || mVideoRotationOut == -90) {
            mFrameSize = new Resolution(mFrameSize.height(), mFrameSize.width());
        }

        try {
            mVideoBitRateInKBytes = formatToCopy.getVideoBitRateInKBytes();
        }
        catch(RuntimeException ex) {
            LogUtils.e(ex);
        }

        try {
            mVideoFrameRate = formatToCopy.getVideoFrameRate();
        }
        catch(RuntimeException ex) {
            LogUtils.e(ex);
        }

        try {
            mVideoIFrameInterval = formatToCopy.getVideoIFrameInterval();
        }
        catch(RuntimeException ex) {
            LogUtils.e(ex);
        }
    }

    private void transcode(IProgressListener listener) throws IOException {
        mMediaComposer = new MediaComposer(mFactory, listener);
        mMediaComposer.addSourceFile(mSourceUri);
        mMediaComposer.setTargetFile(mOutputFile.getAbsolutePath());
        mMediaComposer.setTargetVideoFormat(getTargetVideoFormat());
        mMediaComposer.setTargetAudioFormat(getTargetAudioFormat());
        mMediaComposer.addVideoEffect(getOrientationCorrectionEffect());

        if (mTrimEndTime > mTrimStartTime) {
            MediaFile mediaFile = mMediaComposer.getSourceFiles().get(0);
            mediaFile.addSegment(new Pair<>(mTrimStartTime, mTrimEndTime));
        }

        mMediaComposer.start();
    }

    private VideoFormatAndroid getTargetVideoFormat() {
        VideoFormatAndroid videoFormat = new VideoFormatAndroid(
                mVideoMimeType, mFrameSize.width(), mFrameSize.height());

        videoFormat.setVideoBitRateInKBytes(mVideoBitRateInKBytes);
        videoFormat.setVideoFrameRate(mVideoFrameRate);
        videoFormat.setVideoIFrameInterval(mVideoIFrameInterval);
        videoFormat.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 0);

        return videoFormat;
    }

    private AudioFormatAndroid getTargetAudioFormat() {
        AudioFormatAndroid audioFormat = new AudioFormatAndroid(
                mAudioMimeType, mAudioSampleRate, mAudioChannelCount);

        audioFormat.setAudioBitrateInBytes(mAudioBitRate);
        audioFormat.setAudioProfile(mAudioProfile);
        audioFormat.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 0);

        return audioFormat;
    }

    private VideoEffect getOrientationCorrectionEffect() {
        VideoEffect effect = new VideoEffect(mVideoRotationOut, mFactory.getEglUtil());
        effect.setSegment(new Pair<>(1L, 1L));

        return effect;
    }

    private void setOutputVideoOrientation() {
        MediaMetadataRetriever retriever = new MediaMetadataRetriever();
        retriever.setDataSource(mContext, android.net.Uri.parse(mSourceUri.getString()));
        String orientation = retriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_ROTATION);
        mVideoRotationOut = Integer.parseInt(orientation);
    }
}

Harshdeep_B_Intel · ‎07-08-2015

Hi John,

I am able to reproduce the issue. Currently, investigating this issue and will update with a workaround soon.

Thanks,

John_L_9 · ‎07-14-2015

Harsh,

Thanks for looking into this, I really appreciate it. Please let me know if I can be of any more help.

Thanks,

John

Parth_B_ · ‎08-25-2015

Hi - I am also facing the same issue. INDE does not seem to respect the rotation metadata on a video. Whenever I transcode or cut a video, the output is rotated incorrectly.

Are there any known solutions to this issue?

Thank you!

Harshdeep_B_Intel · ‎08-26-2015

Hi Parth,

Can you please provided more background on your issue. We have Media for mobile samples here: https://github.com/INDExOS/media-for-mobile which include transcode or cut video features. Please use the sample and let me know using the samples if you are still seeing the issue. I was not able to reproduce " transcode or cut a video, the output is rotated incorrectly." issue with the samples.

Please create a new thread on the issue as it will help us better to track the issue individually.

Thanks,

John_L_9 · ‎09-02-2015

Harsh,

Is there any update on this?

Thanks again for the help!

Harshdeep_B_Intel · ‎09-02-2015

Hi John,

I understand its frustrating and currently still waiting on updates from our development team. I will update once, I have any info on this issue. Thank you for your patience.

Thanks,

George_M_3 · ‎09-22-2015

Ran into this myself. The library isn't respecting the rotation metadata that gets stored inside the video when recording video using Android's standard camera classes for recording.

As a workaround I'm reading the rotation metadata myself with FFmpegMediaMetadataRetriever and applying the generic RotateEffect as found here:

https://github.com/INDExOS/media-for-mobile/blob/77615f5309e753c35b1faf12a8dda72024eca570/Android/samples/effects/src/com/intel/inde/mp/effects/RotateEffect.java

You can also just use MediaMetadataRetriever which is part of the Android SDK if you want to avoid additional dependencies and the API calls are basically exactly the same (I'm using FFmpegMediaMetadataRetriever already for other reasons in other parts of the code, so I just use it here as well).

http://developer.android.com/reference/android/media/MediaMetadataRetriever.html

Attached the relevant snippet of my code below, hope this helps. Keep in mind if Intel ever fixes this in their library you'll need to remove this code when linking against future versions to avoid double rotations.

-----------------

mediaComposer.addSourceFile(srcPath);

int videoRotation = 0;

FFmpegMediaMetadataRetriever retriever = new FFmpegMediaMetadataRetriever();
retriever.setDataSource(srcPath);

String rotationMeta = retriever.extractMetadata(FFmpegMediaMetadataRetriever.METADATA_KEY_VIDEO_ROTATION);

if (!TextUtils.isEmpty(rotationMeta)) {
    videoRotation = Integer.parseInt(rotationMeta);
}

retriever.release();

mediaComposer.addVideoEffect(new RotateEffect(videoRotation, factory.getEglUtil()));
mediaComposer.setTargetFile(destPath);

Also to avoid letterboxing the video in weird ways you should check for 90/270 degree rotations and if they occur, flip the width and height of the output video, something like this (values hardcoded here for clarity, for flexibility pull these from the source video's width/height:

int targetWidth = 1920;
int targetHeight = 1080;

if (Math.abs(videoRotation) == 90 || Math.abs(videoRotation) == 270) {
   targetWidth = 1080;
   targetHeight = 1920;
}

VideoFormatAndroid videoFormat = new VideoFormatAndroid("video/avc", targetWidth, targetHeight);
mediaComposer.setTargetVideoFormat(videoFormat);

John_L_9 · ‎09-22-2015

George M,

Thanks for the reply. It's been a while since I looked at this stuff - Looking at my original code above it looks like I attempted to do exactly what you are saying but with no luck, but I will try again using what you posted as a reference. Do you notice anything I am doing above in the

setVideoEncodingParameters() or

getOrientationCorrectionEffect() that might me wrong?

Thanks!

Daniel_B_5 · ‎10-13-2016

@George M, thanks a lot for the directions!!

@Harsh Jain (Intel) in the Media for mobile sample I just modified the ComposerTranscodeCoreActivity.java at two points:

protected void setTranscodeParameters(org.m4m.MediaComposer mediaComposer) throws IOException {

        mediaComposer.addSourceFile(mediaUri1);

        //--------- START CHANGE ----------
        //Corrects the video orientation
        int videoRotation = mediaFileInfo.getRotation();;
        mediaComposer.addVideoEffect(new RotateEffect(videoRotation, factory.getEglUtil()));
        //--------- END CHANGE ------------ 
        
        mediaComposer.setTargetFile(dstMediaPath);

        configureVideoEncoder(mediaComposer, videoWidthIn, videoHeightIn, videoRotation);
        configureAudioEncoder(mediaComposer);
}

Incorrect Video Orientation / Rotation