Android MediaCodec，解码MJPEG帧

编辑： 没关系，我解决了我的问题

//------------------------------------------------------------------------------

我在尝试使用Android MediaCodec API进行MJPEG流解码时遇到了一些问题。我想做的是从我的网络摄像头中传输jpeg帧，并在我的android设备上使用硬件加速解码来显示它们（目前我使用的是Nvidia Shield平板电脑，android 6.0）

现在我正在接收jpeg帧，我可以使用BitmapFactory.decodeStream（）函数在屏幕上显示它。然而，我想做的是在专用硬件上解码图像，这样我可以获得更好的性能和更好的电池寿命。为了做到这一点，我想使用MediaCodec API和“video/mjpeg”deocoder，我很确定这在我的设备上是可用的（我检查了MediaCodecList，它就在“OMX.Nvidia.mjpeg.decoder”的名称下）

我在SurfaceView上使用我的MediaCodec，与《格拉菲卡》回购中的做法差不多。（）

这是在surfaceCreated（）回调中调用的解码器初始化代码，完成时没有任何异常：

private Surface testSurface; 
private MediaCodec decoder;

//....

private void initMediaApi() throws IOException {
    decoder = MediaCodec.createDecoderByType("video/mjpeg");

    MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/mjpeg", 720, 480);
    mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 25);

    decoder.configure(mediaFormat, testSurface, null, 0);
    decoder.start();
}

之后，我有一个函数，每次接收到新的jpeg帧时都会调用它。它采用两个参数：带有jpeg图像的字节数组和此图像的字节大小：

private void testMediaApi(final byte[] frameBuffer, final int length) throws IOException {

    final int TIMEOUT_USEC = 10000;

    long firstInputTimeNsec = System.nanoTime();

    int inputBufIndex = decoder.dequeueInputBuffer(TIMEOUT_USEC);
    if (inputBufIndex < 0) {
        return;
    }

    ByteBuffer inputBuffer = decoder.getInputBuffer(inputBufIndex);
    inputBuffer.clear();
    inputBuffer.put(frameBuffer, 0, length);

    decoder.queueInputBuffer(inputBufIndex, 0, length, 0, 0);

    boolean outputDone = false;

    while (!outputDone) {
        outputDone = true;

        MediaCodec.BufferInfo mBufferInfo = new MediaCodec.BufferInfo();
        int decoderStatus = decoder.dequeueOutputBuffer(mBufferInfo, 1000000);
        Log.d(LOG_TAG, "decoderStatus: " + decoderStatus);

        if (decoderStatus == MediaCodec.INFO_TRY_AGAIN_LATER) {
            // no output available yet
            Log.d(TAG, "no output from decoder available");
        } else if (decoderStatus == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
            // not important for us, since we're using Surface
            Log.d(TAG, "decoder output buffers changed");
        } else if (decoderStatus == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
            MediaFormat newFormat = decoder.getOutputFormat();
            Log.d(TAG, "decoder output format changed: " + newFormat);
        } else if (decoderStatus < 0) {
            throw new RuntimeException("unexpected result from decoder.dequeueOutputBuffer: " + decoderStatus);
        } else { // decoderStatus >= 0
            if (firstInputTimeNsec != 0) {
                // Log the delay from the first buffer of input to the first buffer
                // of output.
                long nowNsec = System.nanoTime();
                Log.d(TAG, "startup lag " + ((nowNsec - firstInputTimeNsec) / 1000000.0) + " ms");
                firstInputTimeNsec = 0;
            }

            Log.d(TAG, "surface decoder given buffer " + decoderStatus + " (size=" + mBufferInfo.size + ")");
            if ((mBufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
                Log.d(TAG, "output EOS");
                outputDone = true;
            }

            boolean doRender = (mBufferInfo.size != 0);

            // As soon as we call releaseOutputBuffer, the buffer will be forwarded
            // to SurfaceTexture to convert to a texture.  We can't control when it
            // appears on-screen, but we can manage the pace at which we release
            // the buffers.
            decoder.releaseOutputBuffer(decoderStatus, doRender);
            Log.d(TAG, "Reached EOS, looping");
            decoder.flush();    // reset decoder state
        }
    }
}

如果不是连续排队帧，而是等待第一个帧解码，则程序进入无限循环，输出如下所示：

07-20 21:58:29.970 10577-10641/piotrek.androidfpvtest I/OMXClient: Using client-side OMX mux.
07-20 21:58:30.110 10577-10640/piotrek.androidfpvtest I/MediaCodec: [OMX.Nvidia.mjpeg.decoder] setting surface generation to 10830849
07-20 21:58:30.115 10577-10641/piotrek.androidfpvtest I/ACodec: Enable timestamp filtering for Video Decoder
07-20 21:58:30.126 10577-10641/piotrek.androidfpvtest D/SurfaceUtils: set up nativeWindow 0xae47c708 for 720x480, color 0x106, rotation 0, usage 0x2b00
07-20 21:58:56.719 10577-10641/piotrek.androidfpvtest D/SurfaceUtils: set up nativeWindow 0xae47c708 for 720x480, color 0x147, rotation 0, usage 0x2b00
07-20 21:58:56.720 10577-10641/piotrek.androidfpvtest W/ACodec: [OMX.Nvidia.mjpeg.decoder] setting nBufferCountActual to 11 failed: -1010
07-20 21:58:56.720 10577-10641/piotrek.androidfpvtest W/ACodec: [OMX.Nvidia.mjpeg.decoder] setting nBufferCountActual to 10 failed: -1010
07-20 21:58:57.713 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:58:57.713 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:58:57.714 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -3
07-20 21:58:57.714 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoder output buffers changed
07-20 21:58:58.714 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:58:58.715 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:58:59.717 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:58:59.718 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:00.720 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:00.720 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:01.721 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:01.721 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:02.723 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:02.723 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:03.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:03.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:04.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:04.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:05.726 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:05.726 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:06.727 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:06.727 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available

---

如果有人对这个API有任何经验，我将非常感谢任何帮助

我自己解决了问题。事实证明，这个特定的解码器不支持输出到surface，因此解决方案是获取原始ByteBuffer，然后使用GLES进行YUV->RGB转换并将其显示在屏幕上；将引发ItLaulalAgMuthEnter异常初始化视频/MJPEG，错误0xFFFFFFE< /P>如果这个问题没有任何贡献，请考虑删除它。@ FuffyMaI，它确实如此。用户可以询问和回答自己的问题。这个对我也有帮助。@Nexen:我很高兴听到它对你有帮助！请对您的答案添加更多解释，以便对未来的用户更有价值。我想他说createDecoderByType在没有mjpeg解码器的设备上抛出异常。在我的设备上也是一样的。

07-20 21:58:29.970 10577-10641/piotrek.androidfpvtest I/OMXClient: Using client-side OMX mux.
07-20 21:58:30.110 10577-10640/piotrek.androidfpvtest I/MediaCodec: [OMX.Nvidia.mjpeg.decoder] setting surface generation to 10830849
07-20 21:58:30.115 10577-10641/piotrek.androidfpvtest I/ACodec: Enable timestamp filtering for Video Decoder
07-20 21:58:30.126 10577-10641/piotrek.androidfpvtest D/SurfaceUtils: set up nativeWindow 0xae47c708 for 720x480, color 0x106, rotation 0, usage 0x2b00
07-20 21:58:56.719 10577-10641/piotrek.androidfpvtest D/SurfaceUtils: set up nativeWindow 0xae47c708 for 720x480, color 0x147, rotation 0, usage 0x2b00
07-20 21:58:56.720 10577-10641/piotrek.androidfpvtest W/ACodec: [OMX.Nvidia.mjpeg.decoder] setting nBufferCountActual to 11 failed: -1010
07-20 21:58:56.720 10577-10641/piotrek.androidfpvtest W/ACodec: [OMX.Nvidia.mjpeg.decoder] setting nBufferCountActual to 10 failed: -1010
07-20 21:58:57.713 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:58:57.713 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:58:57.714 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -3
07-20 21:58:57.714 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoder output buffers changed
07-20 21:58:58.714 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:58:58.715 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:58:59.717 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:58:59.718 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:00.720 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:00.720 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:01.721 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:01.721 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:02.723 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:02.723 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:03.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:03.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:04.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:04.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:05.726 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:05.726 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:06.727 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:06.727 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available