Opengl es 使用CVPixelBufferRef和着色器在ffmpeg的OpenGL中渲染YUV视频

我正在使用iOS 5.0方法“CVOpenGLESTextureCacheCreateTextureFromImage”渲染ffmpeg的YUV帧

我用的是苹果的例子

我在iPhone屏幕上的结果如下：

我要知道我做错了

我把我的代码的一部分，以找到错误

ffmpeg配置帧：

ctx->p_sws_ctx = sws_getContext(ctx->p_video_ctx->width, 
                                ctx->p_video_ctx->height, 
                                ctx->p_video_ctx->pix_fmt, 
                                ctx->p_video_ctx->width, 
                                ctx->p_video_ctx->height,
                                PIX_FMT_YUV420P, SWS_FAST_BILINEAR, NULL, NULL, NULL);


// Framebuffer for RGB data
ctx->p_frame_buffer = malloc(avpicture_get_size(PIX_FMT_YUV420P,
                                                ctx->p_video_ctx->width, 
                                                ctx->p_video_ctx->height));

avpicture_fill((AVPicture*)ctx->p_picture_rgb, ctx->p_frame_buffer,PIX_FMT_YUV420P, 
               ctx->p_video_ctx->width, 
               ctx->p_video_ctx->height);

我的渲染方法：

if (NULL == videoTextureCache) {
    NSLog(@"displayPixelBuffer error");
    return;
}    


CVPixelBufferRef pixelBuffer;    
   CVPixelBufferCreateWithBytes(kCFAllocatorDefault, mTexW, mTexH, kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, buffer, mFrameW * 3, NULL, 0, NULL, &pixelBuffer);



CVReturn err;    
// Y-plane
glActiveTexture(GL_TEXTURE0);
err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault, 
                                                   videoTextureCache,
                                                   pixelBuffer,
                                                   NULL,
                                                   GL_TEXTURE_2D,
                                                   GL_RED_EXT,
                                                   mTexW,
                                                   mTexH,
                                                   GL_RED_EXT,
                                                   GL_UNSIGNED_BYTE,
                                                   0,
                                                   &_lumaTexture);
if (err) 
{
    NSLog(@"Error at CVOpenGLESTextureCacheCreateTextureFromImage %d", err);
}   

glBindTexture(CVOpenGLESTextureGetTarget(_lumaTexture), CVOpenGLESTextureGetName(_lumaTexture));
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);     

// UV-plane
glActiveTexture(GL_TEXTURE1);
err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault, 
                                                   videoTextureCache,
                                                   pixelBuffer,
                                                   NULL,
                                                   GL_TEXTURE_2D,
                                                   GL_RG_EXT,
                                                   mTexW/2,
                                                   mTexH/2,
                                                   GL_RG_EXT,
                                                   GL_UNSIGNED_BYTE,
                                                   1,
                                                   &_chromaTexture);
if (err) 
{
    NSLog(@"Error at CVOpenGLESTextureCacheCreateTextureFromImage %d", err);
}

glBindTexture(CVOpenGLESTextureGetTarget(_chromaTexture), CVOpenGLESTextureGetName(_chromaTexture));
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);     

glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebuffer);

// Set the view port to the entire view
glViewport(0, 0, backingWidth, backingHeight);

static const GLfloat squareVertices[] = {
    1.0f, 1.0f,
    -1.0f, 1.0f,
    1.0f,  -1.0f,
    -1.0f,  -1.0f,
};

GLfloat textureVertices[] = {
    1, 1,
    1, 0,
    0, 1,
    0, 0,
};

// Draw the texture on the screen with OpenGL ES 2
[self renderWithSquareVertices:squareVertices textureVertices:textureVertices];


// Flush the CVOpenGLESTexture cache and release the texture
CVOpenGLESTextureCacheFlush(videoTextureCache, 0);    
CVPixelBufferRelease(pixelBuffer);     

 [moviePlayerDelegate bufferDone];

RenderWithSquareVertices方法

    - (void)renderWithSquareVertices:(const GLfloat*)squareVertices textureVertices:(const GLfloat*)textureVertices
{


  // Use shader program.
    glUseProgram(shader.program);

// Update attribute values.
glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, squareVertices);
glEnableVertexAttribArray(ATTRIB_VERTEX);
glVertexAttribPointer(ATTRIB_TEXTUREPOSITON, 2, GL_FLOAT, 0, 0, textureVertices);
glEnableVertexAttribArray(ATTRIB_TEXTUREPOSITON);

glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

// Present
glBindRenderbuffer(GL_RENDERBUFFER, colorRenderbuffer);
[context presentRenderbuffer:GL_RENDERBUFFER];

}

我的片段着色器：

uniform sampler2D SamplerY;
uniform sampler2D SamplerUV;


varying highp vec2 _texcoord;

void main()
{

mediump vec3 yuv;
lowp vec3 rgb;

yuv.x = texture2D(SamplerY, _texcoord).r;
yuv.yz = texture2D(SamplerUV, _texcoord).rg - vec2(0.5, 0.5);

// BT.601, which is the standard for SDTV is provided as a reference

/* rgb = mat3(    1,       1,     1,
 0, -.34413, 1.772,
 1.402, -.71414,     0) * yuv;*/


// Using BT.709 which is the standard for HDTV
rgb = mat3(      1,       1,      1,
           0, -.18732, 1.8556,
           1.57481, -.46813,      0) * yuv;

   gl_FragColor = vec4(rgb, 1);

}

---

非常感谢，

我想问题在于YUV420（或I420）是一种三平面图像格式。I420是一个8位Y平面，后跟8位2x2二次采样U和V平面。GLCameraRipple中的代码需要NV12格式：8位Y平面，后跟一个带2x2子采样的交错U/V平面。鉴于此，我预计您将需要三种纹理。luma_tex，u_chroma_tex，v_chroma_tex

---

还请注意，GLCameraRipple也可能需要“视频范围”。换句话说，平面格式的值是luma=[16235]色度=[16240]。

您解码的是哪种视频？您是使用FFmpeg的libavcodec还是iOS的解码设备执行视频解码？那么这个应用程序有什么问题？您好，住户，我正在尝试做同样的事情，我还有一个绿色屏幕。你找到解决这个问题的办法了吗？谢谢大家!@住院医师：您的问题有解决方案吗？您的意思是kCVPixelFormatTypeƤYPCBCCR8BIPLANARFULLRANGE是NV12？