iOS-如何使用openGL绘制YUV图像

目前，我正在尝试使用openGL绘制图像（图像经常更新，因此必须重新绘制）。以前，我将我的图像从YUV转换为RGB，然后使用这个新图像用openGL绘制。所有这些都很好，但转换过程并不是特别快

我现在尝试更改代码，以便在openGL着色器中处理转换。环顾四周后，我发现了一些代码片段（特别是着色器和我的渲染功能的大部分），它们帮助我获得了基线，但我似乎无法正确地绘制图像-我所看到的只是一幅黑色图像

我很可能错过了一些非常简单和重要的东西——我使用openGL的经验相当有限。如果有人能看一看，看看他们是否认识到任何错误，请让我知道

我应该指出，我正在尝试支持iOS 4.x，因此CVOpenGlestextureCacheCheCreateTureFromimage不应该是可用的（即使我想，我也不确定如何设置以使用它）

任何帮助都将不胜感激。代码如下-

我的顶点着色器：

attribute vec4 position;
attribute vec4 inputTextureCoordinate;

varying vec2 textureCoordinate;

void main()
{
    gl_Position = position;
    textureCoordinate = inputTextureCoordinate.xy;
}

片段着色器：

#ifdef GL_ES
precision mediump float;
#endif

varying vec2 textureCoordinate;

uniform sampler2D videoFrame;
uniform sampler2D videoFrameUV;

const mat3 yuv2rgb = mat3(
                          1, 0, 1.2802,
                          1, -0.214821, -0.380589,
                          1, 2.127982, 0
                          );

void main() {
    vec3 yuv = vec3(
                    1.1643 * (texture2D(videoFrame, textureCoordinate).r - 0.0625),
                    texture2D(videoFrameUV, textureCoordinate).r - 0.5,
                    texture2D(videoFrameUV, textureCoordinate).a - 0.5
                    );
    vec3 rgb = yuv * yuv2rgb;

    gl_FragColor = vec4(rgb, 1.0);
}

渲染功能：

-(void)renderImage:(ImageBuffer *)image
{
    if (image)
    {        
        int bufferHeight = image->GetHeight();
        int bufferWidth = image->GetWidth();

        if(!imageTexture){
            // Dealing with the Y portion of the YCbCr
            glActiveTexture(GL_TEXTURE0);
            glGenTextures(1, &imageTexture);
            //Bind Y texture
            glBindTexture(GL_TEXTURE_2D, imageTexture);
            glUniform1i(uniforms[UNIFORM_VIDEOFRAME], 0);
            // For fitting
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
            // This is necessary for non-power-of-two textures
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

            glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, bufferWidth, bufferHeight, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, image->GetPlanePointer(0));

            // Dealing with the CbCr potion of the YCbCr
            glActiveTexture(GL_TEXTURE1);
            glGenTextures(1, &imageTextureUV);
            //Bind CbCr texture
            glBindTexture(GL_TEXTURE_2D, imageTextureUV);
            glUniform1i(uniforms[UNIFORM_VIDEOFRAMEUV], 1);
            // For fitting
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
            // This is necessary for non-power-of-two textures
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

            glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE_ALPHA, bufferWidth/2, bufferHeight/2, 0, GL_LUMINANCE_ALPHA, GL_UNSIGNED_BYTE, image->GetPlanePointer(1));

        }

        [self drawFrame];
    }
}

- (void)drawFrame
{
    [self setFramebuffer];

    // Replace the implementation of this method to do your own custom drawing.
    static const GLfloat squareVertices[] = {
        -1.0f, -1.0f,
        1.0f, -1.0f,
        -1.0f, 1.0f,
        1.0f, 1.0f
    };

    static const GLfloat textureVertices[] = {
//        1.0f, 1.0f,
//        1.0f, 0.0f,
//        0.0f,  1.0f,
//        0.0f,  0.0f,
        0.0f, 1.0f,
        1.0f, 1.0f,
        0.0f, 0.0f,
        1.0f, 0.0f
//        0.0f, 0.8f,
//        1.0f, 0.8f,
//        0.0f, 0.2f,
//        1.0f, 0.2f
    };

    static float transY = 0.0f;


    if ([context API] == kEAGLRenderingAPIOpenGLES2) {
        // Use shader program.
        glUseProgram(program);

        glActiveTexture(GL_TEXTURE0);
        glBindTexture(GL_TEXTURE_2D, imageTexture);

        // Update attribute values.
        glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, squareVertices);
        glEnableVertexAttribArray(ATTRIB_VERTEX);
        glVertexAttribPointer(ATTRIB_TEXTUREPOSITON, 2, GL_FLOAT, 0, 0, textureVertices);
        glEnableVertexAttribArray(ATTRIB_TEXTUREPOSITON);

        glActiveTexture(GL_TEXTURE1);
        glBindTexture(GL_TEXTURE_2D, imageTextureUV);

        // Update attribute values.
        glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, squareVertices);
        glEnableVertexAttribArray(ATTRIB_VERTEX);
        glVertexAttribPointer(ATTRIB_TEXTUREPOSITON, 2, GL_FLOAT, 0, 0, textureVertices);
        glEnableVertexAttribArray(ATTRIB_TEXTUREPOSITON);

        // Validate program before drawing. This is a good check, but only really necessary in a debug build.
        // DEBUG macro must be defined in your debug configurations if that's not already the case.
#if defined(DEBUG)
        if (![self validateProgram:program]) {
            NSLog(@"Failed to validate program: %d", program);
            return;
        }
#endif
    } else {
        glMatrixMode(GL_PROJECTION);
        glLoadIdentity();
        glMatrixMode(GL_MODELVIEW);
        glLoadIdentity();
        glTranslatef(0.0f, (GLfloat)(sinf(transY)/2.0f), 0.0f);
        transY += 0.075f;

        glVertexPointer(2, GL_FLOAT, 0, squareVertices);
        glEnableClientState(GL_VERTEX_ARRAY);
    } 

    glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

    [self presentFramebuffer];
}

最后是drawFrame函数：

-(void)renderImage:(ImageBuffer *)image
{
    if (image)
    {        
        int bufferHeight = image->GetHeight();
        int bufferWidth = image->GetWidth();

        if(!imageTexture){
            // Dealing with the Y portion of the YCbCr
            glActiveTexture(GL_TEXTURE0);
            glGenTextures(1, &imageTexture);
            //Bind Y texture
            glBindTexture(GL_TEXTURE_2D, imageTexture);
            glUniform1i(uniforms[UNIFORM_VIDEOFRAME], 0);
            // For fitting
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
            // This is necessary for non-power-of-two textures
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

            glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, bufferWidth, bufferHeight, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, image->GetPlanePointer(0));

            // Dealing with the CbCr potion of the YCbCr
            glActiveTexture(GL_TEXTURE1);
            glGenTextures(1, &imageTextureUV);
            //Bind CbCr texture
            glBindTexture(GL_TEXTURE_2D, imageTextureUV);
            glUniform1i(uniforms[UNIFORM_VIDEOFRAMEUV], 1);
            // For fitting
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
            // This is necessary for non-power-of-two textures
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
            glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

            glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE_ALPHA, bufferWidth/2, bufferHeight/2, 0, GL_LUMINANCE_ALPHA, GL_UNSIGNED_BYTE, image->GetPlanePointer(1));

        }

        [self drawFrame];
    }
}

- (void)drawFrame
{
    [self setFramebuffer];

    // Replace the implementation of this method to do your own custom drawing.
    static const GLfloat squareVertices[] = {
        -1.0f, -1.0f,
        1.0f, -1.0f,
        -1.0f, 1.0f,
        1.0f, 1.0f
    };

    static const GLfloat textureVertices[] = {
//        1.0f, 1.0f,
//        1.0f, 0.0f,
//        0.0f,  1.0f,
//        0.0f,  0.0f,
        0.0f, 1.0f,
        1.0f, 1.0f,
        0.0f, 0.0f,
        1.0f, 0.0f
//        0.0f, 0.8f,
//        1.0f, 0.8f,
//        0.0f, 0.2f,
//        1.0f, 0.2f
    };

    static float transY = 0.0f;


    if ([context API] == kEAGLRenderingAPIOpenGLES2) {
        // Use shader program.
        glUseProgram(program);

        glActiveTexture(GL_TEXTURE0);
        glBindTexture(GL_TEXTURE_2D, imageTexture);

        // Update attribute values.
        glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, squareVertices);
        glEnableVertexAttribArray(ATTRIB_VERTEX);
        glVertexAttribPointer(ATTRIB_TEXTUREPOSITON, 2, GL_FLOAT, 0, 0, textureVertices);
        glEnableVertexAttribArray(ATTRIB_TEXTUREPOSITON);

        glActiveTexture(GL_TEXTURE1);
        glBindTexture(GL_TEXTURE_2D, imageTextureUV);

        // Update attribute values.
        glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, squareVertices);
        glEnableVertexAttribArray(ATTRIB_VERTEX);
        glVertexAttribPointer(ATTRIB_TEXTUREPOSITON, 2, GL_FLOAT, 0, 0, textureVertices);
        glEnableVertexAttribArray(ATTRIB_TEXTUREPOSITON);

        // Validate program before drawing. This is a good check, but only really necessary in a debug build.
        // DEBUG macro must be defined in your debug configurations if that's not already the case.
#if defined(DEBUG)
        if (![self validateProgram:program]) {
            NSLog(@"Failed to validate program: %d", program);
            return;
        }
#endif
    } else {
        glMatrixMode(GL_PROJECTION);
        glLoadIdentity();
        glMatrixMode(GL_MODELVIEW);
        glLoadIdentity();
        glTranslatef(0.0f, (GLfloat)(sinf(transY)/2.0f), 0.0f);
        transY += 0.075f;

        glVertexPointer(2, GL_FLOAT, 0, squareVertices);
        glEnableClientState(GL_VERTEX_ARRAY);
    } 

    glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

    [self presentFramebuffer];
}

---

下面是我的项目“iOS电影播放器”中的一些代码片段

1.片段着色器 2.从YUV420p帧创建纹理 4.渲染帧

[EAGLContext setCurrentContext:_context]；
glBindFramebuffer（GL\u FRAMEBUFFER，\u FRAMEBUFFER）；
glViewport（0,0，_backingWidth，_backingHeight）；
glClearColor（0.0f、0.0f、0.0f、1.0f）；
glClear（GLU颜色缓冲位）；
glUseProgram（_程序）；
对于（int i=0；i<3；++i）{
玻璃结构（GL_纹理0+i）；
glBindTexture（GL_TEXTURE_2D，_textures[i]）；
gluniformm1i（_uniformSamplers[i]，i）；
}  
GLVertexAttributePointer（属性\顶点，2，GLU浮点，0，0，顶点）；
GlenableVertexAttributeArray（属性_顶点）；
glvertexattributepointer（属性_TEXCOORD，2，GL_FLOAT，0，0，texCoords）；
GlenableVertexAttributeArray（属性_TEXCOORD）；
glbinderbuffer（GL_RENDERBUFFER，_RENDERBUFFER）；
[_contextpresentrenderbuffer:GL_RENDERBUFFER]；

---

调用.renderImage:之前是否编译过着色器程序？你可能需要放一个

glUseProgram（程序）glUniform1i（）

以确保程序处于活动状态并且正确设置了纹理之前，请执行code>。此外，在执行此操作之前，请确保已设置了当前上下文。在过去，有两件事让我绊倒了。是的，我在一个单独的函数中进行了各种基本的设置，这个函数叫做前面的函数——设置上下文、编译着色器和链接程序，所有这些都应该在调用renderImage（）之前完成。在glUniform1i（）调用之前，我已经移动了一个glUseProgram（）调用，但不幸的是，似乎没有改变任何东西。请原谅我不知道iOS的OpenGL ES中公开了什么，但PowerVR GPU硬件可以解码某些YUV纹理（至少根据。没有OpenGL ES扩展吗？

static const GLfloat texCoords[] = { 0.0f, 1.0f, 1.0f, 1.0f, 0.0f, 0.0f, 1.0f, 0.0f };  
static const GLfloat vertices[]= {-1.0f, -1.0f, 1.0f, -1.0f, -1.0f, 1.0f, 1.0f, 1.0f };

[EAGLContext setCurrentContext:_context];  
glBindFramebuffer(GL_FRAMEBUFFER, _framebuffer);  
glViewport(0, 0, _backingWidth, _backingHeight);  
glClearColor(0.0f, 0.0f, 0.0f, 1.0f);  
glClear(GL_COLOR_BUFFER_BIT);  
glUseProgram(_program);  
for (int i = 0; i < 3; ++i) {  
    glActiveTexture(GL_TEXTURE0 + i);  
    glBindTexture(GL_TEXTURE_2D, _textures[i]);  
    glUniform1i(_uniformSamplers[i], i);  
}  
glVertexAttribPointer(ATTRIBUTE_VERTEX, 2, GL_FLOAT, 0, 0, vertices);  
glEnableVertexAttribArray(ATTRIBUTE_VERTEX);  
glVertexAttribPointer(ATTRIBUTE_TEXCOORD, 2, GL_FLOAT, 0, 0, texCoords);  
glEnableVertexAttribArray(ATTRIBUTE_TEXCOORD);  
glBindRenderbuffer(GL_RENDERBUFFER, _renderbuffer);  
[_context presentRenderbuffer:GL_RENDERBUFFER];