Ios 使用metalkit导出拼贴视频

如何使用不同分辨率的视频导出拼贴视频？我正在尝试实现如下第一张图片，我正在使用演示，到目前为止已经完成了，我创建了

AVMutableVideoComposition

将所有视频轨迹ID传递给

customVideoCompositorClass

并获取所有视频

CVPixelBuffer

，然后在

MTLTexture

中转换以渲染所有纹理，但问题是我的视频输出大小是方形的（destinationTexture）视频的大小是纵向或横向的，这就是为什么每个视频都会被压缩的原因。我如何旋转缩放位置和遮罩形状每个视频？还有，如何应用cifilters？我是否应该将每个

CVPixelBuffer

转换为ciImage，并将ciImage转换回

CVPixelBuffer

---

我建议使用一个名为MetalPetal的库。这是一个基于金属的图像处理框架。您必须将CVPixelBuffer转换为MTIImage的MetalImage。然后你可以在图像中做任何事情，比如有预先制作的过滤器，你可以应用到它，或者你甚至可以使用CIFilter或你的自定义过滤器，你可以变换、旋转、裁剪每一帧，以便拼贴帧是准确的。然后，您必须再次将MTIimage转换为cvpixelbuffer。在这里，你也可以使用CIImage，但我想它会很慢。您可能会得到渲染大小的长方体图像。请查看渲染大小

override func renderPixelBuffer(backgroundTexture: MTLTexture,
                                firstPixelBuffer: CVPixelBuffer,
                                secondPixelBuffer: CVPixelBuffer,
                                thirdPixelBuffer: CVPixelBuffer,
                                fourthPixelBuffer: CVPixelBuffer,
                                destinationPixelBuffer: CVPixelBuffer) {

    // Create a MTLTexture from the CVPixelBuffer.
    guard let firstTexture = buildTextureForPixelBuffer(firstPixelBuffer) else { return }
    guard let secondTexture = buildTextureForPixelBuffer(secondPixelBuffer) else { return }
    guard let thirdTexture = buildTextureForPixelBuffer(thirdPixelBuffer) else { return }
    guard let fourthTexture = buildTextureForPixelBuffer(fourthPixelBuffer) else { return }
    guard let destinationTexture = buildTextureForPixelBuffer(destinationPixelBuffer) else { return }

    /*
     We must maintain a reference to the pixel buffer until the Metal rendering is complete. This is because the
     'buildTextureForPixelBuffer' function above uses CVMetalTextureCacheCreateTextureFromImage to create a
     Metal texture (CVMetalTexture) from the IOSurface that backs the CVPixelBuffer, but
     CVMetalTextureCacheCreateTextureFromImage doesn't increment the use count of the IOSurface; only the
     CVPixelBuffer, and the CVMTLTexture own this IOSurface. Therefore we must maintain a reference to either
     the pixel buffer or Metal texture until the Metal rendering is done. The MTLCommandBuffer completion
     handler below is then used to release these references.
     */

    pixelBuffers = RenderPixelBuffers(firstBuffer: firstPixelBuffer,
                                      secondBuffer: secondPixelBuffer,
                                      thirdBuffer: thirdPixelBuffer,
                                      fourthBuffer: fourthPixelBuffer,
                                      destinationBuffer: destinationPixelBuffer)

    // Create a new command buffer for each renderpass to the current drawable.
    let commandBuffer = commandQueue.makeCommandBuffer()!
    commandBuffer.label = "MyCommand"

    /*
     Obtain a drawable texture for this render pass and set up the renderpass
     descriptor for the command encoder to render into.
     */
    let renderPassDescriptor = setupRenderPassDescriptorForTexture(destinationTexture)

    // Create a render command encoder so we can render into something.
    let renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor)!
    renderEncoder.label = "MyRenderEncoder"

    guard let renderPipelineState = renderPipelineState else { return }

    modelConstants.modelViewMatrix = matrix_identity_float4x4

    // Render background texture.
    renderTexture(renderEncoder, texture: backgroundTexture, pipelineState: renderPipelineState)

    var translationMatrix = matrix_float4x4(translation: simd_float3(-0.5, 0.5, 0))
    // var rotationMatrix = matrix_float4x4(rotationZ: radians(fromDegrees: -90))
    var scaleMatrix = matrix_float4x4(scaling: 0.25)
    var modelMatrix = translationMatrix * scaleMatrix
    modelConstants.modelViewMatrix = modelMatrix

    // Render first texture.
    renderTexture(renderEncoder, texture: firstTexture, pipelineState: renderPipelineState)

    //        translationMatrix = matrix_float4x4(translation: simd_float3(0.5, -0.5, 0))
    //        rotationMatrix = matrix_float4x4(rotationZ: radians(fromDegrees: -45))
    //        scaleMatrix = matrix_float4x4(scaling: 0.5)
    //        modelMatrix = translationMatrix * scaleMatrix * rotationMatrix
    //        modelConstants.modelViewMatrix = modelMatrix

    //        // Render second texture.
    //        renderTexture(renderEncoder, texture: secondTexture, pipelineState: renderPipelineState)
    //
    //        // Render third texture.
    //        renderTexture(renderEncoder, texture: thirdTexture, pipelineState: renderPipelineState)
    //
    //        // Render fourth texture.
    //        renderTexture(renderEncoder, texture: fourthTexture, pipelineState: renderPipelineState)

    // We're done encoding commands.
    renderEncoder.endEncoding()

    // Use the command buffer completion block to release the reference to the pixel buffers.
    commandBuffer.addCompletedHandler({ _ in
        self.pixelBuffers = nil // Release the reference to the pixel buffers.
    })

    // Finalize rendering here & push the command buffer to the GPU.
    commandBuffer.commit()
}