ios – Does AVAssetWriter somehow wait for GPU rendering affecting a CVPixelBuffer to finish?


Does AVAssetWriter (or AVAssetWriterInputPixelBufferAdaptor) do something internally to wait for the GPU to finish modifying any pixel buffers I feed it? It seems as though it might, unless I am missing some side effect of another part of my rendering procedure, which operates roughly as follows:

  1. Receive a pixel buffer inside captureOutput(_ captureOutput: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection)
  2. Create a MTLTexture backed by that pixel buffer using CVMetalTextureCacheCreateTextureFromImage
  3. Create a command buffer and encode a compute pipeline which modifies the texture from (2). Commit the command buffer.
  4. Pass the original pixel buffer to an AVAssetWriter by way of AVAssetWriterInputPixelBufferAdaptor.append()

Even if I forego using MTLCommandBuffer.waitUntilCompleted() before step 4 (or similarly waiting for the GPU to finish using MTLCommandBuffer.addCompletedHandler), I don’t get unmodified pixel buffers written to the output video file as one might expect. To verify, I did the following:

  1. Added a hugely expensive but largely pointless step to the shader that modifies each incoming texture. Something along the lines of
kernel void exampleKernel(texture2d<half, access::read_write> texture [[ texture(0) ]],
                          uint2 gid [[thread_position_in_grid]]) {

    // Irrelevant actual shader code here

    for (int i = 0; i < 10000; i++) {
        outputColor.r += float(i) * 0.00000000001;
    }

    // Write outputColor back to the texture
}
  1. Set up log entries to record

    a) When each pixel buffer is appended, along with its memory address

    b) When the command buffer modifying each pixel buffer actually completes (using addCompletedHandler)

The resulting log looks something like this, showing that received pixel buffers are passed on to the AVAssetWriter well before their respective GPU commands finish. Despite this, the output file shows all the rendering effects.

Appending <CVPixelBuffer 0x282b6acb0> // <--- Happens almost immediately after pixel buffer is received
Finished rendering <CVPixelBuffer [some other address]>
Finished rendering <CVPixelBuffer [some other address]>
Appending <CVPixelBuffer [some other address]>
Finished rendering <CVPixelBuffer [some other address]>
Appending <CVPixelBuffer [some other address]>
Finished rendering <CVPixelBuffer [some other address]>
Finished rendering <CVPixelBuffer 0x282b6acb0> // <--- Happens a considerable time later, once the extremely slow shader finishes executing, yet the output file contains the results of this rendering, not the unmodified pixel buffer!

Latest articles

spot_imgspot_img

Related articles

Leave a reply

Please enter your comment!
Please enter your name here

spot_imgspot_img