I’m trying to do some realtime image processing using Xamarin.iOS C#, but seems like the AVCaptureVideoDataOutputSampleBufferDelegate is hanging after about a dozen samples. If I touch the screen, it will startup for another dozen samples and stops again.
The basis for this code is a Swift example here: https://anuragajwani.medium.com/how-to-process-images-real-time-from-the-ios-camera-9c416c531749 which I converted to a Xamarin project.
public class UICameraPreview : UIView, IAVCaptureVideoDataOutputSampleBufferDelegate
{
public AVCaptureSession captureSession = new AVCaptureSession();
private AVCaptureVideoPreviewLayer previewLayer;
private AVCaptureVideoDataOutput videoOutput;
public UICameraPreview(CameraOptions options)
{
AddCameraInput();
AddPreviewLayer(); ;
AddVideoOutput();
Task.Run(() =>
{
captureSession.StartRunning();
});
}
public override void LayoutSubviews()
{
base.LayoutSubviews();
previewLayer.Frame = Bounds;
}
[Export("captureOutput:didOutputSampleBuffer:fromConnection:")]
public void DidOutputSampleBuffer(AVCaptureOutput output, CMSampleBuffer sampleBuffer, AVCaptureConnection connection)
{
var frame = sampleBuffer.GetImageBuffer();
if (frame == null)
{
Console.WriteLine("unable to get image from sample buffer");
return;
}
Console.WriteLine("did receive image frame");
// process image here
}
private void AddCameraInput()
{
...
}
private void AddPreviewLayer()
{
previewLayer = new AVCaptureVideoPreviewLayer(captureSession)
{
VideoGravity = AVLayerVideoGravity.ResizeAspect
};
Layer.AddSublayer(previewLayer);
}
private void AddVideoOutput()
{
videoOutput = new AVCaptureVideoDataOutput();
var videoSettings = new NSDictionary<NSString, NSObject>(new NSString("PixelFormatType"), new NSNumber((int)CVPixelFormatType.CV420YpCbCr8BiPlanarFullRange));
videoOutput.WeakVideoSettings = videoSettings;
videoOutput.SetSampleBufferDelegateQueue(this, DispatchQueue.MainQueue);
captureSession.AddOutput(videoOutput);
}
}```
sampleBuffer.GetImageBuffer() works as expected...I just need it to run without hanging.
I have tried sampleBuffer.Dispose() as well but I'm thinking some resource has to be released somewhere.
Nothing I have tried works except adding a delay of 3 seconds or more in the delegate. I believe it some kind memory leak, I am at a loss as to how find a solution.
