如何使用C#创建Windows Webcam应用

最近想用C#写一个camera的应用。搜索了Google和StackOverflow，发现大部分的sample用了WIA或者DirectShow。WIA和DirectShow都没有直接提供C#接口，所以实现起来也比较复杂。试着运行了几个WIA的工程，在Windows10上还不起作用。发现微软提供了一个MediaCapture类，包含了丰富的C#接口，可用于音频，视频。要用这个类，就需要创建一个UWP工程。这里分享下如何使用MediaCapture的C#接口来获取camera的每一帧数据。

微软示例

微软在GitHub上放了大量的UWP示例，里面包含了各种camera的接口使用方法。

在Android中要获取camera的每一帧数据，可以通过onPreviewCallback的回调函数实现。参考CameraFrames这个示例，也可以实现类似的功能。

如何创建Windows Webcam应用

1. 在package.appxmanifest中获取webcam权限：

2. 创建image element用于绘制webcam的预览界面：

3. 通过Image Element初始化FrameRenderer ：

_frameRenderer = new FrameRenderer(PreviewImage);

4. 初始化MediaCapture对象：

// Create a new media capture object.
_mediaCapture = new MediaCapture();
 
var settings = new MediaCaptureInitializationSettings()
{
    // Select the source we will be reading from.
    SourceGroup = groupModel.SourceGroup,
 
    // This media capture has exclusive control of the source.
    SharingMode = MediaCaptureSharingMode.ExclusiveControl,
 
    // Set to CPU to ensure frames always contain CPU SoftwareBitmap images,
    // instead of preferring GPU D3DSurface images.
    MemoryPreference = MediaCaptureMemoryPreference.Cpu,
 
    // Capture only video. Audio device will not be initialized.
    StreamingCaptureMode = StreamingCaptureMode.Video,
};
 
try
{
    // Initialize MediaCapture with the specified group.
    // This can raise an exception if the source no longer exists,
    // or if the source could not be initialized.
    await _mediaCapture.InitializeAsync(settings);
    _logger.Log($"Successfully initialized MediaCapture for {groupModel.DisplayName}");
}
catch (Exception exception)
{
    _logger.Log(exception.Message);
    DisposeMediaCapture();
}

5. 使用DeviceWatcher 列出所有设备：

var deviceSelector = MediaFrameSourceGroup.GetDeviceSelector();
_watcher = DeviceInformation.CreateWatcher(deviceSelector);
_watcher.Added += Watcher_Added;
_watcher.Removed += Watcher_Removed;
_watcher.Updated += Watcher_Updated;
_watcher.Start();
 
private async void Watcher_Added(DeviceWatcher sender, DeviceInformation args)
{
    await AddDeviceAsync(args.Id);
}
 
private async Task AddDeviceAsync(string id)
{
    var group = await MediaFrameSourceGroup.FromIdAsync(id);
    if (group != null)
    {
        await _dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
            {
                _sourceCollection.Add(new FrameSourceGroupModel(group));
            });
    }
}

6. 针对用户的选择更新设备源：

_mediaCapture.FrameSources.TryGetValue(info.SourceInfo.Id, out _source);

7. 通过MediaFrameReader 注册回调函数：

if (_source != null)
{
    _reader = await _mediaCapture.CreateFrameReaderAsync(_source);
    _reader.FrameArrived += Reader_FrameArrived;
}

8. 启动webcam：

MediaFrameReaderStartStatus result = await _reader.StartAsync();

9. 读取并绘制每一帧数据：

private void Reader_FrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
{
    // TryAcquireLatestFrame will return the latest frame that has not yet been acquired.
    // This can return null if there is no such frame, or if the reader is not in the
    // "Started" state. The latter can occur if a FrameArrived event was in flight
    // when the reader was stopped.
    using (var frame = sender.TryAcquireLatestFrame())
    {
        _frameRenderer.ProcessFrame(frame);
    }
}
 
public void ProcessFrame(MediaFrameReference frame)
{
    var softwareBitmap = FrameRenderer.ConvertToDisplayableImage(frame?.VideoMediaFrame);
 
    if (softwareBitmap != null)
    {
        // Swap the processed frame to _backBuffer and trigger UI thread to render it
        softwareBitmap = Interlocked.Exchange(ref _backBuffer, softwareBitmap);
 
        // UI thread always reset _backBuffer before using it.  Unused bitmap should be disposed.
        softwareBitmap?.Dispose();
 
        // Changes to xaml ImageElement must happen in UI thread through Dispatcher
        var task = _imageElement.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
            async () =>
            {
                // Don't let two copies of this task run at the same time.
                if (_taskRunning)
                {
                    return;
                }
                _taskRunning = true;
 
                // Keep draining frames from the backbuffer until the backbuffer is empty.
                SoftwareBitmap latestBitmap;
                while ((latestBitmap = Interlocked.Exchange(ref _backBuffer, null)) != null)
                {
                    var imageSource = (SoftwareBitmapSource)_imageElement.Source;
                    await imageSource.SetBitmapAsync(latestBitmap);
                    latestBitmap.Dispose();
                }
 
                _taskRunning = false;
            });
    }
}

参考

Basic photo, video, and audio capture with MediaCapture

源码

https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CameraFrames