一、简介
Qt对音视频的播放和控制,相机拍照,收音机等多媒体应用提供了强大的支持。Qt5使用了全新的Qt Multimedia模块来实现多媒体应用,而原来Qt4中用于实现多媒体功能的Phonon模块已经被移除。
新的Qt Multimedia模块提供了丰富的接口,使读者可以轻松地使用平台的多媒体功能,例如,进行多媒体播放,使用相机和收音机等。该模块还分别提供了一组QML类型和一组C++类来处理多媒体内容。
C++ 中多媒体模块可以实现的功能,对应的示例程序以及需要使用的C++类如:
Qt的多媒体接口建立在底层平台的多媒体框架之上,这就意味着对于各种编解码器的支持依赖于使用的平台。如果要访问一些平台相关的设置,或者将Qt多媒体接口移植到新的平台,则可以参考Qt帮助中的Multimedia Backend Development文档。
另外,如果要使用多媒体模块中的内容,则需要在.pro项目文件中添加如下代码,引入多媒体模块:
QT += multimedia
二、几个主要类介绍
QMediaPlayer
提供给外部应用程序的主要API,应用程序可以通过调用其成员函数play,setVolume,setPosition等控制视频文件的播放。大部分成员函数都是通过调用QMediaPlayerControl类型指针的方法来实现的。
QMediaControl
控制媒体的抽象类,包含大量控制媒体的成员函数。
QMediaServiceProvider
提供媒体服务的抽象类,主要功能是requestService得到QMediaService对象。
QPluginServiceProvider
QPluginServiceProvider继承QMediaServiceProvider类,通过requestService方法给QMediaPlayer提供QMediaService。
QMediaService
媒体服务的抽象类,主要功能是requestControl得到QMediaControl对象。
QMediaServiceProviderPlugin
所有提供媒体服务的plugin都必须继承这个抽象类。create成员函数用来得到实现后的QMediaService派生类实例的指针,key成员函数用来得到一个QStringList,里面包含这个plugin中能提供的所有媒体服务的id。
注:由于一个plugin可能包含几个QMediaServiceProvider的实现,一个QMediaServiceProvider的实现又可能提供几个QMediaService的实现,一个QMediaService的实现也可能提供几个QMediaControl的实现...所以他们的每个派生类都有一个id来识别。
三、QMediaServiceProvider
提供媒体服务的抽象类,主要功能是requestService得到QMediaService对象
class QMediaService; class Q_MULTIMEDIA_EXPORT QMediaServiceProvider : public QObject { Q_OBJECT public: virtual QMediaService* requestService(const QByteArray &type, const QMediaServiceProviderHint &hint = QMediaServiceProviderHint()) = 0; virtual void releaseService(QMediaService *service) = 0; virtual QMediaServiceProviderHint::Features supportedFeatures(const QMediaService *service) const; virtual QMultimedia::SupportEstimate hasSupport(const QByteArray &serviceType, const QString &mimeType, const QStringList& codecs, int flags = 0) const; virtual QStringList supportedMimeTypes(const QByteArray &serviceType, int flags = 0) const; virtual QByteArray defaultDevice(const QByteArray &serviceType) const; virtual QList<QByteArray> devices(const QByteArray &serviceType) const; virtual QString deviceDescription(const QByteArray &serviceType, const QByteArray &device); virtual QCamera::Position cameraPosition(const QByteArray &device) const; virtual int cameraOrientation(const QByteArray &device) const; static QMediaServiceProvider* defaultServiceProvider(); static void setDefaultServiceProvider(QMediaServiceProvider *provider); };
四、QPluginServiceProvider
QPluginServiceProvider继承QMediaServiceProvider类,在requestService方法中相应的加载QMediaServiceProviderPlugin实现类(windows下的DSServicePlugin、WMFServicePlugin、Linux下的QGstreamerPlayerServicePlugin),在QMediaServiceProviderPlugin实现类中会创建对应的QMediaService。
class QPluginServiceProvider : public QMediaServiceProvider { struct MediaServiceData { QByteArray type; QMediaServiceProviderPlugin *plugin; MediaServiceData() : plugin(nullptr) { } }; QMap<const QMediaService*, MediaServiceData> mediaServiceData; public: // type: "org.qt-project.qt.mediaplayer" QMediaService* requestService(const QByteArray &type, const QMediaServiceProviderHint &hint) override { // ... // 选择合适的QMediaServiceProviderPlugin,有DSServicePlugin、WMFServicePlugin、QGstreamerPlayerServicePlugin // 相应的Plugin中创建对应的QMediaService } void releaseService(QMediaService *service) override { // ... } QMediaServiceProviderHint::Features supportedFeatures(const QMediaService *service) const override { // ... } QMultimedia::SupportEstimate hasSupport(const QByteArray &serviceType, const QString &mimeType, const QStringList& codecs, int flags) const override { // ... } QStringList supportedMimeTypes(const QByteArray &serviceType, int flags) const override { // ... } QByteArray defaultDevice(const QByteArray &serviceType) const override { // ... } QList<QByteArray> devices(const QByteArray &serviceType) const override { // ... } QString deviceDescription(const QByteArray &serviceType, const QByteArray &device) override { // ... } QCamera::Position cameraPosition(const QByteArray &device) const override { // ... } int cameraOrientation(const QByteArray &device) const override { // ... } };
五、QMediaServiceProviderPlugin
class Q_MULTIMEDIA_EXPORT QMediaServiceProviderPlugin : public QObject, public QMediaServiceProviderFactoryInterface { Q_OBJECT Q_INTERFACES(QMediaServiceProviderFactoryInterface) public: QMediaService* create(const QString& key) override = 0; void release(QMediaService *service) override = 0; };
5.1 DSServicePlugin
class DSServicePlugin : public QMediaServiceProviderPlugin , public QMediaServiceSupportedDevicesInterface , public QMediaServiceDefaultDeviceInterface , public QMediaServiceFeaturesInterface { #if QT_CONFIG(directshow_player) Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "directshow.json") #else Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "directshow_camera.json") #endif // ... }
5.2 WMFServicePlugin
class WMFServicePlugin : public QMediaServiceProviderPlugin , public QMediaServiceSupportedDevicesInterface , public QMediaServiceDefaultDeviceInterface , public QMediaServiceFeaturesInterface { #if QT_CONFIG(wmf_player) Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "wmf.json") #else Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "wmf_audiodecode.json") #endif // ... }
5.3 QGstreamerPlayerServicePlugin
class QGstreamerPlayerServicePlugin : public QMediaServiceProviderPlugin , public QMediaServiceFeaturesInterface , public QMediaServiceSupportedFormatsInterface { Q_PLUGIN_METADATA(IID "org.qt-project.qt.mediaserviceproviderfactory/5.0" FILE "mediaplayer.json") // ... }
六 gstreamer
gstreamer,其插件类为QGstreamerPlayerServicePlugin,其Service类为QGstreamerPlayerService。
#linux下播放插件为libgstmediaplayer.so,其链接到了libgstreamer-1.0.so.0上,最终使用了gstreamer ldd libgstmediaplayer.so | grep libgstreamer* libgstreamer-1.0.so.0 => /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0 (0x00007f66d65b6000)
ubuntu下的qt和gstreamer1.0环境报错问题解决
ubuntu16.04 18.04 Qt5.11安装Gstreamer
【FFMPEG】gstreamer插件调用ffmpeg 详解
七 ffmpeg
八 QMediaPlayer初始化过程
1、QMediaPlayer实例化
// playerService QMediaPlayer::QMediaPlayer(QObject *parent, QMediaPlayer::Flags flags): QMediaObject(*new QMediaPlayerPrivate, parent, playerService(flags)) // 详见1.1 { Q_D(QMediaPlayer); d->provider = QMediaServiceProvider::defaultServiceProvider(); if (d->service == nullptr) { d->error = ServiceMissingError; } else { // 此处control为QGstreamerPlayerService中创建的QGstreamerPlayerControl d->control = qobject_cast<QMediaPlayerControl*>(d->service->requestControl(QMediaPlayerControl_iid)); #ifndef QT_NO_BEARERMANAGEMENT QT_WARNING_PUSH QT_WARNING_DISABLE_DEPRECATED d->networkAccessControl = qobject_cast<QMediaNetworkAccessControl*>(d->service->requestControl(QMediaNetworkAccessControl_iid)); QT_WARNING_POP #endif if (d->control != nullptr) { connect(d->control, SIGNAL(mediaChanged(QMediaContent)), SLOT(_q_handleMediaChanged(QMediaContent))); connect(d->control, SIGNAL(stateChanged(QMediaPlayer::State)), SLOT(_q_stateChanged(QMediaPlayer::State))); connect(d->control, SIGNAL(mediaStatusChanged(QMediaPlayer::MediaStatus)), SLOT(_q_mediaStatusChanged(QMediaPlayer::MediaStatus))); connect(d->control, SIGNAL(error(int,QString)), SLOT(_q_error(int,QString))); connect(d->control, &QMediaPlayerControl::durationChanged, this, &QMediaPlayer::durationChanged); connect(d->control, &QMediaPlayerControl::positionChanged, this, &QMediaPlayer::positionChanged); connect(d->control, &QMediaPlayerControl::audioAvailableChanged, this, &QMediaPlayer::audioAvailableChanged); connect(d->control, &QMediaPlayerControl::videoAvailableChanged, this, &QMediaPlayer::videoAvailableChanged); connect(d->control, &QMediaPlayerControl::volumeChanged, this, &QMediaPlayer::volumeChanged); connect(d->control, &QMediaPlayerControl::mutedChanged, this, &QMediaPlayer::mutedChanged); connect(d->control, &QMediaPlayerControl::seekableChanged, this, &QMediaPlayer::seekableChanged); connect(d->control, &QMediaPlayerControl::playbackRateChanged, this, &QMediaPlayer::playbackRateChanged); connect(d->control, &QMediaPlayerControl::bufferStatusChanged, this, &QMediaPlayer::bufferStatusChanged); d->state = d->control->state(); d->status = d->control->mediaStatus(); if (d->state == PlayingState) addPropertyWatch("position"); if (d->status == StalledMedia || d->status == BufferingMedia) addPropertyWatch("bufferStatus"); d->hasStreamPlaybackFeature = d->provider->supportedFeatures(d->service).testFlag(QMediaServiceProviderHint::StreamPlayback); d->audioRoleControl = qobject_cast<QAudioRoleControl*>(d->service->requestControl(QAudioRoleControl_iid)); if (d->audioRoleControl) { connect(d->audioRoleControl, &QAudioRoleControl::audioRoleChanged, this, &QMediaPlayer::audioRoleChanged); d->customAudioRoleControl = qobject_cast<QCustomAudioRoleControl *>( d->service->requestControl(QCustomAudioRoleControl_iid)); if (d->customAudioRoleControl) { connect(d->customAudioRoleControl, &QCustomAudioRoleControl::customAudioRoleChanged, this, &QMediaPlayer::customAudioRoleChanged); } } } #ifndef QT_NO_BEARERMANAGEMENT if (d->networkAccessControl != nullptr) { QT_WARNING_PUSH QT_WARNING_DISABLE_DEPRECATED connect(d->networkAccessControl, &QMediaNetworkAccessControl::configurationChanged, this, &QMediaPlayer::networkConfigurationChanged); QT_WARNING_POP } #endif } }
1.1 playerService
// 通过静态方法获取QMediaService static QMediaService *playerService(QMediaPlayer::Flags flags) { // 获取默认的QMediaServiceProvider ==> 1.1.1 QMediaServiceProvider *provider = QMediaServiceProvider::defaultServiceProvider(); if (flags) { QMediaServiceProviderHint::Features features; if (flags & QMediaPlayer::LowLatency) features |= QMediaServiceProviderHint::LowLatencyPlayback; if (flags & QMediaPlayer::StreamPlayback) features |= QMediaServiceProviderHint::StreamPlayback; if (flags & QMediaPlayer::VideoSurface) features |= QMediaServiceProviderHint::VideoSurface; return provider->requestService(Q_MEDIASERVICE_MEDIAPLAYER, QMediaServiceProviderHint(features)); } // ==> 1.1.2 QPluginServiceProvider->requestService return provider->requestService(Q_MEDIASERVICE_MEDIAPLAYER); }
1.1.1 defaultServiceProvider
QMediaServiceProvider *QMediaServiceProvider::defaultServiceProvider() { // qt_defaultMediaServiceProvider默认为空, // pluginProvider()是由Q_GLOBAL_STATIC(QPluginServiceProvider, pluginProvider)定义, // 返回类型为QPluginServiceProvider类型的对象 return qt_defaultMediaServiceProvider != nullptr ? qt_defaultMediaServiceProvider : static_cast<QMediaServiceProvider *>(pluginProvider()); } // 定义命名空间为pluginProvider的QPluginServiceProvider对象 ? Q_GLOBAL_STATIC(QPluginServiceProvider, pluginProvider);