Linux平台x86_64|aarch64架构如何实现轻量级RTSP服务

本文涉及的产品
视觉智能开放平台,图像通用资源包5000点
视觉智能开放平台,分割抠图1万点
视觉智能开放平台,视频通用资源包5000点
简介: 为满足在Linux平台(x86_64与aarch64架构)上实现轻量级RTSP服务的需求,我们开发了一套解决方案。该方案通过调用`start_rtsp_server()`函数启动RTSP服务,并设置端口号及认证信息。支持AAC音频和H.264视频编码,可推送纯音频、纯视频或音视频流。此外,还支持X11屏幕采集、部分V4L2摄像头采集、帧率/GOP/码率调整、摄像头设备选择与预览等功能。对于音频采集,支持alsa-lib和libpulse接口。整体设计旨在提供150-400ms的低延迟体验,适用于多种应用场景。

技术背景

我们在做Linux平台x86_64架构或aarch64架构的推送模块的时候,有公司提出这样的技术需求,希望在Linux平台,实现轻量级RTSP服务,实现对摄像头或屏幕对外RTSP拉流,同步到大屏上去。

技术实现

废话不多说,直接上代码,先调用start_rtsp_server()指定端口号,启动RTSP服务。

image.gif

LogInit();
    NT_SmartPublisherSDKAPI push_api;
    if (!PushSDKInit(push_api))
    {
        XDestroyWindow(display, sub_wid);
        XDestroyWindow(display, main_wid);
        XCloseDisplay(display);
        return 0;
    }
    // auto rtsp_server_handle = start_rtsp_server(&push_api, 8554, "test", "12345");
    auto rtsp_server_handle = start_rtsp_server(&push_api, 8554, "", "");
    if (nullptr == rtsp_server_handle) {
        fprintf(stderr, "start_rtsp_server failed.\n");
        XDestroyWindow(display, sub_wid);
        XDestroyWindow(display, main_wid);
        XCloseDisplay(display);
        push_api.UnInit();
        return 0;
    }
    auto push_handle = open_config_instance(&push_api, 20);
    if (nullptr == push_handle) {
        fprintf(stderr, "open_config_instance failed.\n");
        XDestroyWindow(display, sub_wid);
        XDestroyWindow(display, main_wid);
        XCloseDisplay(display);
        stop_rtsp_server(&push_api, rtsp_server_handle);
        push_api.UnInit();
        return 0;
    }

image.gif

PushSDKInit()实现如下:

/*
     * publisherdemo.cpp
     * Author: daniusdk.com
     */
    bool PushSDKInit(NT_SmartPublisherSDKAPI& push_api)
    {
        memset(&push_api, 0, sizeof(push_api));
        NT_GetSmartPublisherSDKAPI(&push_api);
        auto ret = push_api.Init(0, nullptr);
        if (NT_ERC_OK != ret)
        {
            fprintf(stderr, "push_api.Init failed!\n");
            return false;
        }
        else
        {
            fprintf(stdout, "push_api.Init ok!\n");
        }
        return true;
    }

image.gif

启动RTSP服务对应的代码如下:

NT_HANDLE start_rtsp_server(NT_SmartPublisherSDKAPI* push_api, int port, std::string user_name, std::string password) {
        NT_HANDLE rtsp_server_handle = nullptr;
        if (NT_ERC_OK != push_api->OpenRtspServer(&rtsp_server_handle, 0)) {
            fprintf(stderr, "OpenRtspServer failed\n");
            return nullptr;
        }
        if (nullptr == rtsp_server_handle) {
            fprintf(stderr, "rtsp_server_handle is null\n");
            return nullptr;
        }
        if (NT_ERC_OK != push_api->SetRtspServerPort(rtsp_server_handle, port)) {
            push_api->CloseRtspServer(rtsp_server_handle);
            return nullptr;
        }
        if (!user_name.empty() && !password.empty())
            push_api->SetRtspServerUserNamePassword(rtsp_server_handle, user_name.c_str(), password.c_str());
        if (NT_ERC_OK == push_api->StartRtspServer(rtsp_server_handle, 0))
            return rtsp_server_handle;
        fprintf(stderr, "StartRtspServer failed\n");
        push_api->CloseRtspServer(rtsp_server_handle);
        
        return nullptr;
    }

image.gif

open_config_instance()实现如下,可以获取摄像头或屏幕数据,并做基础的编码等参数配置:

NT_HANDLE open_config_instance(NT_SmartPublisherSDKAPI* push_api, int dst_fps) {
        NT_INT32 pulse_device_number = 0;
        if (NT_ERC_OK == push_api->GetAuidoInputDeviceNumber(2, &pulse_device_number))
        {
            fprintf(stdout, "[daniusdk.com]Pulse device num:%d\n", pulse_device_number);
            char device_name[512];
            for (auto i = 0; i < pulse_device_number; ++i)
            {
                if (NT_ERC_OK == push_api->GetAuidoInputDeviceName(2, i, device_name, 512))
                {
                    fprintf(stdout, "[daniusdk.com]index:%d name:%s\n", i, device_name);
                }
            }
        }
        NT_INT32 alsa_device_number = 0;
        if (pulse_device_number < 1)
        {
            if (NT_ERC_OK == push_api->GetAuidoInputDeviceNumber(1, &alsa_device_number))
            {
                fprintf(stdout, "Alsa device num:%d\n", alsa_device_number);
                char device_name[512];
                for (auto i = 0; i < alsa_device_number; ++i)
                {
                    if (NT_ERC_OK == push_api->GetAuidoInputDeviceName(1, i, device_name, 512))
                    {
                        fprintf(stdout, "[daniusdk.com]index:%d name:%s\n", i, device_name);
                    }
                }
            }
        }
        NT_INT32 capture_speaker_flag = 0;
        if (NT_ERC_OK == push_api->IsCanCaptureSpeaker(2, &capture_speaker_flag))
        {
            if (capture_speaker_flag)
                fprintf(stdout, "[daniusdk.com]Support speaker capture\n");
            else
                fprintf(stdout, "[daniusdk.com]UnSupport speaker capture\n");
        }
        NT_INT32 is_support_window_capture = 0;
        if (NT_ERC_OK == push_api->IsCaptureXWindowSupported(NULL, &is_support_window_capture))
        {
            if (is_support_window_capture)
                fprintf(stdout, "[daniusdk.com]Support window capture\n");
            else
                fprintf(stdout, "[daniusdk.com]UnSupport window capture\n");
        }
        if (is_support_window_capture)
        {
            NT_INT32 win_count = 0;
            if (NT_ERC_OK == push_api->UpdateCaptureXWindowList(NULL, &win_count) && win_count > 0)
            {
                fprintf(stdout, "X Capture Winows list++\n");
                for (auto i = 0; i < win_count; ++i)
                {
                    NT_UINT64 wid;
                    char title[512];
                    if (NT_ERC_OK == push_api->GetCaptureXWindowInfo(i, &wid, title, sizeof(title) / sizeof(char)))
                    {
                        x_win_list.push_back(wid);
                        fprintf(stdout, "wid:%llu, title:%s\n", wid, title);
                    }
                }
                fprintf(stdout, "[daniusdk.com]X Capture Winows list--\n");
            }
        }
        std::vector<CameraInfo> cameras;
        GetCameraInfo(push_api, cameras);
        if (!cameras.empty())
        {
            fprintf(stdout, "cameras count:%d\n", (int)cameras.size());
            for (const auto& c : cameras)
            {
                fprintf(stdout, "camera name:%s, id:%s, cap_num:%d\n", c.name_.c_str(), c.id_.c_str(), (int)c.capabilities_.size());
                for (const auto& i : c.capabilities_)
                {
                    fprintf(stdout, "[daniusdk.com]cap w:%d, h:%d, fps:%d\n", i.width_, i.height_, i.max_frame_rate_);
                }
            }
        }
        NT_UINT32 auido_option = NT_PB_E_AUDIO_OPTION_NO_AUDIO;
        if (pulse_device_number > 0 || alsa_device_number > 0)
        {
            auido_option = NT_PB_E_AUDIO_OPTION_CAPTURE_MIC;
        }
        else if (capture_speaker_flag)
        {
            auido_option = NT_PB_E_AUDIO_OPTION_CAPTURE_SPEAKER;
        }
        //auido_option = NT_PB_E_AUDIO_OPTION_CAPTURE_MIC_SPEAKER_MIXER;
        NT_UINT32 video_option = NT_PB_E_VIDEO_OPTION_SCREEN;
        if (!cameras.empty())
        {
            video_option = NT_PB_E_VIDEO_OPTION_CAMERA;
        }
        else if (is_support_window_capture)
        {
            video_option = NT_PB_E_VIDEO_OPTION_WINDOW;
        }
        // video_option = NT_PB_E_VIDEO_OPTION_LAYER;
        //video_option = NT_PB_E_VIDEO_OPTION_NO_VIDEO;
        NT_HANDLE push_handle = nullptr;
        //if (NT_ERC_OK != push_api->Open(&push_handle, NT_PB_E_VIDEO_OPTION_LAYER, NT_PB_E_AUDIO_OPTION_CAPTURE_SPEAKER, 0, NULL))
        if (NT_ERC_OK != push_api->Open(&push_handle, video_option, auido_option, 0, NULL))
        {
            return nullptr;
        }
        push_api->SetEventCallBack(push_handle, nullptr, OnSDKEventHandle);
        //push_api->SetXDisplayName(push_handle, ":0");
        //push_api->SetXDisplayName(push_handle, NULL);
        // 视频层配置方式
        if (NT_PB_E_VIDEO_OPTION_LAYER == video_option)
        {
            std::vector<std::shared_ptr<nt_pb_sdk::layer_conf_wrapper_base> > layer_confs;
            auto index = 0;
            //// 第0层填充RGBA矩形, 目的是保证帧率, 颜色就填充全黑
            auto rgba_layer_c0 = std::make_shared<nt_pb_sdk::RGBARectangleLayerConfigWrapper>(index++, true, 0, 0, 1280, 720);
            rgba_layer_c0->conf_.red_ = 200;
            rgba_layer_c0->conf_.green_ = 200;
            rgba_layer_c0->conf_.blue_ = 200;
            rgba_layer_c0->conf_.alpha_ = 255;
            layer_confs.push_back(rgba_layer_c0);
            ////// 第一层为桌面层
            //auto screen_layer_c1 = std::make_shared<nt_pb_sdk::ScreenLayerConfigWrapper>(index++, true, 0, 0, 1280, 720);
            //screen_layer_c1->conf_.scale_filter_mode_ = 3;
            //layer_confs.push_back(screen_layer_c1);
            //// 第一层为窗口
            if (!x_win_list.empty())
            {
                auto window_layer_c1 = std::make_shared<nt_pb_sdk::WindowLayerConfigWrapper>(index++, true, 0, 0, 640, 360);
                window_layer_c1->conf_.xwindow_ = x_win_list.back();
                layer_confs.push_back(window_layer_c1);
            }
            //// 摄像头层
            if (!cameras.empty())
            {
                auto camera_layer_c1 = std::make_shared<nt_pb_sdk::CameraLayerConfigWrapper>(index++, true,
                    640, 0, 640, 360);
                strcpy(camera_layer_c1->conf_.device_unique_id_, cameras.front().id_.c_str());
                camera_layer_c1->conf_.is_flip_horizontal_ = 0;
                camera_layer_c1->conf_.is_flip_vertical_ = 0;
                camera_layer_c1->conf_.rotate_degress_ = 0;
                layer_confs.push_back(camera_layer_c1);
                if (cameras.size() > 1)
                {
                    auto camera_layer_c2 = std::make_shared<nt_pb_sdk::CameraLayerConfigWrapper>(index++, true,
                        640, 0, 320, 240);
                    strcpy(camera_layer_c2->conf_.device_unique_id_, cameras.back().id_.c_str());
                    camera_layer_c2->conf_.is_flip_horizontal_ = 0;
                    camera_layer_c2->conf_.is_flip_vertical_ = 0;
                    camera_layer_c2->conf_.rotate_degress_ = 0;
                    layer_confs.push_back(camera_layer_c2);
                }
            }
            auto image_layer1 = std::make_shared<nt_pb_sdk::ImageLayerConfigWrapper>(index++, true, 650, 120, 324, 300);
            strcpy(image_layer1->conf_.file_name_utf8_, "./testpng/tca.png");
            layer_confs.push_back(image_layer1);
            auto image_layer2 = std::make_shared<nt_pb_sdk::ImageLayerConfigWrapper>(index++, true, 120, 380, 182, 138);
            strcpy(image_layer2->conf_.file_name_utf8_, "./testpng/t4.png");
            layer_confs.push_back(image_layer2);
            std::vector<const NT_PB_LayerBaseConfig* > layer_base_confs;
            for (const auto& i : layer_confs)
            {
                layer_base_confs.push_back(i->getBase());
            }
            if (NT_ERC_OK != push_api->SetLayersConfig(push_handle, 0, layer_base_confs.data(),
                layer_base_confs.size(), 0, nullptr))
            {
                push_api->Close(push_handle);
                push_handle = nullptr;
                return nullptr;
            }
        }
        // push_api->SetScreenClip(push_handle, 0, 0, 1280, 720);
        if (video_option == NT_PB_E_VIDEO_OPTION_CAMERA)
        {
            if (!cameras.empty())
            {
                push_api->SetVideoCaptureDeviceBaseParameter(push_handle, cameras.front().id_.c_str(),
                    640, 480);
                //push_api->FlipVerticalCamera(push_handle, 1);
                //push_api->FlipHorizontalCamera(push_handle, 1);
                //push_api->RotateCamera(push_handle, 0);
            }
        }
        if (video_option == NT_PB_E_VIDEO_OPTION_WINDOW)
        {
            if (!x_win_list.empty())
            {
                //push_api->SetCaptureXWindow(push_handle, x_win_list[0]);
                push_api->SetCaptureXWindow(push_handle, x_win_list.back());
            }
        }
        push_api->SetFrameRate(push_handle, dst_fps); // 帧率设置
        push_api->SetVideoEncoder(push_handle, 0, 1, NT_MEDIA_CODEC_ID_H264, 0);
        push_api->SetVideoBitRate(push_handle, 2000);  // 平均码率2000kbps
        push_api->SetVideoQuality(push_handle, 26);
        push_api->SetVideoMaxBitRate(push_handle, 4000); // 最大码率4000kbps
        // openh264 配置特定参数
        push_api->SetVideoEncoderSpecialInt32Option(push_handle, "usage_type", 0); //0是摄像头编码, 1是屏幕编码
        push_api->SetVideoEncoderSpecialInt32Option(push_handle, "rc_mode", 1); // 0是质量模式, 1是码率模式
        push_api->SetVideoEncoderSpecialInt32Option(push_handle, "enable_frame_skip", 0); // 0是关闭跳帧, 1是打开跳帧
        push_api->SetVideoKeyFrameInterval(push_handle, dst_fps * 2); // 关键帧间隔
        push_api->SetVideoEncoderProfile(push_handle, 3); // H264 high
        push_api->SetVideoEncoderSpeed(push_handle, 3); // 编码速度设置到3
        if (pulse_device_number > 0)
        {
            push_api->SetAudioInputLayer(push_handle, 2);
            push_api->SetAuidoInputDeviceId(push_handle, 0);
        }
        else if (alsa_device_number > 0)
        {
            push_api->SetAudioInputLayer(push_handle, 1);
            push_api->SetAuidoInputDeviceId(push_handle, 0);
        }
        push_api->SetEchoCancellation(push_handle, 1, 0);
        push_api->SetNoiseSuppression(push_handle, 1);
        push_api->SetAGC(push_handle, 1);
        push_api->SetVAD(push_handle, 1);
        push_api->SetInputAudioVolume(push_handle, 0, 1.0);
        push_api->SetInputAudioVolume(push_handle, 1, 0.2);
        // 音频配置
        push_api->SetPublisherAudioCodecType(push_handle, 1);
        //push_api->SetMute(push_handle, 1);
        return push_handle;
    }

image.gif

发布RTSP流实现如下:

bool start_rtsp_stream(NT_SmartPublisherSDKAPI* push_api, NT_HANDLE rtsp_server_handle, NT_HANDLE handle, const std::string stream_name) {
        push_api->SetRtspStreamName(handle, stream_name.c_str());
        push_api->ClearRtspStreamServer(handle);
        push_api->AddRtspStreamServer(handle, rtsp_server_handle, 0);
        
        if (NT_ERC_OK != push_api->StartRtspStream(handle, 0))
            return false;
        return true;
    }

image.gif

如果需要本地摄像头或者屏幕预览数据,调研预览接口即可:

// 开启预览,也可以不开启, 根据需求来
    push_api.SetPreviewXWindow(push_handle, "", sub_wid);
    push_api.StartPreview(push_handle, 0, nullptr);

image.gif

如需停止:

fprintf(stdout, "StopRtspStream++\n");
    push_api.StopRtspStream(push_handle);
    
    fprintf(stdout, "StopRtspStream--\n");
    fprintf(stdout, "stop_rtsp_server++\n");
    stop_rtsp_server(&push_api, rtsp_server_handle);
    fprintf(stdout, "stop_rtsp_server--\n");
    push_api.StopPreview(push_handle);
    // push_api.StopPublisher(push_handle);
    push_api.Close(push_handle);
    push_handle = nullptr;
    XDestroyWindow(display, sub_wid);
    XDestroyWindow(display, main_wid);
    XCloseDisplay(display);
    push_api.UnInit();

image.gif

总结

Linux平台arm64实现轻量级RTSP服务,目前实现的功能如下:

  • 音频编码:AAC;
  • 视频编码:H.264;
  • 协议:RTSP;
  • [音视频]支持纯音频/纯视频/音视频推送;
  • 支持X11屏幕采集;
  • 支持部分V4L2摄像头设备采集;
  • [屏幕/V4L2摄像头]支持帧率、关键帧间隔(GOP)、码率(bit-rate)设置;
  • [V4L2摄像头]支持V4L2摄像头设备选择(设备文件名范围:[/dev/video0, /dev/video63])、分辨率设置、帧率设置;
  • [V4L2摄像头]支持水平反转、垂直反转、0° 90° 180° 270°旋转;
  • [音频]支持基于alsa-lib接口的音频采集;
  • [音频]支持基于libpulse接口采集本机PulseAudio服务音频;
  • [预览]支持实时预览; 支持RTSP端口设置;
  • 支持RTSP鉴权用户名、密码设置;
  • 支持获取当前RTSP服务会话连接数;
  • 支持x64_64架构、aarch64架构(需要glibc-2.21及以上版本的Linux系统, 需要libX11.so.6, 需要GLib–2.0, 需安装 libstdc++.so.6.0.21、GLIBCXX_3.4.21、 CXXABI_1.3.9)。

配合我们的RTSP播放器,可轻松实现150-400ms低延迟体验,感兴趣的开发者,可以单独跟我沟通。

相关文章
|
1月前
|
Linux 网络安全 Docker
盘古栈云,创建带ssh服务的linux容器
创建带ssh服务的linux容器
263 146
|
6月前
|
安全 Linux 数据安全/隐私保护
Red Hat Enterprise Linux 9.6 (x86_64, aarch64) - 红帽企业 Linux (RHEL)
Red Hat Enterprise Linux 9.6 (x86_64, aarch64) - 红帽企业 Linux (RHEL)
772 36
Red Hat Enterprise Linux 9.6 (x86_64, aarch64) - 红帽企业 Linux (RHEL)
|
2月前
|
缓存 安全 Linux
六、Linux核心服务与包管理
在没有网络的情况下,使用系统安装光盘是获取RPM包的常用方法。场景二:配置本地文件镜像源 (使用系统安装光盘/ISO)(检查RPM包的GPG签名以保证安全) 或。YUM/DNF包管理工具 (yum/dnf)(此处可以放置您为本主题制作的思维导图)处理依赖问题的危险选项 (应极力避免)(覆盖文件、替换已安装包)。(list) 则是列出文件。(query file) 是。(假设系统安装光盘已挂载到。信息 (verbose)。(upgrade) 选项。(all) 已安装的包。(package) 选项
259 11
|
6月前
|
Linux 虚拟化 iOS开发
Rocky Linux 10 aarch64 OVF (sysin) - Apple silicon VMware 虚拟机模板
Rocky Linux 10 aarch64 OVF (sysin) - Apple silicon VMware 虚拟机模板
218 34
Rocky Linux 10 aarch64 OVF (sysin) - Apple silicon VMware 虚拟机模板
|
4月前
|
Ubuntu 编译器 C语言
在Ubuntu22.04平台上交叉编译针对Rv1126架构的GCC13.2.0编译器的步骤。
遵循上述步骤,您应该能够在Ubuntu 22.04平台上成功交叉编译适用于RISC-V架构RV1126的GCC 13.2.0编译器,允许您为目标硬件构建应用程序和操作系统组件。
206 10
|
6月前
|
监控 Linux 应用服务中间件
Linux多节点多硬盘部署MinIO:分布式MinIO集群部署指南搭建高可用架构实践
通过以上步骤,已成功基于已有的 MinIO 服务,扩展为一个 MinIO 集群。该集群具有高可用性和容错性,适合生产环境使用。如果有任何问题,请检查日志或参考MinIO 官方文档。作者联系方式vx:2743642415。
1952 57
|
4月前
|
运维 监控 Java
初创代购选单体,千万级平台用微服务:一张表看懂架构选型红线
在跨境电商代购系统年交易额超3.2万亿元的背景下,本文对比微服务与单体架构的技术原理、适用场景及实战案例,结合性能、运维、成本等维度,为企业提供架构选型指南,助力实现高效扩展与稳定运营。
|
7月前
|
Linux Shell
在Linux、CentOS7中设置shell脚本开机自启动服务
以上就是在CentOS 7中设置shell脚本开机自启动服务的全部步骤。希望这个指南能帮助你更好地管理你的Linux系统。
566 25
|
存储 网络协议 Linux

热门文章

最新文章