作者：叶余
来源：https://www.cnblogs.com/leisure_chn/p/10318145.html
所谓内存IO，在FFmpeg中叫作“buffered IO”或“custom IO”，指的是将一块内存缓冲区用作FFmpeg的输入或输出。与内存IO操作对应的是指定URL作为FFmpeg的输入或输出，比如URL可能是普通文件或网络流地址等。这两种输入输出模式我们暂且称作“内存IO模式”和“URL-IO模式”。
本文源码基于FFmpeg 4.1版本，为帮助理解，可参考FFmpeg工程examples中如下两份代码：
https://github.com/FFmpeg/FFmpeg/blob/n4.1/doc/examples/avio_reading.c
https://github.com/FFmpeg/FFmpeg/blob/n4.1/doc/examples/remuxing.c
1. 内存区作输入
1.1 用法
用法如示例中注释的步骤，如下：
// @opaque : 是由用户提供的参数，指向用户数据 // @buf : 作为FFmpeg的输入，此处由用户准备好buf中的数据 // @buf_size: buf的大小 // @return : 本次IO数据量 static int read_packet(void *opaque, uint8_t *buf, int buf_size) { int fd = *((int *)opaque); int ret = read(fd, buf, buf_size); return ret; } int main() { AVFormatContext *ifmt_ctx = NULL; AVIOContext *avio_in = NULL; uint8_t *ibuf = NULL; size_t ibuf_size = 4096; int fd = -1; // 打开一个FIFO文件的读端 fd = open_fifo_for_read("/tmp/test_fifo"); // 1. 分配缓冲区 ibuf = av_malloc(ibuf_size); // 2. 分配AVIOContext，第三个参数write_flag为0 avio_in = avio_alloc_context(ibuf, ibuf_size, 0, &fd, &read_packet, NULL, NULL); // 3. 分配AVFormatContext，并指定AVFormatContext.pb字段。必须在调用avformat_open_input()之前完成 ifmt_ctx = avformat_alloc_context(); ifmt_ctx->pb = avio_in; // 4. 打开输入(读取封装格式文件头) avformat_open_input(&ifmt_ctx, NULL, NULL, NULL); ...... }
当启用内存IO模式后(即ifmt_ctx->pb有效时)，将会忽略avformat_open_input()第二个参数url的值。在上述示例中，打开了FIFO的读端，并在回调函数中将FIFO中的数据填入内存缓冲区ibuf，内存缓冲区ibuf将作为FFmpeg的输入。在上述示例中，因为打开的是一个命名管道FIFO，FIFO的数据虽然在内存中，但FIFO有名字("/tmp/test_fifo")，所以此例也可以使用URL-IO模式，如下：
AVFormatContext *ifmt_ctx = NULL; avformat_open_input(&ifmt_ctx, "/tmp/test_fifo", NULL, NULL);
而对于其他一些场合，当有效音视频数据位于内存，而这片内存并无一个URL属性可用时，则只能使用内存IO模式来取得输入数据。
1.2 回调时机
回调函数何时被回调呢？所有需要从输入源中读取数据的时刻，都将调用回调函数。和输入源是普通文件相比，只不过输入源变成了内存区，其他各种外在表现并无不同。
如下各函数在不同的阶段从输入源读数据，都会调用回调函数：
avformat_open_input() 从输入源读取封装格式文件头
avformat_find_stream_info() 从输入源读取一段数据，尝试解码，以获取流信息
av_read_frame() 从输入源读取数据包
2. 内存区作输出
2.1 用法
用法如示例中注释的步骤，如下：
// @opaque : 是由用户提供的参数，指向用户数据 // @buf : 作为FFmpeg的输出，此处FFmpeg已准备好buf中的数据 // @buf_size: buf的大小 // @return : 本次IO数据量 static int write_packet(void *opaque, uint8_t *buf, int buf_size) { int fd = *((int *)opaque); int ret = write(fd, buf, buf_size); return ret; } int main() { AVFormatContext *ofmt_ctx = NULL; AVIOContext *avio_out = NULL; uint8_t *obuf = NULL; size_t obuf_size = 4096; int fd = -1; // 打开一个FIFO文件的写端 fd = open_fifo_for_write("/tmp/test_fifo"); // 1. 分配缓冲区 obuf = av_malloc(obuf_size); // 2. 分配AVIOContext，第三个参数write_flag为1 AVIOContext *avio_out = avio_alloc_context(obuf, obuf_size, 1, &fd, NULL, write_packet, NULL); // 3. 分配AVFormatContext，并指定AVFormatContext.pb字段。必须在调用avformat_write_header()之前完成 avformat_alloc_output_context2(&ofmt_ctx, NULL, NULL, NULL); ofmt_ctx->pb=avio_out; // 4. 将文件头写入输出文件 avformat_write_header(ofmt_ctx, NULL); ...... }
当启用内存IO模式后(即ofmt_ctx->pb有效时)，FFmpeg会将输出写入内存缓冲区obuf，用户可在回调函数中将obuf中的数据取走。在上述示例中，因为打开的是一个命名管道FIFO，FIFO的数据虽然在内存中，但FIFO有名字("/tmp/test_fifo")，所以此例也可以使用URL-IO模式，如下：
AVFormatContext *ofmt_ctx = NULL; avformat_alloc_output_context2(&ofmt_ctx, "/tmp/test_fifo", NULL, NULL);
而对于其他一些场合，需将数据输出到内存，而这片内存并无一个URL属性可用时，则只能使用内存IO模式。
2.2 回调时机
回调函数何时被回调呢？所有输出数据的时刻，都将调用回调函数。和输出是普通文件相比，只不过输出变成了内存区，其他各种外在表现并无不同。
如下各函数在不同的阶段会输出数据，都会调用回调函数：
avformat_write_header() 将流头部信息写入输出区
av_interleaved_write_frame() 将数据包写入输出区
av_write_trailer() 将流尾部信息写入输出区
3. 实现机制
如下是与内存IO操作相关的一些关键数据结构及函数，我们从API接口层面来看一下内存IO的实现机制，而不深入分析内部源码。FFmpeg的API注释非常详细，从注释中能得到很多有用信息。
3.1 struct AVIOContext
/** * Bytestream IO Context. * New fields can be added to the end with minor version bumps. * Removal, reordering and changes to existing fields require a major * version bump. * sizeof(AVIOContext) must not be used outside libav*. * * @note None of the function pointers in AVIOContext should be called * directly, they should only be set by the client application * when implementing custom I/O. Normally these are set to the * function pointers specified in avio_alloc_context() */ typedef struct AVIOContext { ...... /* * The following shows the relationship between buffer, buf_ptr, * buf_ptr_max, buf_end, buf_size, and pos, when reading and when writing * (since AVIOContext is used for both): * ********************************************************************************** * READING ********************************************************************************** * * | buffer_size | * |---------------------------------------| * | | * * buffer buf_ptr buf_end * +---------------+-----------------------+ * |/ / / / / / / /|/ / / / / / /| | * read buffer: |/ / consumed / | to be read /| | * |/ / / / / / / /|/ / / / / / /| | * +---------------+-----------------------+ * * pos * +-------------------------------------------+-----------------+ * input file: | | | * +-------------------------------------------+-----------------+ * * ********************************************************************************** * WRITING ********************************************************************************** * * | buffer_size | * |--------------------------------------| * | | * * buf_ptr_max * buffer (buf_ptr) buf_end * +-----------------------+--------------+ * |/ / / / / / / / / / / /| | * write buffer: | / / to be flushed / / | | * |/ / / / / / / / / / / /| | * +-----------------------+--------------+ * buf_ptr can be in this * due to a backward seek * * pos * +-------------+----------------------------------------------+ * output file: | | | * +-------------+----------------------------------------------+ * */ unsigned char *buffer; /**< Start of the buffer. */ int buffer_size; /**< Maximum buffer size */ unsigned char *buf_ptr; /**< Current position in the buffer */ unsigned char *buf_end; /**< End of the data, may be less than buffer+buffer_size if the read function returned less data than requested, e.g. for streams where no more data has been received yet. */ void *opaque; /**< A private pointer, passed to the read/write/seek/... functions. */ int (*read_packet)(void *opaque, uint8_t *buf, int buf_size); int (*write_packet)(void *opaque, uint8_t *buf, int buf_size); ...... } AVIOContext;
注意：此数据结构中的成员不应由用户程序直接访问。当使用内存IO模式时，用户应调用avio_alloc_context()对此结构的read_packet和write_packet函数指针进行赋值。
3.2 AVIOContext* AVFormatContext.pb
/** * Format I/O context. * ...... */ typedef struct AVFormatContext { ...... /** * I/O context. * * - demuxing: either set by the user before avformat_open_input() (then * the user must close it manually) or set by avformat_open_input(). * - muxing: set by the user before avformat_write_header(). The caller must * take care of closing / freeing the IO context. * * Do NOT set this field if AVFMT_NOFILE flag is set in * iformat/oformat.flags. In such a case, the (de)muxer will handle * I/O in some other way and this field will be NULL. */ AVIOContext *pb; ...... }
struct AVFormatContext结构中与内存IO操作相关的重要成员是AVIOContext *pb，有如下规则：
- 解复用过程：在调用
avformat_open_input()前由用户手工设置，因为从avformat_open_input()开始有读输入的操作。 - 复用过程：在调用
avformat_write_header()前由用户手工设置，因为从avformat_write_header()开始有写输出的操作。
3.3 输入时：avformat_open_input()
/** * Open an input stream and read the header. The codecs are not opened. * The stream must be closed with avformat_close_input(). * * @param ps Pointer to user-supplied AVFormatContext (allocated by avformat_alloc_context). * May be a pointer to NULL, in which case an AVFormatContext is allocated by this * function and written into ps. * Note that a user-supplied AVFormatContext will be freed on failure. * @param url URL of the stream to open. * @param fmt If non-NULL, this parameter forces a specific input format. * Otherwise the format is autodetected. * @param options A dictionary filled with AVFormatContext and demuxer-private options. * On return this parameter will be destroyed and replaced with a dict containing * options that were not found. May be NULL. * * @return 0 on success, a negative AVERROR on failure. * * @note If you want to use custom IO, preallocate the format context and set its pb field. */ int avformat_open_input(AVFormatContext **ps, const char *url, AVInputFormat *fmt, AVDictionary **options);
打开输入流读取头部信息。如果使用内存IO模式，应在此之前分配AVFormatContext并设置其pb成员。
3.4 输出时：avformat_write_header()
/** * Allocate the stream private data and write the stream header to * an output media file. * * @param s Media file handle, must be allocated with avformat_alloc_context(). * Its oformat field must be set to the desired output format; * Its pb field must be set to an already opened AVIOContext. * @param options An AVDictionary filled with AVFormatContext and muxer-private options. * On return this parameter will be destroyed and replaced with a dict containing * options that were not found. May be NULL. * * @return AVSTREAM_INIT_IN_WRITE_HEADER on success if the codec had not already been fully initialized in avformat_init, * AVSTREAM_INIT_IN_INIT_OUTPUT on success if the codec had already been fully initialized in avformat_init, * negative AVERROR on failure. * * @see av_opt_find, av_dict_set, avio_open, av_oformat_next, avformat_init_output. */ av_warn_unused_result int avformat_write_header(AVFormatContext *s, AVDictionary **options);
将流头部信息写入输出文件。在调用此函数前，AVFormatContext.pb成员必须设置为一个已经打开的AVIOContext。AVFormatContext.pb赋值方式分为两种情况：
[1]. URL-IO模式：调用avio_open()或avio_open2()，形如
avio_open(&ofmt_ctx->pb, out_filename, AVIO_FLAG_WRITE);
[2]. 内存IO模式：调用avio_alloc_context()分配AVIOContext，然后为pb赋值，形如：
avformat_alloc_output_context2(&ofmt_ctx, NULL, NULL, NULL); ofmt_ctx->pb=avio_out;
3.5 内存IO模式：avio_alloc_context()
/** * Allocate and initialize an AVIOContext for buffered I/O. It must be later * freed with avio_context_free(). * * @param buffer Memory block for input/output operations via AVIOContext. * The buffer must be allocated with av_malloc() and friends. * It may be freed and replaced with a new buffer by libavformat. * AVIOContext.buffer holds the buffer currently in use, * which must be later freed with av_free(). * @param buffer_size The buffer size is very important for performance. * For protocols with fixed blocksize it should be set to this blocksize. * For others a typical size is a cache page, e.g. 4kb. * @param write_flag Set to 1 if the buffer should be writable, 0 otherwise. * @param opaque An opaque pointer to user-specific data. * @param read_packet A function for refilling the buffer, may be NULL. * For stream protocols, must never return 0 but rather * a proper AVERROR code. * @param write_packet A function for writing the buffer contents, may be NULL. * The function may not change the input buffers content. * @param seek A function for seeking to specified byte position, may be NULL. * * @return Allocated AVIOContext or NULL on failure. */ AVIOContext *avio_alloc_context( unsigned char *buffer, int buffer_size, int write_flag, void *opaque, int (*read_packet)(void *opaque, uint8_t *buf, int buf_size), int (*write_packet)(void *opaque, uint8_t *buf, int buf_size), int64_t (*seek)(void *opaque, int64_t offset, int whence));
- opaque是
read_packet/write_packet的第一个参数，指向用户数据。 - buffer和buffer_size是
read_packet/write_packet的第二个和第三个参数，是供FFmpeg使用的数据区。buffer用作FFmpeg输入时，由用户负责向buffer中填充数据，FFmpeg取走数据。buffer用作FFmpeg输出时，由FFmpeg负责向buffer中填充数据，用户取走数据。 - write_flag是缓冲区读写标志，读写的主语是指FFmpeg。
write_flag为1时，buffer用于写，即作为FFmpeg输出。write_flag为0时，buffer用于读，即作为FFmpeg输入。 - read_packet和write_packet是函数指针，指向用户编写的回调函数。
3.6 URL-IO模式：avio_open()
/** * Create and initialize a AVIOContext for accessing the * resource indicated by url. * @note When the resource indicated by url has been opened in * read+write mode, the AVIOContext can be used only for writing. * * @param s Used to return the pointer to the created AVIOContext. * In case of failure the pointed to value is set to NULL. * @param url resource to access * @param flags flags which control how the resource indicated by url * is to be opened * @return >= 0 in case of success, a negative value corresponding to an * AVERROR code in case of failure */ int avio_open(AVIOContext **s, const char *url, int flags);
4. 修改记录
2019-01-24 V1.0 初稿
「视频云技术」你最值得关注的音视频技术公众号，每周推送来自阿里云一线的实践技术文章，在这里与音视频领域一流工程师交流切磋。
