ffmpeg-api记录

ffmpeg-api记录

1. Frame

基于引用计数的frame

• av_frame_ref: ref
• av_frame_unref: unref
• av_frame_move_ref:所有权转移
• av_frame_clone: copy一份，引用计数=1

2. Packet

3. AVAudioFifo

音频重采样时，用来缓存aac的帧(1024才能送去编码)

• av_audio_fifo_alloc
• av_audio_fifo_size: 大小查询
• av_audio_fifo_read: 消费fifo
• av_audio_fifo_write: 写入fifo

4. mux

目标: 根据配置, 给定图片、音频，转码(如果需要)并mux

graph LR
    init_muxer_using_config[根据配置初始化mux]
    init_muxer_using_origin_context[根据原视频初始化mux]
    interact_input_image_audio_saple[交替输入图像/音频] --> convert_if_need[如果需要则进行转码] --> mux
    convert_if_need --> video_convert[视频转码逻辑]
    video_convert --> init_sws_getContext_using_format_width_height[使用src/dst像素格式、宽、高初始化]
    init_sws_getContext_using_format_width_height --> call_sws_scale[调用sws_scale接口得到转码过后的frame]
    call_sws_scale --> call_send_frame_and_mux[正常调用send_frame接口进行mux]

    convert_if_need --> audio_convert[音频转码设置]
    audio_convert --> init_swr_using_swr_alloc_set_opts2[使用src/dst布局、编码格式、采样率初始化 swr_alloc_set_opts2 转换器]
    audio_convert --> init_audio_fifo[使用av_audio_fifo_alloc+dst编码格式、通道布局初始化音频fifo队列]
    init_swr_using_swr_alloc_set_opts2 --> call_swr_convert[调用swr_convert得到转码过后的frame]
    call_swr_convert --> push_fifo[将结果push到audio fifo中]
    push_fifo --> consume_fifo[消费fifo如果samples大于1024]
    consume_fifo --> call_send_frame_and_mux

4.1 mux.Stream

enum class stream_type {none, video, audio};
struct Stream {
	stream_type type;
	int index{-1};
	AVCodecContext *codec_ctx{nullptr};
	// for video
	SwsContext *vid_sws_ctx{nullptr};
	int64_t vid_next_pts{0};
	// for audio
	SwrContext *aud_swr_ctx{nullptr};
	int64_t handled_samples{0};
	AVAudioFifo *fifo{nullptr};

	// common frame
	AVFrame *frame{nullptr};
};