Media I/O API Reference (libobs/media-io)

#include <obs.h>

Video Handler

type video_t

Video output handler object

enum video_format

Video format. Can be one of the following values:

  • VIDEO_FORMAT_I420

  • VIDEO_FORMAT_NV12

  • VIDEO_FORMAT_YVYU

  • VIDEO_FORMAT_YUY2

  • VIDEO_FORMAT_UYVY

  • VIDEO_FORMAT_RGBA

  • VIDEO_FORMAT_BGRA

  • VIDEO_FORMAT_BGRX

  • VIDEO_FORMAT_Y800

  • VIDEO_FORMAT_I444

  • VIDEO_FORMAT_BGR3

  • VIDEO_FORMAT_I422

  • VIDEO_FORMAT_I40A

  • VIDEO_FORMAT_I42A

  • VIDEO_FORMAT_YUVA

  • VIDEO_FORMAT_AYUV

  • VIDEO_FORMAT_I010

  • VIDEO_FORMAT_P010

  • VIDEO_FORMAT_I210

  • VIDEO_FORMAT_I412

  • VIDEO_FORMAT_YA2L

  • VIDEO_FORMAT_P216

  • VIDEO_FORMAT_P416

  • VIDEO_FORMAT_V210

  • VIDEO_FORMAT_R10L

enum video_trc

Transfer characteristics. Can be one of the following values:

  • VIDEO_TRC_DEFAULT - sRGB TRC for SDR, PQ TRC for HDR

  • VIDEO_TRC_SRGB - sRGB TRC

  • VIDEO_TRC_PQ - PQ

  • VIDEO_TRC_HLG - HLG

enum video_colorspace

YUV color space. Can be one of the following values:

  • VIDEO_CS_DEFAULT - Equivalent to VIDEO_CS_709

  • VIDEO_CS_601 - Rec. 601 color space

  • VIDEO_CS_709 - Rec. 709 color space

  • VIDEO_CS_SRGB - sRGB color space

  • VIDEO_CS_2100_PQ - Rec. 2100 color space, PQ transfer

  • VIDEO_CS_2100_HLG - Rec. 2100 color space, HLG transfer

enum video_range_type

YUV color range.

  • VIDEO_RANGE_DEFAULT - Equivalent to VIDEO_RANGE_PARTIAL

  • VIDEO_RANGE_PARTIAL - Partial range

  • VIDEO_RANGE_FULL - Full range

struct video_data

Video frame structure.

uint8_t *video_data.data[MAX_AV_PLANES]
uint32_t video_data.linesize[MAX_AV_PLANES]
uint64_t video_data.timestamp
struct video_output_info

Video output handler information

const char *video_output_info.name
enum video_format video_output_info.format
uint32_t video_output_info.fps_num
uint32_t video_output_info.fps_den
uint32_t video_output_info.width
uint32_t video_output_info.height
size_t video_output_info.cache_size
enum video_colorspace video_output_info.colorspace
enum video_range_type video_output_info.range
enum video_format video_format_from_fourcc(uint32_t fourcc)

Converts a fourcc value to a video format.

Parameters:

  • forcecc – Fourcc value

Returns:

Video format

bool video_format_get_parameters(enum video_colorspace color_space, enum video_range_type range, float matrix[16], float min_range[3], float max_range[3])

Converts a color space/range to matrix/min/max values.

Parameters:

  • color_space – Color space to convert

  • range – Color range to convert

  • matrix – Pointer to the matrix

  • min_range – Pointer to get the minimum range value

  • max_range – Pointer to get the maximum range value

bool video_format_get_parameters_for_format(enum video_colorspace color_space, enum video_range_type range, enum video_format format, float matrix[16], float min_range[3], float max_range[3])

Converts a color space/range to matrix/min/max values for a given video format.

Parameters:

  • color_space – Color space to convert

  • range – Color range to convert

  • format – Video format

  • matrix – Pointer to the matrix

  • min_range – Pointer to get the minimum range value

  • max_range – Pointer to get the maximum range value

bool video_output_connect(video_t *video, const struct video_scale_info *conversion, void (*callback)(void *param, struct video_data *frame), void *param)

Connects a raw video callback to the video output handler.

Parameters:

  • video – Video output handler object

  • callback – Callback to receive video data

  • param – Private data to pass to the callback

void video_output_disconnect(video_t *video, void (*callback)(void *param, struct video_data *frame), void *param)

Disconnects a raw video callback from the video output handler.

Parameters:

  • video – Video output handler object

  • callback – Callback

  • param – Private data

const struct video_output_info *video_output_get_info(const video_t *video)

Gets the full video information of the video output handler.

Parameters:

  • video – Video output handler object

Returns:

Video output info structure pointer

uint64_t video_output_get_frame_time(const video_t *video)

Gets the frame interval of the video output handler.

Parameters:

  • video – Video output handler object

Returns:

Video frame interval in nanoseconds

enum video_format video_output_get_format(const video_t *video)

Gets the video format of the video output handler.

Parameters:

  • video – Video output handler object

Returns:

Video format

uint32_t video_output_get_width(const video_t *video)
uint32_t video_output_get_height(const video_t *video)

Gets the width/height of the video output handler.

Parameters:

  • video – Video output handler object

Returns:

Width/height

double video_output_get_frame_rate(const video_t *video)

Gets the frame rate (as a floating point) of the video output handler.

Parameters:

  • video – Video output handler object

Returns:

Frame rate

uint32_t video_output_get_skipped_frames(const video_t *video)

Gets the skipped frame count of the video output handler.

Parameters:

  • video – Video output handler object

Returns:

Skipped frame count

uint32_t video_output_get_total_frames(const video_t *video)

Gets the total frames processed of the video output handler.

Parameters:

  • video – Video output handler object

Returns:

Total frames processed

Audio Handler

type audio_t
enum audio_format

Audio format. Can be one of the following values:

  • AUDIO_FORMAT_UNKNOWN

  • AUDIO_FORMAT_U8BIT

  • AUDIO_FORMAT_16BIT

  • AUDIO_FORMAT_32BIT

  • AUDIO_FORMAT_FLOAT

  • AUDIO_FORMAT_U8BIT_PLANAR

  • AUDIO_FORMAT_16BIT_PLANAR

  • AUDIO_FORMAT_32BIT_PLANAR

  • AUDIO_FORMAT_FLOAT_PLANAR

enum speaker_layout

Speaker layout. Can be one of the following values:

  • SPEAKERS_UNKNOWN

  • SPEAKERS_MONO

  • SPEAKERS_STEREO

  • SPEAKERS_2POINT1

  • SPEAKERS_4POINT0

  • SPEAKERS_4POINT1

  • SPEAKERS_5POINT1

  • SPEAKERS_7POINT1

struct audio_data

Audio data structure.

uint8_t *audio_data.data[MAX_AV_PLANES]
uint32_t audio_data.frames
uint64_t audio_data.timestamp
struct audio_output_data
float *audio_output_data.data[MAX_AUDIO_CHANNELS]
struct audio_output_info
const char *audio_output_info.name
uint32_t audio_output_info.samples_per_sec
enum audio_format audio_output_info.format
enum speaker_layout audio_output_info.speakers
audio_input_callback_t audio_output_info.input_callback
void *audio_output_info.input_param
struct audio_convert_info
uint32_t audio_convert_info.samples_per_sec
enum audio_format audio_convert_info.format
enum speaker_layout audio_convert_info.speakers
typedef bool (*audio_input_callback_t)(void *param, uint64_t start_ts, uint64_t end_ts, uint64_t *new_ts, uint32_t active_mixers, struct audio_output_data *mixes)

Audio input callback (typically used internally).

uint32_t get_audio_channels(enum speaker_layout speakers)

Converts a speaker layout to its audio channel count.

Parameters:

  • speakers – Speaker layout enumeration

Returns:

Channel count

size_t get_audio_bytes_per_channel(enum audio_format format)

Gets the audio bytes per channel for a specific audio format.

Parameters:

  • format – Audio format

Returns:

Bytes per channel

bool is_audio_planar(enum audio_format format)

Returns whether the audio format is a planar format.

Parameters:

  • format – Audio format

Returns:

true if audio is planar, false otherwise

size_t get_audio_planes(enum audio_format format, enum speaker_layout speakers)

Gets the number of audio planes for a specific audio format and speaker layout.

Parameters:

  • format – Audio format

  • speakers – Speaker layout

Returns:

Number of audio planes

size_t get_audio_size(enum audio_format format, enum speaker_layout speakers, uint32_t frames)

Gets the audio block size for a specific frame out with the given format and speaker layout.

Parameters:

  • format – Audio format

  • speakers – Speaker layout

  • frames – Audio frame count

Returns:

Audio block size

uint64_t audio_frames_to_ns(size_t sample_rate, uint64_t frames)

Helper function to convert a specific number of audio frames to nanoseconds based upon its sample rate.

Parameters:

  • sample_rate – Sample rate

  • frames – Frame count

Returns:

Nanoseconds

uint64_t ns_to_audio_frames(size_t sample_rate, uint64_t ns)

Helper function to convert a specific number of nanoseconds to audio frame count based upon its sample rate.

Parameters:

  • sample_rate – Sample rate

  • ns – Nanoseconds

Returns:

Frame count

typedef void (*audio_output_callback_t)(void *param, size_t mix_idx, struct audio_data *data)

Audio output callback. Typically used internally.

bool audio_output_connect(audio_t *audio, size_t mix_idx, const struct audio_convert_info *conversion, audio_output_callback_t callback, void *param)

Connects a raw audio callback to the audio output handler. Optionally allows audio conversion if necessary.

Parameters:

  • audio – Audio output handler object

  • mix_idx – Mix index to get raw audio from

  • conversion – Audio conversion information, or NULL for no conversion

  • callback – Raw audio callback

  • param – Private data to pass to the callback

void audio_output_disconnect(audio_t *audio, size_t mix_idx, audio_output_callback_t callback, void *param)

Disconnects a raw audio callback from the audio output handler.

Parameters:

  • audio – Audio output handler object

  • mix_idx – Mix index to get raw audio from

  • callback – Raw audio callback

  • param – Private data to pass to the callback

size_t audio_output_get_block_size(const audio_t *audio)

Gets the audio block size of an audio output handler.

Parameters:

  • audio – Audio output handler object

Returns:

Audio block size

size_t audio_output_get_planes(const audio_t *audio)

Gets the plane count of an audio output handler.

Parameters:

  • audio – Audio output handler object

Returns:

Audio plane count

size_t audio_output_get_channels(const audio_t *audio)

Gets the channel count of an audio output handler.

Parameters:

  • audio – Audio output handler object

Returns:

Audio channel count

uint32_t audio_output_get_sample_rate(const audio_t *audio)

Gets the sample rate of an audio output handler.

Parameters:

  • audio – Audio output handler object

Returns:

Audio sample rate

const struct audio_output_info *audio_output_get_info(const audio_t *audio)

Gets all audio information for an audio output handler.

Parameters:

  • audio – Audio output handler object

Returns:

Pointer to audio output information structure

Resampler

FFmpeg wrapper to resample audio.

typedef struct audio_resampler audio_resampler_t
struct resample_info
uint32_t resample_info.samples_per_sec
enum audio_format resample_info.format
enum speaker_layout resample_info.speakers
audio_resampler_t *audio_resampler_create(const struct resample_info *dst, const struct resample_info *src)

Creates an audio resampler.

Parameters:

  • dst – Destination audio information

  • src – Source audio information

Returns:

Audio resampler object

void audio_resampler_destroy(audio_resampler_t *resampler)

Destroys an audio resampler.

Parameters:

  • resampler – Audio resampler object

bool audio_resampler_resample(audio_resampler_t *resampler, uint8_t *output[], uint32_t *out_frames, uint64_t *ts_offset, const uint8_t *const input[], uint32_t in_frames)

Resamples audio frames.

Parameters:

  • resampler – Audio resampler object

  • output – Pointer to receive converted audio frames

  • out_frames – Pointer to receive converted audio frame count

  • ts_offset – Pointer to receive timestamp offset (in nanoseconds)

  • input – Input frames to convert

  • in_frames – Input frame count