FreeFrame Open Video Plugin System Specification

Copyright (c) 2002-2007 www.freeframe.org
All rights reserved.

This document is a functional specification for the FreeFrame application programmers interface (API). It is intended to be subject to the same license as the code supplied as examples of use of the FreeFrame API.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
• Neither the name of FreeFrame nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

FreeFrame plugins are distributed and used as compiled shared objects (.so) in Linux, as Bundles in Mac OS X and as Dynamic Link Libraries (.dll) in Windows.

FreeFrame Plugins export a single function: plugMain. This is passed 3 values: a 32-bit function code, a 32-bit input value and a 32-bit instance identifier. It returns a 32-bit output value.
The input values and output values have different types according to the function code. This may be implemented differently on different platforms in different languages, but the format of the values must be consistent.

Parameters:

Return value:

Remarks:

All numbers in this document are ordinary decimal numbers, expect if specified otherwise.

The following types are used:

Type	Range of type	Notes
32-bit unsigned integer	0 to 4294967295	-
32-bit pointer	0 to 4294967295	needs to be a valid memory location
32-bit IEEE float	1 sign bit, 8 bit exponent, 23 bit fraction	Smallest Non-Zero Values: + or - 1.4012985 × (10 ^ ?45)	Largest Non-Infinite Values: + or - 3.4028235 × (10 ^ 38)
64-bit IEEE float	1 sign bit, 11 bit exponent, 52 bit fraction	Smallest Non-Zero Values: + or - 5.0 × (10 ^ ?324)	Largest Non-Infinite Values: + or - 1.7976931348623157 × (10 ^ 308)

 

The following constants are used as return values:

Remarks:

• In functions that return error codes any value other than 0 is assumed to be an error code. These are to be defined in each function below.
• In functions that return a pointer, (hex) FFFFFFFF also represents a failure condition.

Definition:

Definition:

Definition:

Definition:

Definition:

Remarks:

Apart from text parameters, FreeFrame parameter values are always 32-bit floats, and the range of values permitted is STRICTLY 0-1 (0 <= ParameterValue <= 1). This allows faster processing and a good range of values over a standard range, so the host can run up sliders or similar for the plugin. The Use of any values outside this range will result in hideous incompatibilities. The ParameterDisplayValue can be used to display whatever actual values the plugin likes e.g. 0-255, 0-767, 1-256 or whatever. The plugin should translate the standard 0-1 float range into the values it needs for its processing.
For text parameters, this 32-bit value represents a pointer to a null terminated string.

Definition:

Remarks:

Plugins should always specify default values. Sometimes a host may not implement all parameters on a plugin, so the plugin must use default values until told to do otherwise by the host. Apart from text parameters, FreeFrame parameter values are always 32-bit floats, and the range of values permitted is STRICTLY 0-1.
For text parameters, this 32-bit value represents a pointer to a null terminated string.

Definition:

Definition:

Remarks:

See getPluginCaps for more information about the usage of this structure.

See VideoInfoStruct for more information about the pixelformats.

Definition:

Remarks:

The range of values permitted for all variables (except text) is strictly 0-1.

For the xpos and ypos parameter types, GPU plugin coordinates (0,0) refer to the bottom left corner of the processed frame, and coordinates (1,1) refer to the upper right corner. CPU plugin coordinates (0,0) may be at the bottom left or top left, and (1,1) at the top right or bottom right, depending on the orientation defined in the VideoInfoStruct.

Definition:

Definition:

PluginInfoStruct {
  APIMajorVersion: 32-bit unsigned integer
  APIMinorVersion: 32-bit unsigned integer
  PluginUniqueID: 4 1-byte ASCII characters *not null terminated*
  PluginName: 16 1-byte ASCII characters *not null terminated*
  PluginType: 32-bit unsigned integer
}

Fields:

Remarks:

Plugins of PluginType effect process one or more input frames to create an output frame. Source plugins create an output based only on the plugin parameters, and do not expect or use any input frames.

FreeFrame APIMinorVersion numbers should always have 3 digits - so divide the minor version number by 1000 and add the major version number to get the full version number.
Example: for version 0.511, APIMajorVersion=0 and APIMinorVersion=511.

VideoInfoStruct {
  FrameWidth: 32-bit unsigned integer
  FrameHeight: 32-bit unsigned integer
  BitDepth: 32-bit unsigned integer
  Orientation: 32-bit unsigned integer
}

Fields:

Remarks:

This structure is provided by the host when a CPU plugin is instantiated. When a GPU plugin is instantiated, it is passed a FFGLViewportStruct. The remarks below are not relevant to GPU plugins.

Video frames are provided to CPU plugins as large buffers of pixels. The Red, Green, Blue, and sometimes Alpha values of a single pixel are located at consecutive memory locations. A video buffer consists of FrameWidth x FrameHeight pixels.

Plugins using 32-bit as byte aligned 24-bit video should be careful not to overwrite the alpha (4th) byte of each pixel (e.g. by using it as a processing space) as this may be used soon by hosts with 32-bit video processing becoming more accessible.

If Orientation == 1 the first pixel at the pointer to the frame is the top left pixel. If Orientation == 2, it is the bottom left pixel. This is particularly important for text and live input (to avoid mirroring of the image).

The byte order of color values depends on the target platform (i.e. big-endian versus little-endian). For little-endian platforms (e.g. Intel PC), the byte order is BGRA for 32-bit and BGR for 24-bit colors. For big-endian machines (e.g. MAC), the byte order for 32-bit is ARGB and RGB for 24-bit.
For all target platforms, the unsigned integer representation of a 32-bit color is the same, namely (in C syntax):

unsigned int color = (A << 24) + (R << 16) + (G << 8) + (B << 0);

SetParameterStruct {
  ParameterNumber: ParameterNumber
  NewParameterValue: ParameterValue
}

Fields:

PluginExtendedInfoStruct{
  PluginMajorVersion: 32-bit unsigned integer
  PluginMinorVersion: 32-bit unsigned integer
  Description: 32-bit pointer to null terminated string
  About: 32-bit pointer to null terminated string
  FreeFrameExtendedDataSize: 32-bit unsigned integer
  FreeFrameExtendedDataBlock: 32-bit pointer
}

Fields:

Remarks:

FreeFrame PluginMinorVersion numbers should always have 3 digits - so divide the minor version number by 1000 and add the major version number to get the full version number. Example: 0.751 APIMajorVersion=0, APIMinorVersion=751

ProcessFrameCopyStruct{
  numInputFrames: 32-bit unsigned integer
  ppInputFrames: 32-bit pointer to array of pointers
  pOutputFrame: 32-bit pointer
}

Fields:

FFGLTextureStruct {
  Width: 32-bit unsigned integer
  Height: 32-bit unsigned integer
  HardwareWidth: 32-bit unsigned integer
  HardwareHeight: 32-bit unsigned integer
  Handle: 32-bit unsigned integer (GLuint)
}

Fields:

Remarks:

Hosts must provide textures of type GL_TEXTURE_2D stored with the standard OpenGL image orientation where texture coordinate (0,0) is the lower left corner of the image. GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_3D, etc, are not supported.

FFGLViewportStruct {
   X: 32-bit unsigned integer
   Y: 32-bit unsigned integer
   Width: 32-bit unsigned integer
   Height: 32-bit unsigned integer
}

Fields:

Remarks:

This structure is provided by the host when a GPU plugin is instantiated. When a CPU plugin is instantiated, it is passed a VideoInfoStruct. The remarks below are not relevant to CPU plugins.

GPU plugins draw their output into the host's OpenGL viewport during calls to processOpenGL. Plugins can expect that this viewport will not change during the lifetime of the plugin. If the host's viewport changes (perhaps because the user changed their output window size or output resolution), any loaded plugins must be deinstantiated and reinstantiated and provided with the new viewport information.

ProcessOpenGLStruct {
   numInputTextures: 32-bit unsigned integer
   ppInputTextures: 32-bit pointer to array of pointers to FFGLTextureStruct
   HostFBO: 32-bit unsigned integer (GLuint)
}

Fields:

Remarks:

Hosts must provide textures of type GL_TEXTURE_2D stored with the standard OpenGL image orientation where texture coordinate (0,0) is the lower left corner of the image. GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_3D, etc, are not supported.

Hosts will often wish to capture the output of a GPU plugin to an offscreen texture. Hosts and plugins *must* use the the EXT_framebuffer_object ("FBO") extension to do this. Use of any other methods (such as 'PBuffers') is not supported.

When a host calls ProcessOpenGL with an FBO active, the host must provide the handle to the FBO in the 'HostFBO' field. Plugins that use their own FBOs internally must restore the host's FBO with a call to glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, HostFBO) so that the host can successfully capture the output of the plugin.

When a host calls ProcessOpenGL with no FBO active, it must pass a value of 0 in the 'HostFBO' field.

5. Functions

Some functions are specific to an instance of a plugin, and thus require a valid InstanceID in the InstanceID field, others concern global parameters that are the same for all instances, and so do not require a valid InstanceID.

Global functions:

Instance specific functions:

Input value:

Return value:

Remarks:

Gets information about the plugin - version, unique id, short name and type (see PluginInfoStruct).
This function should be identical in all future versions of the FreeFrame API.

Input value:

Return value:

Remarks:

Input value:

Return value:

Remarks:

Input value:

Return value:

Remarks:

Input value:

Return value:

Input value:

Return value:

Input value:

Return value:

Remarks:

Plugins must support either CPU or GPU processing, but not both. CPU processing involves repeated calls to processFrame and/or processFrameCopy. GPU processing involves repeated calls to processOpenGL.

CPU plugins should return FF_UNSUPPORTED (0) for FF_CAP_PROCESSOPENGL.

GPU plugins should return FF_SUPPORTED (1) for FF_CAP_PROCESSOPENGL.

GPU plugins should return FF_UNSUPPORTED (0) for FF_CAP_16BITVIDEO, FF_CAP_24BITVIDEO, FF_CAP_32BITVIDEO, FF_CAP_PROCESSFRAMECOPY, and FF_CAP_COPYORINPLACE.

FF_CAP_MINIMUMINPUTFRAMES, FF_CAP_MAXIMUMINPUTFRAMES, and FF_CAP_SETTIME apply to both CPU and GPU plugins, so all plugins should return appropriate values for these queries.

CPU PLUGIN/HOST BIT DEPTH
Typically a host program has a preferred bit depth that it uses to perform its operations, and it will ask the plugin if it supports that bit depth. If the plugin does not support it, the host may decide not to use the plugin and deinitialize it. If the host supports a 2nd or 3rd format, it may check with the plugin to see if those are supported, and use one of them instead.

Input value:

Return value:

Input value:

Return value:

Remarks:

Hosts may decide to present an appropriate visual interface depending on the parameter type. All FreeFrame data for the moment is passed as 32-bit floats from 0-1, except text data.

Input value:

Return value:

Remarks:

Input value:

Return value:

Input value:

Return value:

Remarks:

Sets the value of a parameter.

Input value:

Return value:

Remarks:

Returns value of a parameter. Note that the return value has different interpretations, depending on the type of the parameter (see ParameterValue).

Input value:

Return value:

Remarks:

Input value:

Return value:

Remarks:

Input value:

Return value:

Remarks:

CPU PLUGINS:	The processFrameCopy function performs a source->dest buffer frame process in CPU effect plugins. Multiple input frames can be processed by processFrameCopy, so plugins should indicate the number of input frames they prefer with getPluginCaps. CPU source plugins should not support processFrameCopy as they do not require input frames. Plugins do not have to support processFrameCopy, but they must support the processFrame (in-place) method of processing. Support for processFrameCopy should be indicated in getPluginCaps. The plugin can also define (via getPluginCaps) which of processFrameCopy or processFrame should be used for optimal performance.
GPU PLUGINS:	Do not implement this function, return FF_FAIL;

Input value:

Return value:

Remarks:

This function is provided to allow hosts to optimize the rendering of input frames. Due to user input a plugin may only plan to render certain input channels - this function allows hosts to ask the plugin which input channels it plans to render.

Input value:

Return value:

Remarks:

#define FF_INSTANTIATEGL 18

DWORD instantiateGL(const FFGLViewportStruct *pHostGLViewport)

Input value:

Return value:

Remarks:

#define FF_DEINSTANTIATEGL 19

DWORD deInstantiateGL(void *instanceID)

Input value:

Return value:

Remarks:

Input value:

Return value:

Remarks:

Plugins that support the setTime function should return FF_TRUE when asked for the FF_CAP_SETTIME capability in getPluginCaps.

Hosts must not call SetTime while any of the plugin's process functions are still executing.