FreeFrame Open Video Plugin System Specification

Version 1.0 - 03

1. OS integration
2. Basic types and constants
3. Enumerated and derived types
4. Structures
5. Functions
    5.1. Global functions
    5.2. Instance specific functions

 

Copyright (c) 2002, 2003 www.freeframe.org
All rights reserved.

This document is a functional specification for the FreeFrame application programmers interface (API). It is intended to be subject to the same license as the code supplied as examples of use of the FreeFrame API.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

 

1. OS integration

FreeFrame plugins are distributed and used as compiled shared objects (.so) in Linux, as Frameworks in Mac OS X and as Dynamic Link Libraries (.dll) in Windows.

FreeFrame Plugins export a single function: plugMain. This is passed 3 values: a 32-bit function code, a 32-bit input value and a 32-bit instance identifier. It returns a 32-bit output value.
The input values and output values have different types according to the function code. This may be implemented differently on different platforms in different languages, but the format of the values must be consistent.


plugMain

(unsigned integer functionCode, unsigned integer inputValue, unsigned integer instanceID)

Parameters:

functionCode: Tells the plugin which function is being called (see function table)
inputValue: 32-bit value or 32-bit pointer to some structure, depending on function code.
instanceID: 32-bit instance identifier. Only used for instance specific functions (see function table)

Return value:

Depends on function code. See function table for details.

Remarks:

PLUGIN DEVELOPERS: You shouldn't need to change this function. This is the one and only function exposed by the library containing the actual plugin.

 

2. Basic types and constants

All numbers in this document are ordinary decimal numbers, expect if specified otherwise.

The following types are used:

Type Range of type Notes
32-bit unsigned integer 0 to 4294967295 -
32-bit pointer 0 to 4294967295 needs to be a valid memory location
32-bit IEEE float -lots to +lots 1 sign bit, 8 bit exponent, 24 bit mantissa
 

The following constants are used as return values:

FF_SUCCESS 0
FF_FAIL (hex) FFFFFFFF
FF_TRUE 1
FF_FALSE 0
FF_SUPPORTED 1
FF_UNSUPPORTED 0

Remarks:

 

3. Enumerated and derived types

InstanceIdentifier

 (32-bit unsigned integer)

Definition:

InstanceIdentifier: Unique identifier for the instance of a plugin. For some plugins, this value might represent a pointer to a plugin-specific data structure containing the instance's current state. Or it could be an index to a table of all active instances of a plugin, or any other value that uniquely identifies an instance.

NumParameters

 (32-bit unsigned integer)

Definition:

NumParameters: Specifies the number of parameters that the plugin implements.

ParameterNumber

 (32-bit unsigned integer)

Definition:

ParameterNumber: Index of a parameter, starting at 0 for the first parameter.

ParameterName

 (array of 16 1-byte ASCII characters, *not null terminated*)

Definition:

ParameterName: The name of the Parameter as it will be displayed by the host on the UI.

ParameterValue

 (32-bit float value OR 32-bit pointer)

Definition:

ParameterValue: Float value from 0-1 or pointer to null terminated string. (See remarks)

Remarks:

Apart from text parameters, FreeFrame parameter values are always 32-bit floats, and the range of values permitted is STRICTLY 0-1 (0 <= ParameterValue <= 1). This allows faster processing and a good range of values over a standard range, so the host can run up sliders or similar for the plugin. The Use of any values outside this range will result in hideous incompatibilities. The ParameterDisplayValue can be used to display whatever actual values the plugin likes e.g. 0-255, 0-767, 1-256 or whatever. The plugin should translate the standard 0-1 float range into the values it needs for its processing.
For text parameters, this 32-bit value represents a pointer to a null terminated string.


ParameterDefaultValue

 (32-bit float value OR 32-bit pointer)

Definition:

ParameterDefaultValue: The initial default value for this parameter. 32-bit float value or 32-bit pointer to null terminated string. (See remarks)

Remarks:

Plugins should always specify default values. Sometimes a host may not implement all parameters on a plugin, so the plugin must use default values until told to do otherwise by the host. Apart from text parameters, FreeFrame parameter values are always 32-bit floats, and the range of values permitted is STRICTLY 0-1.
For text parameters, this 32-bit value represents a pointer to a null terminated string.


ParameterDisplayValue

 (array of 16 1-byte ASCII characters, *not null terminated*)

Definition:

ParameterDisplayValue: String representing the current display value of this parameter. The plugin can display whatever it likes here e.g. just the float, a rounded 0-100 '%' representation for the user, words representing states like 'on' / 'off', different effects that the one plugin can do etc...

PluginCapsIndex

 (32-bit unsigned integer)

Definition:

PluginCapsIndex: 32-bit unsigned integer. Specifies certain capabilities of a plugin that the host may want to enquire about.:
0 = 16-bit 5-6-5 pixelformat support
1 = 24-bit pixelformat support
2 = 32-bit (or 24-bit 32-bit aligned) pixelformat support
 
3 = Plugin supports processFrameCopy function
 
Calls to these caps indexes are only meaningful if the plugin has reported that it supports processFrameCopy:
10 = Minimum input frames
11 = Maximum input frames
15 = Plugin optimized for copy or in-place processing

Remarks:

See getPluginCaps for more information about the usage of this structure.

See VideoInfoStruct for more information about the pixelformats.


ParameterType

 (32-bit unsigned integer)

Definition:

ParameterType: 32-bit unsigned integer. Tells the host what kind of data the parameter is. Current meaningful values:
Value Type Description
0 boolean 0.0 defined as false and anything else defined as true - e.g. checkbox
1 event Similar to boolean but for a momentary push button style trigger. 1.0 is set momentarily to denote a simple event - e.g. pushbutton / keystroke.
2 red The 3 colors e.g. for a colorpicker
3 green
4 blue
5 xpos For x, y video interaction e.g. cursor - these should denote position within the video frames as specified in the VideoInfoStruct.
6 ypos
10 standard A standard parameter representing an unspecified float value
100 text A null terminated text input type - Note: only this type has a different data type for the moment

Remarks:

The range of values permitted for all variables (except text) is strictly 0-1. This also holds for xpos/ypos variables (where, for example, xpos=0 corresponds to a screen x-coordinate of 0, and xpos=1 corresponds to an actual x-coordinate of FrameWidth-1. See VideoInfoStruct).


InputChannel

 32-bit unsigned integer

Definition:

InputChannel: Input channel as relates to the use of multiple inputs in processFrameCopy The first channel is 0.

InputStatus

 32-bit unsigned integer

Definition:

InputStatus: Status of input channels as relates to the use of multiple inputs in processFrameCopy.
0 = Not in use
1 = In Use

 

4. Structures

PluginInfoStruct

 (size = 32 bytes)
PluginInfoStruct {
  APIMajorVersion: 32-bit unsigned integer
  APIMinorVersion: 32-bit unsigned integer
  PluginUniqueID: 4 1-byte ASCII characters *not null terminated*
  PluginName: 16 1-byte ASCII characters *not null terminated*
  PluginType: 32-bit unsigned integer
}

Fields:

APIMajorVersion: Represents number before decimal point in version numbers
APIMinorVersion: Represents number after decimal point in version numbers
PluginUniqueID: Unique ID for plugin
PluginName: Name of plugin
PluginType: Current meaningful values:
0 = effect
1 = source
(see remarks)

Remarks:

Plugins of PluginType effect are passed frames of video, which they then modify. Source plugins are simply passed a pointer where they paint frames of video. One example of a source plugin would by a visual synthesizer which uses the parameters to synthesize video.

FreeFrame APIMinorVersion numbers should always have 3 digits - so divide the minor version number by 1000 and add the major version number to get the full version number.
Example: for version 0.511, APIMajorVersion=0 and APIMinorVersion=511.


VideoInfoStruct

 (size = 16 bytes)
VideoInfoStruct {
  FrameWidth: 32-bit unsigned integer
  FrameHeight: 32-bit unsigned integer
  BitDepth: 32-bit unsigned integer
  Orientation: 32-bit unsigned integer
}

Fields:

FrameWidth: Width of video frame in pixels
FrameHeight: Height of video frame in pixels
BitDepth: Current meaningful values:
0 = 16-bit, 5-6-5 packed
1 = 24-bit, packed (see remarks)
2 = 32-bit, also suitable for 32-bit unsigned integer aligned 24-bit (see remarks).
(see remarks)
Orientation: Current meaningful values:
1 = origin at top left
2 = origin at bottom left
(see remarks)

Remarks:

All video data in FreeFrame is pixel ordered, so the R, G, B values of a single pixel are located at consecutive memory locations. A video buffer consists of FrameWidth x FrameHeight pixels.

Plugins using 32-bit as byte aligned 24-bit video should be careful not to overwrite the alpha (4th) byte of each pixel (e.g. by using it as a processing space) as this may be used soon by hosts with 32-bit video processing becoming more accessible.

If Orientation == 1 the first pixel at the pointer to the frame is the top left pixel. If Orientation == 2, it is the bottom left pixel. This is particularly important for text and live input (to avoid mirroring of the image).

The byte order of color values depends on the target platform (i.e. big-endian versus little-endian). For little-endian platforms (e.g. Intel PC), the byte order is BGRA for 32-bit and BGR for 24-bit colors. For big-endian machines (e.g. MAC), the byte order for 32-bit is ARGB and RGB for 24-bit.
For all target platforms, the unsigned integer representation of a 32-bit color is the same, namely (in C syntax):

unsigned int color = (A << 24) + (R << 16) + (G << 8) + (B << 0);
where A, R, G, B are the 8-bit color components. Plugin developers who aim for platform-independence should be aware of these differences and program their plugins accordingly.


SetParameterStruct

 (size = 8 bytes)
SetParameterStruct {
  ParameterNumber: ParameterNumber
  NewParameterValue: ParameterValue
}

Fields:

ParameterNumber: Index of the parameter to set.
NewParameterValue: New value for this parameter. For float parameters, this is a float from 0-1. For text parameters, this is a pointer to a null terminated string (see ParameterValue).

PluginExtendedInfoStruct

 (size = 24 bytes)
PluginExtendedInfoStruct{
  PluginMajorVersion: 32-bit unsigned integer
  PluginMinorVersion: 32-bit unsigned integer
  Description: 32-bit pointer to null terminated string
  About: 32-bit pointer to null terminated string
  FreeFrameExtendedDataSize: 32-bit unsigned integer
  FreeFrameExtendedDataBlock: 32-bit pointer
}

Fields:

PluginMajorVersion: Represents number before decimal point in version numbers.
PluginMinorVersion: Represents number after decimal point in version numbers.
Description: A description of the plugin - intended to be made available to the user in hosts supporting this function via the UI. Pointer value can be 0 if not provided by plugin!
About: Author and license information. Pointer value can be 0 if not provided by plugin!
FreeFrameExtendedDataSize: Size in bytes of the FreeFrameExtendedDataBlock - or 0 if not provided by plugin.
FreeFrameExtendedDataBlock: 32-bit pointer to a FreeFrame ExtendedDataBlock - this interface is not yet in use, but provided to allow the FreeFrame project to establish an extended data block format in the future. Please do not use until a data format has been agreed.

Remarks:

FreeFrame PluginMinorVersion numbers should always have 3 digits - so divide the minor version number by 1000 and add the major version number to get the full version number. Example: 0.751 APIMajorVersion=0, APIMinorVersion=751


ProcessFrameCopyStruct

 (size = 12 bytes)
ProcessFrameCopyStruct{
  numInputFrames: 32-bit unsigned integer
  ppInputFrames: 32-bit pointer to array of pointers
  pOutputFrame: 32-bit pointer
}

Fields:

numInputFrames: Number of input frames contained in ppInputFrames.
ppInputFrames: A pointer to an array of pointers to input frames.
pOutputFrame: Pointer to output frame.

 

5. Functions

Some functions are specific to an instance of a plugin, and thus require a valid InstanceID in the InstanceID field, others concern global parameters that are the same for all instances, and so do not require a valid InstanceID.

Function Code Table

Global functions:

Code Function Input value type Output value type
0 getInfo none Pointer to a PluginInfoStruct
1 initialise none Success/error code
2 deInitialise none Success/error code
4 getNumParameters none NumParameters
5 getParameterName ParameterNumber Pointer to ParameterName
6 getParameterDefault ParameterNumber ParameterDefaultValue
10 getPluginCaps PluginCapsIndex Supported/unsupported/value
13 getExtendedInfo none Pointer to PluginExtendedInfoStruct
15 getParameterType ParameterNumber ParameterType

Instance specific functions:

Code Function Input value type Output value type
3 processFrame Pointer to a frame of video Success/error code
7 getParameterDisplay ParameterNumber Pointer to ParameterDisplayValue
8 setParameter Pointer to SetParameterStruct Success/error code
9 getParameter ParameterNumber ParameterValue
11 instantiate Pointer to VideoInfoStruct InstanceIdentifier
12 deInstantiate none Success/error code
14 processFrameCopy Pointer to ProcessFrameCopyStruct Success/error code
16 getInputStatus InputChannel InputStatus

 

5.1. Global functions

getInfo

 (function code = 0)

Input value:

none

Return value:

32-bit pointer to PluginInfoStruct if successful, FF_FAIL otherwise.

Remarks:

Gets information about the plugin - version, unique id, short name and type (see PluginInfoStruct).
This function should be identical in all future versions of the FreeFrame API.

HOST: Call this function first to get version information. The version defines the other function codes that are supported.

initialise

 (function code = 1)

Input value:

none

Return value:

FF_SUCCESS if successful, FF_FAIL or other error-codes if failed. (32-bit unsigned integer)

Remarks:

PLUGIN: Prepare the plug-in for processing. Set default values, allocate memory. When the plug-in returns from this function it must be ready to accept calls to instantiate.
HOST: This function *must* return before a call to instantiate.

deInitialise

 (function code = 2)

Input value:

none

Return value:

FF_SUCCESS if successful, FF_FAIL or other error-codes if failed. (32-bit unsigned integer)

Remarks:

PLUGIN: Tidy up, deallocate memory
HOST: This *must* be the last function called on the plugin

getNumParameters

 (function code = 4)

Input value:

none

Return value:

NumParameters indicating the number of parameters in plugin or FF_FAIL on error.

Remarks:

PLUGIN: Plugin developers should normally expect hosts to expose up to 8 parameters Some hosts may only expose 1 or 4 parameters, some may expose larger numbers. All parameters should have sensible defaults incase the user is unable to control them from the host.
HOST: Host developers should try to implement at least the first 4 parameters. 8 is the recommended number to expose.

getParameterName

 (function code = 5)

Input value:

index: ParameterNumber

Return value:

32-bit pointer to ParameterName (16 byte char array) containing the name of parameter specified by index. Returns FF_FAIL on error.

getParameterDefault

 (function code = 6)

Input value:

index: ParameterNumber

Return value:

ParameterDefaultValue indicating the default value of parameter specified by index. For float parameters, the return value is a 32-bit float (0<=value<=1). Return value for text parameters is a 32-bit pointer to a null terminated string.
Returns FF_FAIL on error.

getPluginCaps

 (function code = 10)

Input value:

capsIndex: PluginCapsIndex

Return value:

Indicates capability of feature specified by capsIndex. See PluginCapsIndex for a more detailed description of the individual capabilities. In every case, the return value can be represented as a 32-bit unsigned integer.
capsIndex value capsIndex meaning return values
0 16 bit video FF_SUPPORTED: plugin supports 16 bit video
FF_UNSUPPORTED: plugin doesn't support 16 bit video
1 24 bit video FF_SUPPORTED: plugin supports 24 bit video
FF_UNSUPPORTED: plugin doesn't support 24 bit video
2 32 bit video FF_SUPPORTED: plugin supports 32 bit video
FF_UNSUPPORTED: plugin doesn't support 32 bit video
3 processFrameCopy support FF_SUPPORTED: plugin supports processFrameCopy
FF_UNSUPPORTED: plugin doesn't support processFrameCopy
10 Minimum input frames Returns minimum number of input frames plugin requires or FF_FALSE if capability not supported.
11 Maximum input frames Returns maximum number of input frames plugin can process or FF_FALSE if capability not supported.
15 Plugin optimization Return value indicates whether plugin is optimized for Copy or InPlace processing.
0 = no preference
1 = InPlace processing is faster
2 = Copy processing is faster
3 = Both are optimized

Remarks:

Bitdepth format of the video:
The host asks the plugin if it is capable of its favorite bit depth, and uses that if it is available. If not the host may decide not to use the plugin and deinitialise it, or it may enquire again to see if a second choice format is supported.


getExtendedInfo

 (function code = 13)

Input value:

none

Return value:

32-bit pointer to PluginExtendedInfoStruct or FF_FAIL on error.

getParameterType

 (function code = 15)

Input value:

index: ParameterNumber

Return value:

ParameterType value which tells the host what kind of data the parameter index is.
Returns FF_FAIL on error.

Remarks:

Hosts may decide to present an appropriate visual interface depending on the parameter type. All FreeFrame data for the moment is passed as 32-bit floats from 0-1, except text data.

 

5.2. Instance specific functions

processFrame

 (function code = 3, needs valid instanceID!)

Input value:

pFrame: 32-bit pointer to byte array containing frame of video.

Return value:

FF_SUCCESS if successful, FF_FAIL if failed. (32-bit unsigned integer)

Remarks:

PLUGIN: Process a frame of video 'in place'. This is also the basic frameport for source plugins.
HOST: pFrame needs to be a valid pointer throughout this call as the plugin processes the frame 'in place'.

getParameterDisplay

 (function code = 7, needs valid instanceID!)

Input value:

index: ParameterNumber

Return value:

Pointer to ParameterDisplayValue containing a string to display as the value of parameter index.
Returns FF_FAIL if failed.

setParameter

 (function code = 8, needs valid instanceID!)

Input value:

setParam: Pointer to SetParameterStruct.

Return value:

Returns FF_SUCCESS if successful or FF_FAIL if failed

Remarks:

Sets the value of a parameter.

PLUGIN: A plugin shouldn't change parameter values by itself. All parameters are set by the plugin host through this function.

getParameter

 (function code = 9, needs valid instanceID!)

Input value:

index: ParameterNumber

Return value:

A ParameterValue or FF_FAIL on error.

Remarks:

Returns value of a parameter. Note that the return value has different interpretations, depending on the type of the parameter (see ParameterValue).


instantiate

 (function code = 11, needs valid instanceID!)

Input value:

videoInfo: 32-bit pointer to VideoInfoStruct.

Return value:

An InstanceIdentifier for the newly instantiated effect. Will be used as instanceID for future function calls.
Returns FF_FAIL on error.

Remarks:

PLUGIN: Prepare an instance of the Plug-in for processing. Set default values, allocate memory When the plug-in returns from this function it must be ready to process a frame.
Make a copy of the VideoInfoStruct locally as pointer may not be valid after function returns. The instance identifier you provide to the host must be the unique identifier of this instance until a call to deinitialise.
HOST: This function *must* return before a call to processFrame. Pointer to VideoInfoStruct *must* be valid until function returns. The InstanceIdentifier provided by the plugin is your handle to this instance of the plugin. The host must keep this instance identifier, and provide it in all calls to the plugin regarding that instance.

deInstantiate

 (function code = 12, needs valid instanceID!)

Input value:

none

Return value:

Returns FF_SUCCESS if successful or FF_FAIL if failed.

Remarks:

PLUGIN: Tidy up, deallocate memory
All memory associated with the instance being deinited must be freed here.
HOST: This function must be called to close an instance of the plugin. All instances should be closed before deInitialising the plugin!

processFrameCopy

 (function code = 14, needs valid instanceID!)

Input value:

copyStruct: 32-bit pointer to ProcessFrameCopyStruct

Return value:

Returns FF_SUCCESS if successful or FF_FAIL if failed.

Remarks:

The ProcessFrameCopy function performs a source->dest buffer frame process in effects plugins.
It is capable of processing multiple input frames - getPluginCaps should be used to see how many input frames the plugin would like.

PLUGIN: Effect Plugins must support the processFrame (in-place) method of processing. This method is optional. Plugins should specify if they are capable of this kind of processing - and if they have optimized a particular method - in the Plugin Caps system.
Source Plugins should not use this function as they do not require input frames. Source plugins should just use processFrame

getInputStatus

 (function code = 16, needs valid instanceID!)

Input value:

channel: InputChannel value.

Return value:

Returns an InputStatus value describing the status of channel.

Remarks:

This function is provided to allow hosts to optimize the rendering of input frames. Due to user input a plugin may only plan to render certain input channels - this function allows hosts to ask the plugin which input channels it plans to render.

HOST: A host may only call this function for channels in this range:
'Minimum input frames' <= channel < 'Maximum input frames' (see getPluginCaps).
One should assume that all channels < 'Minimum input frames' are always in use.