Analyze a recorded video file

classdoc: VideoDetector [c++]

A common use of the SDK is to process previously captured video files. The VideoDetector helps streamline this effort by decoding and processing frames from a video file. Like the FrameDetector, the constructor accepts a parameter for processing frames per second. This parameter regulates how many frames from the video stream get processed. During processing, the VideoDetector decodes and processes frames as fast as possible and actual processing times will depend on CPU speed. Please see this list of accepted file types and recommended video codecs that are compatible with the detector.

1. Create the detector

The VideoDetector [c++] constructor expects two parameters { processFrameRate, maxNumFaces and faceConfig }

                The maximum number of frames processed per second
                If not specified, DEFAULT_PROCESSING_FRAMERATE=30
              double processFrameRate,
                The maximum number of faces to track
                If not specified, DEFAULT_MAX_NUM_FACES=1
              unsigned int maxNumFaces,

                Face detector configuration
                If not specified, defaults to FaceDetectorMode.SMALL_FACES

                FaceDetectorMode.LARGE_FACES=Faces occupying large portions of the photo
                FaceDetectorMode.SMALL_FACES=Faces occupying small portions of the photo
              FaceDetectorMode faceConfig

Here's an example:

affdex::VideoDetector detector(15);

2. Configure the detector

In order to initialize the detector, a valid location of the data folder must be specified:

Data folder
The Affdex classifier data files are used in frame analysis processing. These files are supplied as part of the SDK. The location of the data files on the physical storage needs to be passed to a detector in order to initialize it by calling the following with the fully qualified path to the folder containing them:

std::string classifierPath="/home/abdo/affdex-sdk/data"

3. Configure the callback functions

The Detectors use callback functions defined in interface classes to communicate events and results. The event listeners need to be initialized before the detector is started: The FaceListener is a client callback interface which sends notification when the detector has started or stopped tracking a face. Call setFaceListener to set the FaceListener:

classdoc: FaceListener[c++]

class MyApp : public affdex::FaceListener {
  MyApp() {

  affdex::Detector detector;

The ImageListener is a client callback interface which delivers information about an image which has been handled by the Detector. Call setImageListener to set the ImageListener:

classdoc: ImageListener [c++]

class MyApp : public affdex::ImageListener {
  MyApp() {

  affdex::Detector detector;

The ProcessStatusListener is a callback interface which provides information regarding the processing state of the detector. Call setProcessStatusListener to set the ProcessStatusListener:

classdoc: ProcessStatusListener [c++]

class MyApp : public affdex::ProcessStatusListener {
  MyApp() {

  affdex::Detector detector;

4. Choose the classifiers

The next step is to turn on the detection of the metrics needed. For example, to turn on or off the detection of the smile and joy classifiers:


To turn on or off the detection of all expressions, emotions, emojis, or appearances:


To check the status of a classifier at any time, for example smile:


5. Initialize the detector

After a detector is configured using the methods above, the detector initialization can be triggered by calling the start method:


The start() spawns a thread the connects to the specified camera to capture video frames then processes them and use the callback functions to notify of the captured frames, results and exceptions (if exist). start() is a non-blocking call.

6. Process the video

Once the detector is started, the processing begins by calling the process function, the path to video file you are processing is passed in as a parameter:

void process(String path);

On completion of the video processing, the onProcessingFinished function is called from the ProcessStatusListener. process() is a non-blocking call.

6. Stop the detector

At the end of the interaction with the detection. Stopping the detector can be done as follows:


The processing state can be reset. This method resets the context of the video frames. Additionally Face IDs and Timestamps are set to zero (0):