Intel® Facial Special Effects Filter allows camera pipeline to apply different adjustment effects to the video.

The following features for video adjustment are supported:

Face Special Effect Features

The Lip Color module takes YUV input data. With the facial landmark information fed into the module, the module identifies the lip area when there is a face within the input frame. For input frame with detected faces, the module further performs color modification for lip area with the users’ preference of level of adjustment input from the application. 

The Dark Circles block takes YUV input data. Facial landmark points and skin-tone likelihood are utilized to locate the under eye region for processing. Once identifying the regions, a content-adaptive blending with the users’ preference of level of adjustment input from the application is performed to blend the original YUV.

The Under Eye module takes YUV input data. Facial landmark points and skin-tone likelihood information are utilized jointly to locate the under eye regions for processing. 

The Aging module takes YUV input data. Facial landmark information, semi-circle regions of the detected face, and skin-tone likelihood map fed from the analytic module are utilized to locate the area around eyes for processing. Once identifying the target area, a process with the users’ preference of level of adjustment input from the application is operated on YUV values for all pixels within the area.

The Eye Size module takes YUV input data. With the facial landmark information fed into the module and the users’ preference of level of enlargement input from the application, the module internally derives the proper location within the face and the shape of the eyes users intend to have. 

The Face Shape module takes YUV input data. With the facial landmark information fed into the module and the users’ preference of level of face change-effect input from the application, the module internally derives the different shape version of the original face area and performs morphological warping to create the face special filters effect.

The Skin Foundation module takes YUV input data and blends the input with a user selected foundation color where the per-pixel skin-tone likelihood score serves as the blending factor here.

The Nose feature module takes YUV input data. With the facial landmark information fed into the module and the users’ preference of level of adjustment input from the application, the module internally derives the modified shape of the nose area and performs morphological warping to create the special effect.

The Cheekbone Color block takes YUV input data. Facial landmark and skin-tone likelihood information are utilized to locate the cheekbone regions for processing. Once identifying the regions, a content-adaptive blending with the users’ preference of level of adjustment input from the application is performed to blend the original YUV values and a pre-defined red color value for pixels within the cheekbone region.

Emotions Special Effects Feature

The Emotions module takes YUV input data. With the facial landmark information fed into the module and the users’ preference of level of adjustment input from the application, the module internally derives the modified shape of the mouth area and performs morphological warping to create the smiling face effect via changing the shape of users’ mouths.

Face Tone Special Effects Feature 

The Face Tone block takes YUV input data and adjusts all 3-channel information to produce a smooth version of the input. The skin-tone likelihood map fed from the analytic module with the users’ preference of level of adjustment input from the application is utilized to blend the input image and bilateral filtered image to produce the output smoothed image.

Control Panel

We provide a utility that can enable and adjust the strength of each feature. This utility is in the attachment FBContorl_20180208.7z

Image may be NSFW.
Clik here to view.

Sample Source Code

We provide DirectShow based and MFT0 based sample source code to demonstrate the usage. The two filter samples will be described as following. You are welcome to integrate this filter into your camera driver package if you're camera IHVs.

DirectShow based filter

DirectShow based filter is a combination of enumerated capture filter and custom source filter. Enumerated capture filter is referenced from Microsoft AmCap Sample while custom source filter implementation is referenced from Microsoft Push Source Filters Sample.

Image may be NSFW.
Clik here to view.

DirectShow based filter wrapping camera video input with picture enhancement

Image may be NSFW.
Clik here to view.The Null Renderer filter is a renderer that discards every sample it receives, without displaying or rendering the sample data. Maybe we could think of it like /dev/null in Linux. (Null Renderer Filter)

Image may be NSFW.
Clik here to view.DirectShow standard filter works as an interface to set configure (color format/resolution/FPS) to physical capture device. It exposes PIN_CATEGORY_CAPTURE for downstream to receive sample data. (DirectShow Video Capture Filters)

Image may be NSFW.
Clik here to view.

DirectShow standard filter design to grab sample data from upstream, it exposes both input and output pin. In virtual camera we connect its input pin to Capture Filter’s capture output pin, and connect its output pin to Null Filter. (Sample Grabber Filter)

Image may be NSFW.
Clik here to view.

Implement custom source filter by deriving from DirectShow CSource. (Custom Source Filter)

Image may be NSFW.
Clik here to view.

A module that performs Face Special Effects Filters via D3D11 interface VideoProcessorBlt and video processing extension. Support NV12 input/output, it’s not literally a DirectShow filter, but do the fface special effects function after CSC stage.

Image may be NSFW.
Clik here to view.A module that performs color space conversion via D3D11 interface VideoProcessorBlt. Used to convert YUY2/I420 format into NV12 as FB stage feed-in. It’s not literally a DirectShow filter.

Build and Installation:

In the attachment Intel®_Facial_Special_Effects_Filter_dshow_20170810.7z

1. Install Visual Studio 2015 and Windows 10 SDK (10.0.14393)
2. Build Win32 baseclasses\Release\strmbase.lib via baseclasses\baseclasses.sln
3. Build Win32 filters\vcam\Release\x86\vcam_x86.dll via filters\vcam\vcam.sln
4. Copy and register vcam_x86.dll via amdin command "regsvr32.exe vcam_x86.dll"
5. Download and install vc_redist.x86.exe from Visual C++ Redistributable for Visual Studio 2015 
6. Run 32-bit desktop camera app and select virtual camera device "Face Effects Camera"
7. Change registry keys under [HKEY_CURRENT_USER\SOFTWARE\Intel\Display\DXVA\Intel Dxva Virtual Camera] or run virtual_camera_face_effects_setting.reg
      [0]: default strength
      [1-100]: face special effect strength
      [101]: disable face special effect
8. Check the face effects.

Uninstall: regsvr32.exe /u vcam_x86.dll

Capabilities for DirectShow based filter:

1. Only support camera with MJPG or YUY2
2. Color format: Filter outputs YUY2
3. FPS: 60 fps
4. Three available resolutions 1280x720, 640x480 and 640x360 ^*
5. Not support rotation mode since DShow app isn’t available under Metro mode.
6. Max face special effects to 8 human faces.
7. Only support for DShow app, not support for MF-based app

^* GPU can support to 1920x1080. This filter software could be modified to support 1080P.

MFT0 based filter

Windows 10 offers IHVs and system OEMs the ability to create video processing plug-ins in the form of a Media Foundation Transform (MFT), known as the camera driver MFT. The driver MFT is also referred to as MFT0 to indicate that it's the first MFT to operate in the source reader. We add face special effects for video processing based on Driver MFT Sample. We implement capture and preview pin driver MFT on multi-pin cameras. For more info about creating a driver MFT, see Creating a camera driver MFT.

Notice:

1. Need to install camera driver before the following installation. If you install this MFT0 driver before installing camera driver, the filter could not be applied.
2. If your camera has its proprietary MFT0 filter, this MFT0 filter will replace with it. Only one MTF0 will be effective.

Build and Installation:

In the attachment Intel®_Facial_Special_Effects_Filter_mft0_20170810.7z

1. Install Visual Studio 2015 and Windows 10 SDK (10.0.14393)
2. Build x64 Release\x64\IntelMft0_x64.dll via SampleMft0.sln
3. Download and install vc_redist.x64.exe from Visual C++ Redistributable for Visual Studio 2015
4. Run install_intel_mft0.bat with admin to install MFT0 as step mentioned in Installing and registering the driver MFT
5. Run UWP (Metro mode) camera app.
6. Change registry keys under [HKEY_LOCAL_MACHINE\SOFTWARE\Intel\Display\DXVA\Intel Dxva Virtual Camera] or run create_mft0_regkey_local_machine.reg
      [0]: default strength
      [1-100]: face special effect strength
      [101]: disable face special effect
7. Check the face effects.

Uninstall: regsvr32 /u "C:\Program Files\Intel\Intel MFT0\IntelMft0_x64.dll"

Capabilities for MFT0 based filter:

1. Only support camera with MJPG or YUY2
2. Color format: MFT0 outputs NV12 (if camera support MJPG) or YUY2 (if camera support YUY2)
3. FPS: 60 fps
4. Max resolution 1280x720 defined in config.h ^*
5. Support rotation mode 0, 90, 180 and 270 degree with cropping and black filler (1280x720 -> 1080x720)
6. Max face special effects up to 8 human faces.
7. Only for MF or UWP (Metro) app on Win10 RS1, supports both MF and DShow app on Win10 RS2 with DShow Bridge enabled

^* GPU can support to 1920x1080. This filter software could be modified to support 1080P.

Try it via Installer

Notice:

1. Need to install camera driver before the following installation. If you install this MFT0 driver before installing camera driver, the filter could not be applied.
2. If your camera has its proprietary MFT0 filter, this MFT0 filter will replace with it. Only one MTF0 will be effective.

In the attachment Intel®_Facial_Special_Effects_Filter_Installer_20170427.7z

1. Run Setup.exe, it will install DirectShow based filter and MFT0 based filter.
2. Run 32-bit desktop camera app for DirectShow based or UWP (Metro mode) camera app for MFT0 based filter.
3. Check the face effects.

System Requirements

• OS: Windows 10 RS1+
• Platform: Skylake+
• Intel® Graphics Driver for Windows: 15.46+
• Apps: Skype (Desktop mode) / PotPlayer (Desktop mode) / Camera (Metro mode)