Biometric Capture

Implementing the Biometric Capture Session in a View-based UI using a fragment for the camera preview with all its defined functionalities.

This example shows how to integrate the Biometric Capture Session to your project. The Biometric Capture Session extracts a frames collection from the camera preview.

Integration of Biometric capture

Capture session can be configured by changing values for MBCaptureSessionOptions.

Set auto capturing

Tells whether the capture is automatic or manual:
- Automatic
  - Automatically take an image with a specific time that you can configure with timeBeforeAutomaticCapture.
- Manual
  - Manually take an image with the help of a button that the library shows.

Set number of frame to collect

Describes the number of frames to collect during the capture session.

Set time before automaticCapture

timeBeforeAutomaticCapture
- Number of seconds that the user needs to wait in automatic capture.

Set camera quality

Set the options for the camera.

MBCameraOptions is used to set the options for the camera.
- MBTargetResolution is the resolution in which the frames are collected.
- MBPreviewScaleType is the mode used to scale the camera preview in the view.
  - MBPreviewScaleType.FILL_CENTER will ensure the camera preview is taking up the whole view in both width and height. However, this means that some parts of the image will be cropped from the preview. This does not have any impact on the images captured from the camera.
  - MBPreviewScaleType.FIT_CENTER will ensure that exactly what is captured from the camera is shown in the view.
- MBCameraPosition describes whether is displayed front or rear camera.
- MBOrientation support capturing for both landscape and portrait mode. The value should be set in accordance to the device orientation for the correct capturing performance.
  - MBOrientation.PORTRAIT is used to setting up the environment for a proper capturing performance in portrait mode.
  - MBOrientation.LANDSCAPE is used to setting up the environment for a proper capturing performance in landscape mode.

MBCameraOptions

Variable Name	Type	Default Value
targetResolution	MBTargetResolution	Size(1080, 1920)
previewScaleType	MBPreviewScaleType	MBPreviewScaleType.FIT_CENTER
cameraPosition	MBCameraPosition	MBCameraPosition.FRONT
orientation	MBOrientation	MBOrientation.PORTRAIT

MBTargetResolution

Field	Value
QHD	Size(540, 960)
HD_720	Size(720, 1280)
HD_1080	Size(1080, 1920)
HD_4K	Size(2160, 3840)

Enable debugging

If it is set to false will just display the overlay on the top of the camera. If it is set to true will display all components available ont the top of the camera(timer text, progress bar and face status text).

Enable payload optimization

Used to enable or disable optimization of the payload size generated by the capture session.

Set capture constrains

MBCaptureConstraints is used to ensure to capture high quality data during the capture process. You can select between to types of constraints: MBCaptureConstraints.V1, MBCaptureConstraints.V2.
- MBCaptureConstraints.V1 is the oldest version and uses limited set of constrains.
- MBCaptureConstraints.V2 is the newest version. Ensures more accurate and high quality data captured and better performance.

The listener to use for validation events changes depending on the selected constraint type. See Validating events for more details

In the future the MBCaptureConstraints.V1 will be deprecated, therefore we encourage to use the newest version MBCaptureConstraints.V2.

Device orientation

For the version MBCaptureConstraints.V2 you need to be aware that when the screen mode is portrait the parameter orientation of MBCameraOption should be set to MBOrientation.PORTRAIT, and MBOrientation.LANDSCAPE if the screen orientation is landscape. If we do not set the correct value the face capture sdk will not perform well and will return a wrong resolution in the collected frames.

Set user interface

Create your own UI

To initialize the Biometric Capture is required to extend the class from MBCaptureSessionListener. And the others listeners are optional.

class CaptureFragment : Fragment(R.layout.fragment_capture),
        MBCaptureSessionListener,
        MBCountDownListener,
        MBCaptureProgressListener,
        MBBoundingBoxFaceValidatorListener
{
    override fun onStateChanged(stateEnum: MBCaptureState) { }

    override fun onSuccess(result: MBCaptureSessionResult?) { }

    override fun onFailure(errorEnum: MBCaptureSessionError) { }

    override fun onValidating(faceBoxStatus: MBFaceBoundingBoxStatus, faceGeometry: MBFaceGeometryModel) { }

    override fun onCaptureProgress(captureProgressCounter: Float) { }

    override fun onCountdown(timeCounter: Int) {}

 }

Setup capture session

Instantiate MBCaptureSessionService.

private lateinit var captureSessionService: MBCaptureSessionService

Setup options

Instantiate and initialize MBCaptureSessionOptions:

var captureSessionOptions = MBCaptureSessionOptions.Builder()
                        .debugging(true)
                        .captureConstrains(MBCaptureConstraints.V2)
                        .cameraQuality(MBCameraOptions(orientation = MBOrientation.PORTRAIT))
                        .build()

Initialize MBCaptureSessionService:

 captureSessionService = MBCaptureSessionService(
                requireContext(),
                this,
                captureSessionOptions
        )

Initialize listeners:

        captureSessionService.captureSessionListener = this
        captureSessionService.captureProgressListener = this
        captureSessionService.countDownListener = this
        captureSessionService.faceBoundingBoxValidatorListener = this

Start camera:

   mBinding.container.addView(captureSessionService.getCaptureSessionView())
   captureSessionService.startCamera()

Use our UI

To initialize the Biometric Capture is required to extend the class from MBCaptureSessionFragmentListener.

class OverlayActivity : AppCompatActivity(), MBCaptureSessionFragmentListener {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(binding.root)
    }

    override fun onCaptureFinished(result: MBCaptureSessionResult?) { }

    override fun onFailure(errorEnum: MBCaptureSessionError) { }

    override fun onStateChanged(stateEnum: MBCaptureState) { }
}

Instantiate and initialize MBCaptureSessionFragment:

val captureSessionFragment = MBCaptureSessionFragment()

Initialize Biometric options:

captureSessionFragment.captureSessionOptions = MBCaptureSessionOptions.Builder()
                        .debugging(true)
                        .captureConstrains(MBCaptureConstraints.V2)
                        .cameraQuality(MBCameraOptions(orientation = MBOrientation.PORTRAIT))
                        .build()

Initialize MBUIOptions:

captureSessionFragment.style = MBUIOptions.Builder().build()

Initialize MBCaptureSessionFragmentListener:

 captureSessionFragment.captureSessionFragmentListener = this

Configure

Variable Name	Type	Default Value
textLocalizationBoundingBox	MBTextLocalizationsBoundingBox	MBTextLocalizationsBoundingBox()
showCountdownTimerLabel	boolean	false
showProgressBar	boolean	false
showFaceStatusLabel	boolean	false

Enable timer label.

showCountdownTimerLabel
- Is used to show or hide the label displayed during countdown before capture starts. The text for the countdownLabel can be set using the countdownLabelText option.

Enable Progressbar.

showProgressBar
- Is used to show or hide the progress bar. The progress bar indicates to the user how far in the capture process we are.

Enable face status label.

faceStatusTexts
- Is used to show or hide the face status label. The face status label is used to indicate to the user if the face is in the correct position or not.

Set text localizations.

textLocalizationBoundingBox
- It is used to Describe the text that is displayed on the top of the camera. It receives a string that is used to customize the text shown in the countdown label and a MBFacePositionStatusText a MBFaceDistanceStatusText and a MBFacePoseStatusTexts.

TextLocalizationsBoundingBox

Variable Name	Type	Default Value
countdownLabelText	string	'Hold Still'
facePositionStatusTexts	MMBFacePositionStatusText	MBFacePositionStatusText()
faceDistanceStatusTexts	MBFaceDistanceStatusText	MBFaceDistanceStatusText()
facePoseStatusTexts	MBFacePoseStatusTexts	MBFacePoseStatusTexts()

facePositionStatusText
- MBFacePositionStatusText class contains the default face position status strings.

Member	Type	Description
faceTooFarUp	string	Text to display when the face i too far up with respect to the region of interest
faceTooFarDown	string	Text to display when the face is too far down with respect to the region of interest
faceTooFarLeft	string	Text to display when the face is too far left with respect to the region of interest
faceTooFarRight	string	Text to display when the face is too far right with respect to the region of interest
faceNotFound	string	Text to display when no face is found in the frame

faceDistanceStatusText
- MBFaceDistanceStatusText class contains the default face distance status strings.

Member	Type	Description
faceTooFarAway	string	Text to display when the face is too far away with respect to the region of interest
faceTooClose	string	Text to display when the face is too close with respect to the region of interest
faceValidDistance	string	Text to display when the face has a valid distance with respect to the region of device

facePoseStatusTexts
- MBFacePoseStatusTexts class contains the default face pose status strings.

Member	Type	Description
faceExcessiveYaw	string	Text to display when the face has an excessive yaw pose degree
faceExcessivePitch	string	Text to display when the face has an excessive pitch pose degree
faceExcessiveRoll	string	Text to display when the face has an excessive roll pose degree

Listen for events

Events from your own UI

Success event

The function onSuccess gets a result of the capture process if it is successfully finished, result contains a list of nullable Bitmaps.

  override fun onSuccess(result: MBCaptureSessionResult?) {

  }

faceImage is a high quality image captured after collecting frames.
frames is the frame collection.
serializedData() is the faceImage and frames in Protobuf serialization.

Failure event

Function onFailure is executed when the capture process fails. MBCaptureSessionError tells the type of error that occurred.

  override fun onFailure(errorEnum: MBCaptureSessionError) {

  }

MBCaptureSessionError entries:

UNABLE_TO_OPEN_CAMERA describes failure while opening the camera.
UNABLE_TO_COLLECT_FRAMES the capture has is aborted.

Validating events

MBCaptureConstraints.V2

Function onValidating is executed when frames are being analyzed. It has an instance of MBFaceBoundingBoxStatus and MBFaceGeometryModel as arguments.

 override fun onValidating(faceBoxStatus: MBFaceBoundingBoxStatus, faceGeometry: MBFaceGeometryModel) {

 }

MBFaceBoundingBoxStatus

Field	Type	Default Value
distanceError	MBDistanceError?	null
positionErrors	List of MBPositionError	listOf()
poseErrors	List of MBPoseError	listOf()

MBDistanceError

Is a nullable enum representation of the distance between the device and the face.
- TOO_FAR_AWAY it says the face is too far away.
- TOO_CLOSE it says the face is too close.
- null it says the distance is valid.

MBPositionError

Is an enum representation of the face position in correspondence with the region of interest. This value is returned as member of a list of its own type because the face position can have from one to four errors at the same time. For instance: a face can be too far left and too far up at the same time. Then the list contains [TOO_FAR_LEFT, TOO_FAR_UP]. If the list is empty that means the position of the face is valid.

MBPositionError entries:

TOO_FAR_AWAY Face is too far away.
NOT_FOUND Face is not found.
TOO_FAR_UP Face is too far up.
TOO_FAR_LEFT Face is too far left.
TOO_FAR_DOWN Face is too far down.
TOO_FAR_RIGHT Face is too far right.

MBPoseError

Is an enum representation of the face pose angle. This value is returned as member of a list of its own type because the face pose angle can have from one to three errors at the same time. For instance: a face can have an excessive yaw rotation and an excessive roll rotation. Then the list contains [YAW_EXCESSIVE_FACE_ROTATION, ROLL_EXCESSIVE_FACE_ROTATION]. If the list is empty that means the pose of the face is valid.

MBPoseError entries:

YAW_EXCESSIVE_FACE_ROTATION the face has excessive yaw rotation.
ROLL_EXCESSIVE_FACE_ROTATION the face has excessive roll rotation.
PITCH_EXCESSIVE_FACE_ROTATION the face has excessive pitch rotation.

MBFaceGeometryModel

Field	Type	Default Value
yaw	Float	0.0
pitch	Float	0.0
roll	Float	0.0
boundingBox	BoundingBox	BoundingBox()

yaw is a representation in degrees of the face yaw pose.
pitch is a representation in degrees of the face pitch pose.
roll is a representation in degrees of the face roll pose.
boundingBox is a representation of the face bounding box coordinates.

BoundingBox

Field	Type	Default Value
left	Int	0
top	Int	0
right	Int	0
bottom	Int	0

In a preview view where the origin XY coordinate is the top left of the screen.

left is the left face bounding box edge in the preview view.
top is the top face bounding box edge in the preview view.
right is the right face bounding box edge in the preview view.
bottom is the bottom face bounding box edge in the preview view.

MBCaptureConstraints.V1

    override fun onValidating(faceStatus: MBFaceStatus) { }

faceStatus describes the face location in the camera preview.

Countdown event

Function onCountdown is executed every second before the engine start collecting frames. The time limit is set in the options.

  override fun onCountdown(timeCounter: Int) {

  }

Capture state event

The event onStateChanged Is executed every time the capture state changes. The parameter stateEnum is a representation of the current state.

MBCaptureState

INITIALIZING Is the first state and tell when the camera is ready.
VALIDATING Is the second state and tell when the frames start being analyzed.
COUNT_DOWN Is the third state and tell when a valid face is found in a single frame. If there is an invalid face in the following frames the state return to VALIDATING.
CAPTURING is the fourth state and tell when the frames start being collected.If there is an invalid face in the following frames the stage return to VALIDATING.
CAPTURE_FINISHED Is the last state and tell when the capturing is successfully finished.
PROCESSING Is the state that tells when the captured frames are being precessed.

override fun onStateChanged(stateEnum: MBCaptureState) {

}

Capture progress

The method onCaptureProgress represents the capture progress with arguments values between 0-1.

 override fun onCaptureProgress(captureProgressCounter: Float) {

 }

Events from our UI

Success event

The event onCaptureFinished is executed for MBCaptureSessionService when the capture session has successfully finished.

 override fun onCaptureFinished(result: MBCaptureSessionResult?) { }

MBCaptureSessionResult

faceImage is a high quality image captured after collecting frames.
frames is the frame collection.
serializedData() is the faceImage and frames in Protobuf serialization.

Failure event

The event onFailure is executed when the capture process fails. MBCaptureSessionError tells the type of error that occurred.

  override fun onFailure(errorEnum: MBCaptureSessionError) { }

MBCaptureSessionError entries:

UNABLE_TO_OPEN_CAMERA describes failure while opening the camera.
UNABLE_TO_COLLECT_FRAMES the capture has is aborted.

Capture state event

The event onStateChanged is executed every time the capture state changes. The parameter stateEnum is a representation of the current state.

MBCaptureState
- INITIALIZING is the first state and tell when the camera is ready.
- VALIDATING is the second state and tell when the frames start being analyzed.
- COUNT_DOWN is the third state and tell when a valid face is found in a single frame. If there is an invalid face in the following frames the state return to VALIDATING.
- CAPTURING is the fourth state and tell when the frames start being collected.If there is an invalid face in the following frames the stage return to VALIDATING.
- CAPTURE_FINISHED is the last state and tell when the capturing is successfully finished.
- PROCESSING is the state that tells when the captured frames are being precessed.

override fun onStateChanged(stateEnum: MBCaptureState) { }

Implementing the Biometric Capture Session in a View-based UI using a fragment for the camera preview with all its defined functionalities.​

Integration of Biometric capture​

Set auto capturing​

Set number of frame to collect​

Set time before automaticCapture​

Set camera quality​

Enable debugging​

Enable payload optimization​

Set capture constrains​

Device orientation​

Set user interface​

Create your own UI​

Setup capture session​

Setup options​

Use our UI​

Configure​

Listen for events​

Events from your own UI​

Success event​

Failure event​

Validating events​

MBCaptureConstraints.V2​

MBCaptureConstraints.V1​

Countdown event​

Capture state event​

Capture progress​

Events from our UI​

Success event​

Failure event​

Capture state event​

Implementing the Biometric Capture Session in a View-based UI using a fragment for the camera preview with all its defined functionalities.

Integration of Biometric capture

Set auto capturing

Set number of frame to collect

Set time before automaticCapture

Set camera quality

Enable debugging

Enable payload optimization

Set capture constrains

Device orientation

Set user interface

Create your own UI

Setup capture session

Setup options

Use our UI

Configure

Listen for events

Events from your own UI

Success event

Failure event

Validating events

MBCaptureConstraints.V2

MBCaptureConstraints.V1

Countdown event

Capture state event

Capture progress

Events from our UI

Success event

Failure event

Capture state event