Biometric Capture

Implementing the Biometric Capture Session in a Jetpack Compose-based UI

This example shows how to integrate the Biometric Capture Session to your project. The Biometric Capture Session extracts a frames collection from the camera preview.

Create a compose activity.

ComposeBiometric extends from MBCaptureSessionServiceListener,MBOnValidatingListener,MBCountDownListener and MBCaptureProgressListener.

 class ComposeBiometric: ComponentActivity(),
        MBCaptureSessionListener,
        MBOnValidatingListener,
        MBCountDownListener,
        MBCaptureProgressListener {
    
    
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

          val options = MBCaptureSessionOptions.Builder()
            .automaticCapture(true)
            .build()
        
          val  captureSessionService = MBCaptureSessionService(
                context = this, this as LifecycleOwner,
                options, callback = this
            )

        setContent {
        //Starting screen here
        }
    }


    override fun onCaptureFinished(result: MBCaptureSessionResult?) { }

    override fun onCountdown(timeCounter: Int) { }

    override fun onValidating(faceStatus: MBFaceStatus) { }

    override fun onFailure(errorEnum: MBCaptureSessionError) { }

    override fun onStateChanged(stateEnum: MBCaptureState) { }

    override fun onCaptureProgress(captureProgressCounter: Float) { }
 }

Build an MBCaptureSessionOption.Builder instance.. All arguments are optional. It contains the default options for performing the capture session. Those default options can be redefined. In this case the capture process is set to automatic.

 val options = MBCaptureSessionOptions.Builder()
            .automaticCapture(true)
            .build()

MBCaptureSessionOptions

Inside the library we have some options for changing behaviour of capturing data:

Variable Name	Type	Default Value
numberOfFrameToCollect	number	3
timeBeforeAutomaticCapture	number	4
isDebugging	boolean	false
autoCaptureEnabled	boolean	true
payloadOptimization	boolean	false
cameraQuality	MBCameraOptions	MBCameraOptions

MBCaptureSessionOption.Builder arguments are optional. Options description:

• numberOfFrameToCollect Describes the number of frames to collect during the capture session
• autoCaptureEnabled Tells whether the capture is automatic or manual
• timeBeforeAutomaticCapture Is the set time in seconds the capture should wait to start collecting frames The text for the countdownLabel can be set using the countdownLabelText option.
• isDebugging If it is set to false will just display the overlay on the top of the camera. If it is set to true will display all components available ont the top of the camera(timer text, progress bar and face status text).
• payloadOptimization It is use to enable or disable the optimization of the payload you send to the backend | reduces the payload size.
• cameraQuality It is used to set the options for the camera. It accepts three parameters: A MBTargetResolution instance, this is the resolution in witch the frames are collected. It accepts a MBPreviewScaleTyp instance that is the mode used to scale the camera preview in the view. It can be set to either FIT_CENTER or FILL_CENTER. MBPreviewScaleType.FILL_CENTER will ensure the camera preview is taking up the whole view in both width and height. However, this means that some parts of the image will be cropped from the preview. This does not have any impact on the images captured from the camera. MBPreviewScaleType.FIT_CENTER will ensure that exactly what is captured from the camera is shown in the view. And a MBCameraPosition that describes whether is displayed front or rear camera.

MBTargetResolution

Field	Value
QHD	Size(540, 960)
HD_720	Size(720, 1280)
HD_1080	Size(1080, 1920)
HD_4K	Size(2160, 3840)

Creates an instance of MBCaptureSessionService. This is the entry point configuration for the capture process.

 val captureSessionService = MBCaptureSessionService(
                context = this, this as LifecycleOwner,
                options, callback = this
 )

MBCaptureSessionService arguments.

• context Is an instance of the caller's context needed to implement MBCameraService and MBCaptureSessionView.
• lifecycleOwner Is an instance of the caller's life cycle owner needed to implement MBCameraService.
• options Represents the capture session's entry point configuration for ist correct functioning.
• callback Updates the caller with any change during the whole capture process, including a capture result, it has to be set to null while implementing MBCaptureSessionFragment.

Function onCaptureFinished gets the result of the capture process if it is successfully finished.

 override fun onCaptureFinished(result: MBCaptureSessionResult?) { }

• faceImage A high quality image captured after collecting frames.
• frames Is the frame collection.
• serializedData() Is the faceImage and frames in Protobuf serialization.

Function onCountdown is executed during the automatic capture. timeCounter is the current time in seconds.

  override fun onCountdown(timeCounter: Int) { }

Function onValidating is executed when frames are being analyzed.

  override fun onValidating(faceStatus: MBFaceStatus) { }

MBFaceStatus entries:

• TOO_FAR_AWAY Face is too far away
• TOO_CLOSE Face is too close
• TOO_FAR_UP Face is too far up
• TOO_FAR_LEFT Face is too far left
• TOO_FAR_DOWN Face is too far down
• TOO_FAR_RIGHTFace is too far right
• NOT_FOUND Face not found
• TOO_MANY Too many faces
• VALID Face is valid

Function onFailure is executed when the capture process fails. MBCaptureSessionError tells the type of error that occurred.

  override fun onFailure(errorEnum: MBCaptureSessionError) { }

MBCaptureSessionError entries:

• UNABLE_TO_OPEN_CAMERA Describes failure while opening the camera
• UNABLE_TO_COLLECT_FRAMES The capture has is aborted

The function onStateChanged is executed every time the capture state changes. The parameterstateEnum is a representation of the current state.

MBCaptureState

• INITIALIZING Is the first state and tell when the camera is ready.
• VALIDATING Is the second state and tell when the frames start being analyzed.
• COUNT_DOWN Is the third state and tell when a valid face is found in a single frame. If there is an invalid face in the following frames the state return to VALIDATING.
• CAPTURING CAPTURING is the fourth state and tell when the frames start being collected.If there is an invalid face in the following frames the stage return to VALIDATING.
• CAPTURE_FINISHED Is the last state and tell when the capturing is successfully finished.
• PROCESSING Is the state that tells when the captured frames are being precessed

override fun onStateChanged(stateEnum: MBCaptureState) { }

The method onCaptureProgress represents the capture progress with arguments values between 0-1.

 override fun onCaptureProgress(captureProgressCounter: Float) { }

Add the following dependencies in the build gradle app.

Enables LiveData for Jetpack Compose.

    implementation "androidx.compose.runtime:runtime-livedata:$compose_version"
    def nav_version = "2.5.2"

Enables navigation in Jetpack Compose.

    implementation "androidx.navigation:navigation-compose:$nav_version"

Coil helps to display a bitmap.

    //Coil
    implementation("io.coil-kt:coil-compose:1.4.0")

Implement a capture screen

Create the layout that contains the capture session view.

 //container.xml
 <?xml version="1.0" encoding="utf-8"?>
 <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@color/text_color_disabled"
    android:id="@+id/ll_view_container">

 </LinearLayout>

Implement the composable function that defines the capture screen/camera preview. MBCaptureSessionService instance of the entry point configuration for the capture process. MBFaceStatus gets the face status every time it changes in the camera preview. MBCaptureSessionStatus gets a capture status every time it changes.

//CaptureScreen.kt

 @SuppressLint("UnsafeOptInUsageError")
 @Composable
 fun CaptureScreen(
        captureSessionService: MBCaptureSessionService,
        faceStatus: MBFaceStatus
) {
    Surface(modifier = Modifier.fillMaxSize()) {
        AndroidView(
                factory = { context ->
                        View.inflate(context, R.layout.container, null).apply {
                            val llViewContainer = this.findViewById<LinearLayout>(R.id.ll_view_container)

                            if (captureSessionService.getCaptureSessionView().parent != null) {
                                (captureSessionService.getCaptureSessionView().parent as ViewGroup)
                                        .removeView(captureSessionService.getCaptureSessionView())
                            }
                            llViewContainer.addView(captureSessionService.getCaptureSessionView())
                            captureSessionService.startCamera()
                    }
                },
                modifier = Modifier.fillMaxSize(),
        )
       
        if (!captureSessionService.getAutomaticCapture()) {
            Box(modifier = Modifier
                    .height(200.dp)
                    .fillMaxWidth()
                    .padding(bottom = 100.dp),
                    contentAlignment = Alignment.BottomCenter) {
                Button(modifier = Modifier
                        .width(150.dp)
                        .height(50.dp),
                        onClick = {
                            captureSessionService.startCaptureSession()
                        }) {
                    Text(text = "Start Capture")
                }
            }
        }
    }
 }

Inflates camera preview based UI into a Jetpack Compose UI using AndroidView with that propose

  AndroidView(
                factory = { context ->
                        View.inflate(context, R.layout.container, null).apply {
                            val llViewContainer = this.findViewById<LinearLayout>(R.id.ll_view_container)

                            if (captureSessionService.getCaptureSessionView().parent != null) {
                                (captureSessionService.getCaptureSessionView().parent as ViewGroup)
                                        .removeView(captureSessionService.getCaptureSessionView())
                            }
                            llViewContainer.addView(captureSessionService.getCaptureSessionView())
                            
                            captureSessionService.startCamera()
                        }
                },
                modifier = Modifier.fillMaxSize(),
  )

Inflates the layout container view.

View.inflate(context, R.layout.container, null)

Create a layout container instance to be able to add a child view to it.

val llViewContainer = this.findViewById<LinearLayout>(R.id.ll_view_container)

Add the capture session view to the container ViewContainer.

if (captureSessionService.getCaptureSessionView().parent != null) {
     (captureSessionService.getCaptureSessionView().parent as ViewGroup)
      .removeView(captureSessionService.getCaptureSessionView())
}
   llViewContainer.addView(captureSessionService.getCaptureSessionView())

Start the camera if permission is granted.

 captureSessionService.startCamera()

Enable capture button if capture is manual or disable it if the capture is automatic.

     if (!captureSessionService.getAutomaticCapture()) {
            Box(modifier = Modifier
                    .height(200.dp)
                    .fillMaxWidth()
                    .padding(bottom = 100.dp),
                    contentAlignment = Alignment.BottomCenter) {
                Button(modifier = Modifier
                        .width(150.dp)
                        .height(50.dp),
                        onClick = {
                            captureSessionService.startCaptureSession()
                        }) {
                    Text(text = "Start Capture")
                }
            }
     }

Starts manual capture session if this is set to manual.

 onClick = {
    captureSessionService.startCaptureSession()
 }

Implement a screen to display the collection of frames.

The composable ImagesScreen function displays the collection of frames in a LazyVerticalGrid. The function gets a list of nullable bitmaps as parameter.

 @OptIn(ExperimentalFoundationApi::class)
 @Composable
 fun ImagesScreen(frames: List<Bitmap?>) {
    if (frames.isNotEmpty()) {
        Surface(modifier = Modifier
                .fillMaxSize()
                .padding(3.dp),
                color = Color.DarkGray) {
                
            if (frames.isNotEmpty() && !frames.contains(null)) {
                LazyVerticalGrid(
                        cells = GridCells.Fixed(2),
                        content ={
                            items(frames) { frame ->
                                Box(modifier = Modifier
                                        .width(200.dp)
                                        .height(300.dp)
                                        .padding(3.dp),
                                        contentAlignment = Alignment.Center){
                                    Image(painter = rememberImagePainter(data = frame),"Image")
                                }
                            }
                        } 
                )
            }
        }
    }
 }

ScreensRoute lists the screens contained in the example app.

 sealed class ScreensRoute(val route: String) {
   object CaptureScreen: ScreensRoute("main_screen")
   object ImagesScreen: ScreensRoute("image_screen")
 }

CaptureSessionViewModel updates and observes the different states of the MBFaceStatus by using Livedata.

 class CaptureSessionViewModel: ViewModel() {
    private val _faceStatus: MutableLiveData<MBFaceStatus> = MutableLiveData(MBFaceStatus.NOT_FOUND)
    val faceStatus: LiveData<MBFaceStatus> = _faceStatus

    fun onFaceStatusChange(faceStatus: MBFaceStatus) {
     _faceStatus.value = faceStatus
    }
 }

FrameCollectionViewModel updates and observes the collection of frames we get within the capture session result by using Livedata.

 class FrameCollectionViewModel : ViewModel(){

   private val mutableFrameCollection = MutableLiveData<List<Bitmap?>>()
   var frameCollection: LiveData<List<Bitmap?>> = mutableFrameCollection

    fun setFrameCollection(frames: ArrayList<Bitmap?>) {
    mutableFrameCollection.value = frames
   }
 }

The composable function CaptureNavigation controls the navigation between screens. Gets as parameters an instance of MBCaptureSessionService entry point configuration for the capture process, an instance of NavControllerCallback that provide a navigation controller instance to ComposeBiometric, an instance of CaptureSessionViewModel that get every change in the capture session status, and finally an instance of FrameCollectionViewModel

  @Composable
  fun CaptureNavigation(
        captureService: MBCaptureSessionService,
        provider: NavControllerCallback,
        captureSessionViewModel: CaptureSessionViewModel = viewModel(),
        frameCollectionViewModel: FrameCollectionViewModel = viewModel()
  ){
    val navController = rememberNavController()
    provider.provideController(navController)
    val faceStatus: MBFaceStatus by captureSessionViewModel.faceStatus.observeAsState(MBFaceStatus.NOT_FOUND)
    val frameCollection: List<Bitmap?> by frameCollectionViewModel.frameCollection.observeAsState(listOf())
  

    NavHost(navController = navController, startDestination = ScreensRoute.CaptureScreen.route) {
        composable(ScreensRoute.CaptureScreen.route) {
            CaptureScreen(captureSessionService = captureService, faceStatus = faceStatus)
        }
        composable(ScreensRoute.ImagesScreen.route) {
                ImagesScreen(frames = frameCollection)
        }
    }
  }

The NavController is the central API for the Navigation component. It is stateful and keeps track of the back stack of composables that make up the screens in your app and the state of each screen. Create a NavController by using the rememberNavController() method in the composable:

 val navController = rememberNavController()

Execute provideNavController, this function provides an instance of navController to BiometricCompose activity.

  provider.provideController(navController)

Gets the instance of MBFaceStatus, MBCaptureSessionStatus and frameCollection is the collection of frames. We observe in ComposeBiometric activity.

  val faceStatus: MBFaceStatus by captureSessionViewModel.faceStatus.observeAsState(MBFaceStatus.NOT_FOUND)
  val frameCollection: List<Bitmap?> by frameCollectionViewModel.frameCollection.observeAsState(listOf())

Implement the NavHost function. This Provides in place in the Compose hierarchy for self contained navigation to occur. The CaptureScreen function gets as parameters captureService that is the entry point configuration for the capture process, faceStatus is the status of the face every time it changes and captureStatus the status of the capture process every time it changes The ImageScreen function displays the collection of frames and gets the frame frameCollection as a parameter.

  NavHost(navController = navController, startDestination = ScreensRoute.CaptureScreen.route) {
        composable(ScreensRoute.CaptureScreen.route) {
            CaptureScreen(captureSessionService = captureService, faceStatus = faceStatus, captureStatus = captureStatus)
        }
        composable(ScreensRoute.ImagesScreen.route) {
                ImagesScreen(frames = frameCollection)
        }
  }

Implement FrameCollectionViewModel , CaptureSessionViewModel viewModel and NavControllerCallback in ComposeBiometric activity, as well as execute CaptureNavigation and get the NavController instance.

  class ComposeBiometric : ComponentActivity() , MBCaptureSessionServiceDelegate,NavControllerCallback {
    private lateinit var navController: NavController
    private val framesViewModel: FrameCollectionViewModel by viewModels()
    private val captureSessionViewMode: CaptureSessionViewModel by viewModels()

    @SuppressLint("UnsafeOptInUsageError")
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)


        val options = MBCaptureSessionOptions.Builder()
            .automaticCapture(true)
            .build()
        val  captureSessionService = MBCaptureSessionService(
                context = this, this as LifecycleOwner,
                options, callback = this
        )

        setContent {
            CaptureNavigation(
                    captureService = captureSessionService,
                    provider = this, captureSessionViewModel = captureSessionViewMode,
                    frameCollectionViewModel = framesViewModel
            )
        }
    }


    override fun onCaptureFinished(result: MBCaptureSessionResult?) {
        this.runOnUiThread {
            result?.let {
                framesViewModel.setFrameCollection(it.frames)
                navController.navigate(route = ScreensRoute.ImagesScreen.route)
            }
        }
    }

    override fun onCountdown(timeCounter: Int) { }


    override fun onValidating(faceStatus: MBFaceStatus) {
            this.runOnUiThread {
          captureSessionViewMode.onFaceStatusChange(faceStatus)
         }
    }



    override fun onFailure(errorEnum: MBCaptureSessionError) { }

    override fun onCaptureStarted() { }

    override fun onCaptureProgress(captureProgressCounter: Float) { }

    override fun provideController(navController: NavController) {
    this.navController = navController
    }
  }

Creates instances of NavController , FrameCollectionViewModel and CaptureSessionViewModel.

  private lateinit var navController: NavController
  private val framesViewModel: FrameCollectionViewModel by viewModels()
  private val captureSessionViewMode: CaptureSessionViewModel by viewModels()

Starts CaptureNavigation with captureSessionService , provider and the viewModes as parameters.

  setContent {
        CaptureNavigation(
            captureService = captureSessionService,
             provider = this, captureSessionViewModel = captureSessionViewMode,
              frameCollectionViewModel = framesViewModel
        )
  }

Initialize the NavController instance.

  override fun provideController(navController: NavController) {
      this.navController = navController
    }

Execution of onCaptureFinished.

  override fun onCaptureFinished(result: MBCaptureSessionResult?) {
         this.runOnUiThread {
                 result?.let {
                 framesViewModel.setFrameCollection(it.frames)
                 navController.navigate(route = ScreensRoute.ImagesScreen.route)
               }
         }

  }

Provides the collection of frames to CaptureNavigation composable.

framesViewModel.setFrameCollection(it.frames)

Navigates to ImageScreen

  navController.navigate(route = ScreensRoute.ImagesScreen.route)

Execution of onValidating.

  override fun onValidating(faceStatus: MBFaceStatus) {
    this.runOnUiThread {
        captureSessionViewMode.onFaceStatusChange(faceStatus)
    }
 }

Updates the face status in the composable CaptureNavigation.

  captureSessionViewMode.onFaceStatusChange(faceStatus)