Biometrics recognition

More and more people are using biometrics recognition to securely and quickly identify or authorize themselves on an application. This can be done through fingerprint or facial recognition, for example. Some phone makers are even already experimenting with iris scanners. Biometrics recognition is increasingly supported by apps and is being fully developed by phone makers.

Each smartphone varies which biometrics is supported, how it is integrated into the device and how secure it is. The best known and most widely used then will be the fingerprint scanner. These are integrated in different ways. For example, on the back of the device, on the power button or in the screen. Fingerprint scanners have been around for a while and this is considered a secure form of biometrics recognition (not easily bypassed). Face recognition is often done through the front camera. This form is not always secure. In some cases, it is even possible to use a photograph to bypass this. Therefore, safer methods have now been developed that, for example, project infrared dots on your face, then these dots are scanned and this includes depth and the like in the equation. This way it is safe.

Face recognition

In this case, we are going to talk about Google's Pixel 4(XL). In this phone, the fingerprint scanner has been completely scrapped and continued with the development of facial recognition. This phone uses the secure form of facial recognition as mentioned above. In addition, this phone also has Soli radar technology. This technology can recognize shapes by sending out radar waves and watching those waves come back. Thus, it can recognize that the phone is moving toward a face in order to turn on facial recognition in advance and thereby unlock the phone even faster. This works very well but in practice it turns out that the absence of the fingerprint scanner can also be a miss.

This is where the problem I am running into (or actually ran into) arises. I am the proud owner of a Pixel 4XL and am a big fan of biometrics recognition. But now it turns out that my banking app supports fingerprint but not facial recognition. This means there is only one option left, which is logging in with a PIN. Tjsa that is a bit outdated, not secure and not fast enough. For example, typing the PIN on a big screen in a public place. That just doesn't feel as comfortable because everyone can easily watch. And especially with mobile payments, the debit card is a safer option (while it could have worked so easily).

How to proceed?

Emailed the bank asking if and when support for facial recognition will be added to the app. After a long wait and hope that the next app update would support facial recognition there was still no sign that facial recognition would be added. I knew my banking app would not support this at first, but I did expect it to be built in over time. After all, the support from Android has been there for 2 years.

So if support from the bank would not be there for the time being, I had to resign myself to that. Unless there was another way I could support it myself. There is, called Accessibility. Normally this is used by people with a disability, such as the blind/ partially sighted, deaf/hard of hearing. Think of being able to enlarge texts, have texts read aloud or live subtitle videos that are talked in. As a developer, you can also create something for this yourself. Accessibility apps can help you by, among other things, clicking buttons for you. This is how my question arose:

Is it possible to use the accessibility service to add facial recognition functionality to the banking app?

The short answer is: Yes, it can!

So how exactly does it work under-the-hood?

As a proof-of-concept, I started by figuring out what can be read with the accessibility service. That's not very much, but enough to recognize things like a numpad, which you use to enter the pin. Then I looked at whether it is actually possible to press a button for the user. And investigated whether a biometric prompt (this is a system popup that recognizes your face) could be shown while the other app is visible. This all turned out to be possible, which also made it possible to start building everything. The app is recognized when opened, the numpad is searched and a biometric prompt is shown. If the facial recognition is successful that the PIN is decrypted and the appropriate buttons pressed.

How or where is the PIN stored and is it secure?

The PIN is stored in a separate app that takes care of logging into the bank app. In this app, you can store the PIN using facial recognition. The facial recognition ensures that the PIN is stored encrypted and can only be decrypted again by your biometrics.

What is your experience in using the app?

Now that I can log in with my biometrics I use the app much more often. I experience it as more secure because no one can see along with the PIN. And this certainly applies to mobile payments. Since it has been possible to make mobile payments, I always really wanted to do it myself. Unfortunately for my bank app it always worked the same as contactless payment with your debit card. So up to a certain amount without a PIN and then by entering the PIN. But in such a crowded supermarket with a line behind you where everyone can watch on your big screen, I don't think such a PIN is a safe way and that's why I didn't start using it. Because I can now enter the PIN with my biometrics I actually pay with my mobile phone more often than with my debit card.