This tutorial explains everything needed to detect faces with the front facing camera on Android. It also wraps this all up into one class, so that you can do face detection in your activity with two lines of code.
At first I had no intention of writing a tutorial as I thought face detection would be easy with the Android API’s, who was I kidding! You need to obtain the camera, you are forced to show a preview of what the camera sees, set the camera up for face detection, and release the camera afterwards. That is more than the one liner I was expecting.
So this is what we are going to cover.
First things first, there is official documentation on this, but it is a bit involved you can find it here. I’ve tried to simpify this. At the end of this tutorial you should be able to detect faces with the following api:
Example Useage
FrontCameraRetriever.retrieveFor(this); // 'this' being an activity
camera.initialise(new FaceDetectionCamera.Listener() {
@Override
public void onFaceDetected() {
}
@Override
public void onFaceTimedOut() {
}
@Override
public void onFaceDetectionNonRecoverableError() {
}
});
Ta da, this will load a camera for you, set it up for face detection and give you callbacks when a face is found or lost. Lets break this down into the individual steps needed and build it back up.
First thing you need to do is retrieve the front facing camera. The docs recommend doing this off of the main thread, so lets do it in an AsyncTask. For our purposes we want the front facing camera (so we can detect the person holding the device). This is a bit convulted but basically you loop round a list of all the cameras asking each one are you front facing until you find one. Also when you “open” a camera i.e retrieve a reference to it, this can throw an exception if another application has already opened it. It can throw the exception if you yourself have opened it and forgot to close it before you request it again – been bitten by this before! So watch out.
LoadFrontCameraAsyncTask.java
/**
* manages loading the front facing camera off of the main thread
* can throw an error if no front facing camera
* or camera has not been released by a naughty app
*/
public class LoadFrontCameraAsyncTask extends AsyncTask<Void, Void, FaceDetectionCamera> {
private static final String TAG = "FDT" + LoadFrontCameraAsyncTask.class.getSimpleName();
private final Listener listener;
public LoadFrontCameraAsyncTask(Listener listener) {
this.listener = listener;
}
public void load() {
executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR);
}
@Override
protected FaceDetectionCamera doInBackground(Void... params) {
try {
int id = getFrontFacingCameraId();
Camera camera = Camera.open(id);
if (camera.getParameters().getMaxNumDetectedFaces() == 0) {
Log.e(TAG, "Face detection not supported");
return null;
}
return new FaceDetectionCamera(camera);
} catch (RuntimeException e) {
Log.e(TAG, "Likely hardware / non released camera / other app fail", e);
return null;
}
}
private int getFrontFacingCameraId() {
Camera.CameraInfo cameraInfo = new Camera.CameraInfo();
int i = 0;
for (; i < Camera.getNumberOfCameras(); i++) {
Camera.getCameraInfo(i, cameraInfo);
if (cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
break;
}
}
return i;
}
@Override
protected void onPostExecute(FaceDetectionCamera camera) {
super.onPostExecute(camera);
if (camera == null) {
listener.onFailedToLoadFaceDetectionCamera();
} else {
listener.onLoaded(camera);
}
}
public interface Listener {
void onLoaded(FaceDetectionCamera camera);
void onFailedToLoadFaceDetectionCamera();
}
}
Once you have retrieved the front facing camera, you need to turn on face detection. Luckily the detecting of faces itself is done by the framework and you just need to add a listener for faces. see FaceDetectionListener. However to detect faces the camera needs to be showing the “preview” i.e. turned on and to turn on the camera you need a surface to show what the camera see’s, but we don’t want to show what the camera see’s we just want to find faces! Therefore we add a dummy surface to our camera, if you don’t do this the camera will not work. You also add a listener for faces to the camera, I will come back to this. Here’s how you set up a camera for face detection:
try {
camera.stopPreview();
} catch (Exception swallow) {
// ignore: tried to stop a non-existent preview
}
try {
camera.setPreviewDisplay(new DummySurfaceHolder());
camera.startPreview();
camera.setFaceDetectionListener(new OneShotFaceDetectionListener(this));
camera.startFaceDetection();
} catch (IOException e) {
// oops
}
The DummySurfaceHolder implements the SurfaceHolder interface but does not do anything with the information that is passed to it. This is how we get away with using the camera but not having a preview of the camera image on the screen. If you did want to show what the camera could see, you would need to change this.
DummySurfaceHolder.java
/**
* This is a dummy surface,
* i.e. it can be used as a surface but doesn't actually do anything
* handy if you have an api that requires drawing to the screen but you don't want to see anything ;-)
*/
class DummySurfaceHolder implements SurfaceHolder {
private static final int MAGIC_NUMBER = 1;
@Override
public Surface getSurface() {
return new Surface(new SurfaceTexture(MAGIC_NUMBER));
}
@Override
public void addCallback(Callback callback) {
// do nothing
}
@Override
public void removeCallback(Callback callback) {
// do nothing
}
@Override
public boolean isCreating() {
return false;
}
@Override
public void setType(int type) {
// do nothing
}
@Override
public void setFixedSize(int width, int height) {
// do nothing
}
@Override
public void setSizeFromLayout() {
// do nothing
}
@Override
public void setFormat(int format) {
// do nothing
}
@Override
public void setKeepScreenOn(boolean screenOn) {
// do nothing
}
@Override
public Canvas lockCanvas() {
return null;
}
@Override
public Canvas lockCanvas(Rect dirty) {
return null;
}
@Override
public void unlockCanvasAndPost(Canvas canvas) {
// do nothing
}
@Override
public Rect getSurfaceFrame() {
return null;
}
}
After we setup the preview (dummy preview) we add the listener for faces being detected. Now the android api for this will give you multiple callbacks when it finds a face i.e. if there is a face detected by the camera it will keep firing the callback until that face is gone: face, face, face, face, face, face, face, face, face, face, face. This is not what we want and so I’ve implemented a listener that changes the callbacks so that we get one callback when a face is detected and one callback when the face is gone.
How this “one shot” face detection listener works is, it will receive the multiple callbacks from Android. When the first callback is received it passes this on to anyone else listening and starts a timer. If another face detection event is not received in this time then the face gone event is fired. However if the face is still present it keeps resetting the timer until it is gone. This is how we get a “one shot”: face here, face gone.
There are two facets to this listener, first is the actual listener to manage the Android callbacks.
OneShotFaceDetectionListener.java
/**
* Manage the android face detection callbacks to be more ON, OFF than real time ON ON ON ON OFF
*/
public class OneShotFaceDetectionListener implements Camera.FaceDetectionListener {
private static final int UPDATE_SPEED = 100;
private static final int UPDATE_SPEED_UNITS = 1000;
private final Listener listener;
private boolean timerComplete = true;
OneShotFaceDetectionListener(Listener listener) {
this.listener = listener;
}
/**
* The Android API call's this method over and over when a face is detected
* the idea here is that we de-bounce all these calls, so that we get 1 callback when
* a face is detected and 1 callback when it is lost
* <p/>
* i.e.
* face, face, face, face, face, no face, face, face, face
* becomes
* face, no face, face
*/
@Override
public void onFaceDetection(Camera.Face[] faces, Camera camera) {
if (faces.length == 0) {
return;
}
tickFaceDetectionSession();
if (sameFaceDetectionSession()) {
return;
}
startFaceDetectionSession();
listener.onFaceDetected();
}
private RestartingCountDownTimer tickFaceDetectionSession() {
return timer.startOrRestart();
}
private boolean sameFaceDetectionSession() {
return !timerComplete;
}
private void startFaceDetectionSession() {
timerComplete = false;
}
private RestartingCountDownTimer timer = new RestartingCountDownTimer(UPDATE_SPEED, UPDATE_SPEED_UNITS) {
@Override
public void onFinish() {
completeFaceDetectionSession();
listener.onFaceTimedOut();
}
};
private void completeFaceDetectionSession() {
timerComplete = true;
}
interface Listener {
void onFaceDetected();
void onFaceTimedOut();
}
}
And second is the timer that allows us to have face gone callbacks. This timer is a modification of the Android CountdownTimer. The modification being, when someone attempts to start the timer for a second time, it will reset the current countdown. i.e. say you are counting down from 30 and it gets to 15, then someone says start counting again. It will reset the 15 back up to 30 (rather than let the 15 continue and start another timer from 30).
RestartingCountDownTimer.java
/**
* MODIFICATION - restarts the timer if you call start twice (original would queue them up)
* https://developer.android.com/reference/android/os/CountDownTimer.html
* <p/>
* Schedule a countdown until a time in the future, with
* regular notifications on intervals along the way.
* <p/>
* Example of showing a 30 second countdown in a text field:
* <p/>
* <pre class="prettyprint">
* new CountDownTimer(30000, 1000) {
* <p/>
* public void onTick(long millisUntilFinished) {
* mTextField.setText("seconds remaining: " + millisUntilFinished / 1000);
* }
* <p/>
* public void onFinish() {
* mTextField.setText("done!");
* }
* }.startOrRestart();
* </pre>
* <p/>
* The calls to {@link #onTick(long)} are synchronized to this object so that
* one call to {@link #onTick(long)} won't ever occur before the previous
* callback is complete. This is only relevant when the implementation of
* {@link #onTick(long)} takes an amount of time to execute that is significant
* compared to the countdown interval.
*/
abstract class RestartingCountDownTimer {
/**
* Millis since epoch when alarm should stop.
*/
private final long mMillisInFuture;
/**
* The interval in millis that the user receives callbacks
*/
private final long mCountdownInterval;
private long mStopTimeInFuture;
/**
* @param millisInFuture The number of millis in the future from the call
* to {@link #startOrRestart()} until the countdown is done and {@link #onFinish()}
* is called.
* @param countDownInterval The interval along the way to receive
* {@link #onTick(long)} callbacks.
*/
RestartingCountDownTimer(long millisInFuture, long countDownInterval) {
mMillisInFuture = millisInFuture;
mCountdownInterval = countDownInterval;
}
/**
* Cancel the countdown.
*/
public final void cancel() {
mHandler.removeMessages(MSG);
}
/**
* Start the countdown, If it is already started it will reset the time and start again.
*/
public synchronized final RestartingCountDownTimer startOrRestart() {
if (mMillisInFuture <= 0) {
onFinish();
return this;
}
mHandler.removeMessages(MSG);
mStopTimeInFuture = SystemClock.elapsedRealtime() + mMillisInFuture;
mHandler.sendMessage(mHandler.obtainMessage(MSG));
return this;
}
/**
* Callback fired on regular interval.
*
* @param millisUntilFinished The amount of time until finished.
*/
public void onTick(long millisUntilFinished) {
// override if needed
}
/**
* Callback fired when the time is up.
*/
public abstract void onFinish();
private static final int MSG = 1;
// handles counting down
private Handler mHandler = new Handler(Looper.getMainLooper()) {
@Override
public void handleMessage(Message msg) {
synchronized (RestartingCountDownTimer.this) {
final long millisLeft = mStopTimeInFuture - SystemClock.elapsedRealtime();
if (millisLeft <= 0) {
onFinish();
} else if (millisLeft < mCountdownInterval) {
// no tick, just delay until done
sendMessageDelayed(obtainMessage(MSG), millisLeft);
} else {
long lastTickStart = SystemClock.elapsedRealtime();
onTick(millisLeft);
// take into account user's onTick taking time to execute
long delay = lastTickStart + mCountdownInterval - SystemClock.elapsedRealtime();
// special case: user's onTick took more than interval to
// complete, skip to next interval
while (delay < 0) delay += mCountdownInterval;
sendMessageDelayed(obtainMessage(MSG), delay);
}
}
}
};
}
Now that the camera is setup for face detection, the last thing to do is make sure you release the camera when you are finished with it. This is what I was saying at the beginning if you do not release the camera, other people (including yourself) won’t be able to use it again. This is usually done in onPause.
@Override
public void onPause() {
super.onPause();
if (camera != null) {
camera.release();
}
}
Lastly now that you have used OneShotFaceDetectionListener you need to implement the interface to receive the face detection callbacks.
@Override
public void onFaceDetected() {
helloWorldTextView.setText("I SEE YOU");
}
@Override
public void onFaceTimedOut() {
helloWorldTextView.setText("WHERE HAVE YOU GONE?");
}
@Override
public void onFaceDetectionNonRecoverableError() {
// This can happen if
// Face detection not supported on this device
// Something went wrong in the Android api
// or our app or another app failed to release the camera properly
helloWorldTextView.setText("UH OH");
}
Always remember to add the permissions to your manifest to use the camera. Also adding uses-feature allows the playstore to filter out devices that don’t have a front facing camera (that would be pointless and lead to bad reviews).
AndroidManifest.xml
<uses-feature android:name="android.hardware.camera" /> <uses-feature android:name="android.hardware.camera.front" /> <uses-permission android:name="android.permission.CAMERA" />
And that’s it face detection done! As you can see it’s not too simple and there is a lot of managing of the camera that is needed. So now that you understand it all FORGET IT. And use my wonderful helper class below. This magical class will do everything that I have explained above for you and all you need to do is add a listener for the face detection callbacks. Wonderful!
FrontCameraRetriever.java
/**
* I manage loading and destroying the camera reference for you
*/
public class FrontCameraRetriever implements Application.ActivityLifecycleCallbacks, LoadFrontCameraAsyncTask.Listener {
private final Listener listener;
private FaceDetectionCamera camera;
public static void retrieveFor(Activity activity) {
if (!(activity instanceof Listener)) {
throw new IllegalStateException("Your activity needs to implement FrontCameraRetriever.Listener");
}
Listener listener = (Listener) activity;
retrieve(activity, listener);
}
private static void retrieve(Context context, Listener listener) {
Application application = (Application) context.getApplicationContext();
FrontCameraRetriever frontCameraRetriever = new FrontCameraRetriever(listener);
application.registerActivityLifecycleCallbacks(frontCameraRetriever);
}
FrontCameraRetriever(Listener listener) {
this.listener = listener;
}
@Override
public void onActivityCreated(Activity activity, Bundle savedInstanceState) {
// not used
}
@Override
public void onActivityStarted(Activity activity) {
// not used
}
@Override
public void onActivityResumed(Activity activity) {
new LoadFrontCameraAsyncTask(this).load();
}
@Override
public void onLoaded(FaceDetectionCamera camera) {
this.camera = camera;
listener.onLoaded(camera);
}
@Override
public void onFailedToLoadFaceDetectionCamera() {
listener.onFailedToLoadFaceDetectionCamera();
}
@Override
public void onActivityPaused(Activity activity) {
if (camera != null) {
camera.recycle();
}
}
@Override
public void onActivityStopped(Activity activity) {
// not used
}
@Override
public void onActivitySaveInstanceState(Activity activity, Bundle outState) {
// not used
}
@Override
public void onActivityDestroyed(Activity activity) {
activity.getApplication().unregisterActivityLifecycleCallbacks(this);
}
public interface Listener extends LoadFrontCameraAsyncTask.Listener {
}
}
No need to worry about recycling the camera with the above, it does that for you.
An example use of this class will probably shed the most light on it for you, here it is, front facing camera face detection in an Activity. Two lines of code? WIN
MainActivity.java
/**
* Don't forget to add the permissions to the AndroidManifest.xml!
* <p/>
* <uses-feature android:name="android.hardware.camera" />
* <uses-feature android:name="android.hardware.camera.front" />
* <p/>
* <uses-permission android:name="android.permission.CAMERA" />
*/
public class MainActivity extends Activity implements FrontCameraRetriever.Listener, FaceDetectionCamera.Listener {
private static final String TAG = "FDT" + MainActivity.class.getSimpleName();
private TextView helloWorldTextView;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
helloWorldTextView = (TextView) findViewById(R.id.helloWorldTextView);
// Go get the front facing camera of the device
// best practice is to do this asynchronously
FrontCameraRetriever.retrieveFor(this);
}
@Override
public void onLoaded(FaceDetectionCamera camera) {
// When the front facing camera has been retrieved
// then initialise it i.e turn face detection on
camera.initialise(this);
// If you wanted to show a preview of what the camera can see
// here is where you would do it
}
@Override
public void onFailedToLoadFaceDetectionCamera() {
// This can happen if
// there is no front facing camera
// or another app is using the camera
// or our app or another app failed to release the camera properly
Log.wtf(TAG, "Failed to load camera, what went wrong?");
helloWorldTextView.setText(R.string.error_with_face_detection);
}
@Override
public void onFaceDetected() {
helloWorldTextView.setText(R.string.face_detected_message);
}
@Override
public void onFaceTimedOut() {
helloWorldTextView.setText(R.string.face_detected_then_lost_message);
}
@Override
public void onFaceDetectionNonRecoverableError() {
// This can happen if
// Face detection not supported on this device
// Something went wrong in the Android api
// or our app or another app failed to release the camera properly
helloWorldTextView.setText(R.string.error_with_face_detection);
}
}
One thing to note here, is the implementing of onLoaded(FaceDetectionCamera camera) obviously I could have hidden this as well, but I have highlighted it because if you DID want to show a preview of the camera, this is where you could do it. I can add a tutorial for that if people really want it.
All the source code is available here on github don’t you worry. https://github.com/blundell/FaceDetectionTutorial
Questions, comments just ask.
What is FaceDetectionCamera? This is not an Android class so are you implementing it? Please share its source code.
hi, source is here: https://github.com/blundell/FaceDetectionTutorial/blob/master/app/src/main/java/com/blundell/tutorial/cam/FaceDetectionCamera.java