3D Face Reconstruction with Mobile Phone Cameras for Rare Disease Diagnosis

Abstract

Computer vision technology is advancing rare disease diagnosis to address unmet needs of the more than 300 million individuals affected globally; one in three rare diseases have a known facial phenotype. 3D face model reconstruction is a key driver of these advances. However, the utility of 3D reconstruction from images obtained from mobile phone cameras has been questionable due to relatively low quality 2D data and need external calibration methods (e.g. visual markers) to extract accurate measurements. Herein a novel implementation pipeline, leveraging deep learning technologies, that can successfully reconstruct 3D face models from multiple 2D images taken by mobile phone cameras for clinician usage is described. Specifically, Multi-view Stereo (MVS) has been introduced to this application for providing a cost-effective pipeline of 3D face dense reconstruction. As a state-of-the-art MVS method, deep-learning based MVS has shown its strong generalization capability of using the low quality 2D face images to reconstruct 3D face models without camera calibration. The results demonstrate conceptual proof of a analytic pipeline to satisfy the clinician’s needs.