|Effective removal of user-selected foreground object from facial images|
This work proposes a user-friendly automatic face de-occlusion method. We show the effectiveness of our method on five commonly occurring occluding objects which can be extended to more types of objects. Our system removes one distracting object at a time, however, it is capable of removing multiple distracting objects through repeated application very smoothly. Our model first detects the occlusion object and generates a segmentation map of the object, then uses the segmentation map as guidance information to remove the object and fill in the empty region. We have shown that integration of vanilla and partial convolution operations significantly improves performance in challenging scenarios involving the generation of content for two different segments of the face occluded by the object.
|Efficient generation of multiple sketch styles using a single network|
In real world, different artists draw sketches of a same person with different artistic styles both in texture and shape. Goal of this research is to synthesize realistic face sketches of different styles while retaining the input face identity, only using a single network. To achieve this, we employ a modified conditional GAN with a target style label as input. Given a face photo and target style label, the generator synthesizes a sketch of the face with the input target style, while the discriminator checks whether the synthesized sketch is close to the target style or not. The proposed method can learn multi-stylistic sketch synthesis only using paired datasets of single sketch style.
|Interactive removal of microphone object in facial images|
The goal of this research is to remove the microphone object in facial images. It involves detection of the microphone part, and then inpainting of the holes left behind with plausible correct contents. For the inpainting problem, we propose a novel image-to-image translation based method with “coarse-to-fine structure recovery approach” where the first inpainter network fills coarse information under the microphone part and an additional refiner network is employed to refine the inpainted area with perceptually plausible semantics. To overcome the data scarcity problem, we have created a synthetic dataset by placing microphone object in facial images from CelebA database.
|Face De-occlusion using I2I|
Face occlusion is one of the fundamental problems in the domain of computer vision. To address this problem, we propose image-to-image(I2I) translation based face de-occlusion. We strategically extract the complete facial semantic segmentation map from an occluded facial image using GAN and exploit the extracted segmentation map to complete the occluded facial image using another GAN. This approach removes the dependency of occluding object information while training the deep learning method.
|Unmasking of masked face|
This research presents a novel GAN-based network that automatically removes mask and completes the missing hole so that the completed face not only looks natural and realistic but also has consistency with the rest of the image. For this we employed a two stage network. In the first stage we detect the non-face object, i. e., mask, and generate a binary segmentation map of the object using an encoder-decoder network. We then remove the mask and complete the left behind hole in the second stage by employing a GAN-based approach that learns global coherency and deep missing semantics gradually.
|Recovery of mosaic facial images with correct attributes|
This research presents a GAN-based network that restores mosaic facial images so that the restored facial images can have not only correct semantic structure but also the same facial attributes as the ground truth. Reconstructing facial image often misses generating the attributes such as glasses, hat and gender. To address this problem, our network divides the training images into groups with the same facial attribute using a pretrained VGG-19 classifier and, trains a GAN based network that employs as many discriminators as the number of different facial attributes considered. Then, we more correctly restore the facial attributes considered.
|Exploiting the Context of Object-Action Interaction for Object Recognition|
The goal of this study is to efficiently and effectively incorporate the context of object-action interaction into object recognition in order to boost the recognition performance. We employ a few image frames that contain key poses, which can be used to distinguish human actions. Since an assemblage of key poses can take advantage of the fiducial appearance of the human body in action, representation of human actions by concatenating a few key poses, is quite effective. The main contribution of this work is the establishment of an effective Bayesian approach that exploits the probabilities of objects and actions, through random forest and multi-class AdaBoost algorithms.
|Depth edge detection|
Depth edges play a very important role in many computer vision problems because they represent object contours. We strategically project structured light and exploit distortion of the light pattern in the structured light image along depth discontinuities to reliably detect depth edges.
|Masked fake face detection|
This research presents a novel 2D feature space where real faces and masked fake faces can be effectively discriminated. We exploit the reflectance disparity based on albedo between real faces and make materials. The feature vector used consists of radiance measurements of forehead region under 850nm and 685nm illuminations. Facial skin and mask material show linearly separable distributions in the feature space proposed.
|Face Recognition Using ICA|
We propose an effective part-based local representation method named locally salient ICA (LS-ICA) method for face recognition that is robust to local distortion and partial occlusion. The LS-ICA method only employs locally salient information from important facial parts in order to maximize the benefit of applying the idea of “recognition by parts.”
|CBCT artifacts reduction|
We research a statistical image reconstruction method for X-ray computed tomography (CT) that is based on a physical model that accounts for the polyenergetic X-ray source spectrum and the measurement nonlinearities caused by energydependent attenuation. Applying this method to simulated X-ray CT measurements of objects containing both bone and soft tissue yields images with significantly reduced artifacts.