Image Sketch Based Criminal Face Recognition Using Content Based Image Retrieval

Purpose : Face recognition is a geometric space recording activity that allows it to be used to distinguish the features of a face. Therefore, facial recognition can be used to identify ID cards, ATM card PINs, search for one’s committed crimes, terrorists, and other criminals whose faces were not caught by Close-Circuit Television (CCTV). Based on the face image database and by applying the Content-Base Image Retrieval method (CBIR), committed crimes can be recognized on his face. Moreover, the image segmentation technique was carried out before CBIR was applied. This work tried to recognize an individual who committed crimes based on his or her face by using sketch facial images as a query. Methods: We used an image sketch as a query because CCTV could not have caught the face image. The research used no less than 1,000 facial images were carried out, both normal as well as abnormal faces (with obstacles). Findings: Experiments demonstrated good enough in terms of precision and recall, which are 0,8 and 0,3 respectively, which is better than at least two previous works. The work demonstrates a precision of 80% which means retrieval of effectiveness is good enough. The 75 queries were carried out in this work to compute the precision and recall of image retrieval. Novelty: Most face recognition researchers using CBIR employed an image as a query. Furthermore, previous work still rarely applied image segmentation as well as CBIR .


INTRODUCTION
The development of face recognition is quite difficult because human faces are very complex, multidimensional, and often change according to changes in a person's environment and mental condition [1]. Therefore, according to [2] [3] the automatic facial recognition system is a challenge for experts. There are changes in facial conditions such as changes in facial identity and facial variations that occurred due to different irradiation and angle of taking facial images, which become a challenge in itself how to represent faces for facial recognition purposes. How to or techniques that can be used to accurately represent a face under varying conditions.
Making an application or tool that is very urgent and important can be used to assist law enforcement, such as determining the faces of suspected perpetrators of crime. With a tool that can automatically provide or display faces that are suspected of being by the desired query. However, in many cases, the desired photos were unavailable or not in the police database. To overcome this, law enforcers usually ask for help from people who are good at painting, especially face painters, to make sketches of the faces of the criminals based on several eyewitnesses who had seen the perpetrators of crimes even if only briefly [4]. According to [4] [5] [6] the search for a face image using a sketch of a painter is very helpful in finding faces of criminals and allows law enforcers to determine suspected persons or groups. Besides, a sketch like this is expected to help painters and eyewitnesses to improve the sketch made based on the face retrieved. According to [7] [8], the problem is how to sketch a picture or face image that looks real or like a photo of someone. In this research, we will try how to find or recognize the faces of criminals based on sketches and how to build photos or images of real faces based on sketches.
This study aims to build an application system or tools that can be used automatically for facial recognition of criminals based on a sketch made by a painter or someone trained to sketch faces. In addition to facial recognition applications, it can also be used for other purposes such as searching for certain species in the fields of biology and zoology and can be used to detect diseases in both animals and humans. Applications to be built are applications that can be operated on Windows platforms such as Linux and their families. The results of this research activity are expected to produce an alternative method as well as an application system or tool for criminal face recognition. The application system is expected to help accelerate suspected criminals' recognition based only on a witness. The application system is expected to run on dual platforms, both Windows and open source

Face Image Data Collection
In this research, around 1,000 facial images with various poses or positions and various brightness were used. The database built consists of images of normal faces (facing perpendicular or 90 degrees to the face), and images of faces that are not normal or with obstacles, such as faces wearing hats, glasses, and faces covered with a mask or veil. Moreover, the database includes 700 unsuspected criminal faces and 300 criminal faces.

Sketch Face Recognition
Several works were carried out by [9] [7] [10] [11] [12] and they stated that face sketch recognition in this study will be based on the matching between the sketch and the image in the database. For this reason, in this study, we will use the Principle Component Analysis (PCA) and Bayesian Classifier techniques or methods for facial recognition. To make this method simple, let us suppose that G ε RN1 and I ε RN2 are shapes and texture vectors, respectively, where N1 and N2 are the vector lengths for shapes and textures. The vector features used in this technique are almost the same as the features used for photo-based recognition in the shape model. After that, the Eigenfaces for the shapes and textures are calculated from the training sketch. In PCA Classifier, feature vectors are projected onto Eigefaces to obtain features with small dimensions shown in equations (1) and (2).
Where EG and EI are the Eigenvector matrices of shape and texture, meanwhile mG and mI are an average of shape and texture.

Face Image Engineering
After the stage of generating facial images based on sketches, then to look for similarity we use Eigenfaces calculations, the Eigenfaces method or approach is carried out for extracting vector features which we will later use to classify and recognize faces. Wan and Khastavaneh, [13] [14] [15] introduce simple terms Eigenfaces by the following procedure: i). First, it is assumed that the training set images are I1, I2, I3, ..., In. Where each face image has dimensions I (x, y), then each face image is converted into a vector that has a matrix (m x p), and m is the number of the exercise face image while p is p x y. ii). After that, calculate the mean or average of the face matrix. iii). After that, calculate each face matrix with its mean. iv). After that, the matrix transformation is done, so that the matrix-vector will be reduced. v). Then the eigenvectors and eigenvalues are calculated so that based on these Eigenfaces each image will have a face vector, vi). Finally, the image of the face can be engineered or formed with the respective vector and the previous vectors.

Determination of Face Similarity
To calculate the similarity (similarity) between the query sketches and the face images in the database, it is necessary to have a fairly expensive cost in terms of complex algorithms and a long process [15] [7] [16]. Furthermore, [16] [17] [18] stated that to solve this problem, in their study, there are three stages of matching face images were carried out. First looking for the topological similarity of the query face with the face in the database which is a filter, secondly the use of information to fix the candidate face image that will be called, and finally the matching calculation method is done to determine the similarity between the sketch query and the face image in the database. The recognition of normal and non-normal faces (with obstacles) that we worked on in this research can be explained in Figure 1 and Figure 2.     We used the use case diagram tool to design an application system to evaluate retrieval performance, the diagram can be illustrated in Figure 5. The experiments can be demonstrated well enough in terms of precision and recall, which are 0,8 and 0,3 respectively. The work demonstrates a precision of 80% which means retrieval of effectivity has good enough. The 75 queries were carried out in this work and detail of precision and recall of image retrieval can be examined in Table 1. From 75 queries make in this work 22 queries produce excellent precision of100%, whilst only less than 10 queries show the precision of equal or less than 50%. In this paper we use F-score with β = 0.5 to highlight precision, moreover, we get F-score from the experiment of 0,6. The highest precision of 75 queries is 1 or 100%, whilst the lowest is 0,3 or 30%. Compare to [20] [24] which is 71, our works look better in terms of average precision which is more than 80% as shown in Table 1.

CONCLUSION
From the results and discussion of this research, it can be concluded that our algorithm is quite good to recognize suspected criminals using image sketches. The effectiveness of retrieval has above average which is 80% in terms of precision. In the near future, research should be carried out by using other methods such as segmentation, pattern recognition, or Support Vector Machine and using a bigger image database.