Application of morphological image processing to remove hair noises while enhancing patterns of near-infrared human vein images

Nội dung text: Application of morphological image processing to remove hair noises while enhancing patterns of near-infrared human vein images

1. Research APPLICATION OF MORPHOLOGICAL IMAGE PROCESSING TO REMOVE HAIR NOISES WHILE ENHANCING PATTERNS OF NEAR-INFRARED HUMAN VEIN IMAGES Pham Van Quan1, Phan Nguyen Nhue1, Ta Van Duong1, Le Duy Tuan1, Le Anh Tu2, Dao Nguyen Thuan2* Abstract: Recently, the innovative vein finder system has been studied extensively and has many practical uses in healthcare and security. Developing a better vein finder system often relies on image processing procedures which help to enhance the vein images. Conventional image processing procedures as median filtering and adaptive histogram equalization have shown benefit in enhancing vein patterns. However, in some cases when there are hairs present in the images, most of these procedures are less effective in removing noises from hairs. In this work, we present a new approach employed additional morphological image processing procedures to efficiently remove hair noises. We have successfully constructed a vein finder device to acquire vein images and demonstrate the advantage of our approach. Effects of the size and shape of the structural element in different morphological image processing steps were studied and optimized to achieve the best enhancement effect. Our approach can be applied widely to other vein finder systems and enhance vein images from various parts of the human body. Keywords: Morphological image processing; Image enhancement; Human vein images; Near infrared. 1. INTRODUCTION Infrared vein patterns in the palm or fingers already can be used for biometric purposes [1]. In medical or cosmetic work, manipulation of veins such as injection or blood transfusions is very common. Occasionally, the intravenous injection may fail and has to be repeated many times, which causes a lot of pains and temporary swelling. Therefore, a vein finder system, which can detect and visualize veins of patients on a display [2-4] or project them directly on their skin [5], is an effective solution not only to make venipuncture safe and accurate, but also to enable venous diseases diagnosis. Vein imaging technique works on a principle that near infrared radiation (NIR) is absorbed stronger by deoxyhemoglobin in venous blood than skin tissues [6]. Under NIR illumination, veins appear as dark lines while skin and fat tissues appear as lighter region, hence they are distinguishable. Developing a vein finder system using NIR techniques [7] often faces a challenge, which is the low contrast images that are generally present by light scattering. Enhancement of the contrast of vein images is possible by image processing. Recent years, typical image processing methods to enhance vein images have been used by many groups, both Vietnamese [2, 5, 8] and international [3, 9], employed common procedures including: application of median filtering, local histogram equalization following by threshold filtering and sometimes Gaussian filtering. However, in some cases, hairs on hands or legs of a human can be dense and form a noise in the NIR vein images. The black hair acts just like a black body and can absorb the full spectrum of light. Hence it is almost impossible to separate hair noises/artifacts from vein patterns in NIR vein image by changing the optical Journal of Military Science and Technology, Special Issue, No.72A, 5 - 2021 69
2. Biomedical Physics configurations of the vein finder system. The hair noises also turn up as continuous and thin lines like vein lines. Most of the conventional image processing procedures appear to be less effective in removing noises from hairs. In this work, we propose new image processing procedures using additional morphological image processing steps to minimize (or to remove completely in some cases) noise effect of hairs, while preserving the continuous vein lines. To demonstrate the advantage of our approach, we construct a near-infrared vein imaging device to acquire vein images, and apply and compare the conventional image processing procedures versus ours. 2. MATERIALS AND METHODS 2.1. A near-infrared vein imaging device Figure 1. Diagram of the working principles of our near-infrared vein imaging device (left side) and picture of the real device (right side). We have constructed a near-infrared vein imaging device, which consisted of a light source (850-nm or 950-nm LEDs), a camera to capture the vein images, a computer Raspberry Pi 4 (model B) to process the digital image and a screen to display output vein images (figure 1). The image processing program is developed in Python language with open source library OpenCV and can be loaded and run on the Raspberry Pi 4. 2.2. Conventional Image Processing Procedures Figure 2. Diagram of the typical image processing procedures to enhance vein image. Typical image processing methods to enhance vein images often composes of various procedures (figure 2). Median filtering is a nonlinear method used to remove noise from images without eliminating the edges and lines. Specifically, the median filter works by replacing each pixel of the image with the median of all pixels in a neighborhood. Local histogram equalization following by threshold filtering can be applied then. After processing, the visual contrast and the sharpness of vein images can be enhanced but one problem remains unchallenged is the noise caused by hairs. 2.3. Proposed Image Processing Procedures 70 P. V. Quan, , D. N. Thuan, “Application of morphological image human vein images.”
3. Research To minimize noise effect of hair (in some case can remove hair noises completely) in output vein images, we employ morphological image processing after adaptive histogram equalization process (figure 3). The morphological image processing steps include opening, closing and dilation algorithms. Figure 3. Diagram of our image processing procedures employed morphological image processing to enhance vein image. 2.4. Morphological Image Processing Algorithms Morphological image processing is the processing of images based on the structure and shape of the object [10]. Its language is set theory. By choosing the appropriate structural element (set B), it is possible to construct the morphology sensitive to the special shapes in the input image (set A). In our work, morphological image processing algorithms are applied for binary images. Most morphological image processing algorithm can be defined from two basic operations: dilation and erosion. The dilation of by , denoted as , is the set of all displacements z (z is the center of ’s structuring element on ), such that and overlap by at least one element. It can be defined as: { [( ) ] } (1) Meanwhile, the erosion of by , denoted as is the set of all points z, such that , translated by z, is contained in . It is defined as: { } (2) While dilation tends to expand the image’s boundaries, erosion tends to reduce these boundaries, even remove the small ones. If we perform an erosion of A by B, followed by a dilation of the result by B, we then obtain opening of A by B, denoted as (3) Similarly, if a dilation is performed first, followed by an erosion, we then obtain closing of A by B, denoted as: (4) Opening generally smoothes the contour of an object and eliminate thin protrusions, while closing tends to fill narrow gaps and smoothen lines within the contour. 3. RESULTS AND DISCUSSION 3.1. Vein images captured by our near-infrared vein imaging device and enhanced by conventional image processing procedures Journal of Military Science and Technology, Special Issue, No.72A, 5 - 2021 71
4. Biomedical Physics Firstly, we apply conventional image processing procedures used by various research group to enhance the vein image of a human’s upper hand (figure 4). Although salt-pepper noise can be reduced by median filtering and contrast of the images can be improved after processing, hairs remain and largely affect the output images. Figure 4. Vein image of a human’s upper hand after typical image processing procedures, showing hairs remain and affect output images. 3.2. Effect of the morphological image processing algorithms To minimize hairs on the vein images, we introduce a new image processing approach with employs consecutive morphological image processing steps including opening, closing and dilation algorithms. Effect of each processing step can be elucidated as follows: 3.2.1. Effect of the closing algorithm 0 0 1 0 0 1 1 1 1 1 B  1 1 1 1 1 1 1 1 1 1 0 0 1 0 0 Figure 5. Effect of the structuring element B’ size in closing algorithm on vein image of a human’s upper hand. The size of each image is (480x640) pixels. Median filtering can only remove small dot-shaped noise, while thin and long line-shaped noises are not affected. To remove these line-shape noises, we first apply closing algorithm. The size and shape of structuring element B is the most important factor in the morphological image processing method. Edged shapes like a triangle or rectangular will often break continuous lines, hence we choose an ellipse shape. We then apply closing algorithm with various sizes of structuring element B order to find the optimal one. An example of structuring element B is an ellipse with the size of (5x5) pixels is in figure 5. Figure 5 shows effect of the ellipse’s size on the closing algorithm on vein images of a human’s upper hand. It is clear that the small size of the ellipse – (1x1) or (3x3) pixels – results in little effect of hair removal. While the big size of the ellipse – (9x9) or (11x11) – pixels) not only removes hair noise, but also breaks some continuous line vein and removes some broken vein lines which are important information. Hence, we choose the size of the structuring element B is an ellipse at (5x5) size for the finest effect of hair removal while preserving important information. Applying closing algorithm helps reduce significant line-shaped 72 P. V. Quan, , D. N. Thuan, “Application of morphological image human vein images.”
5. Research noises, however, it also cut some of the thin-lined veins. A simple closing algorithm can not solve the problem and we have to use other following algorithms. 3.2.2. Effect of the opening algorithm Using the same principle for testing resulted image with different sizes of structuring element B, we derive the optimal size of the ellipse for the following algorithms. To restore the broken thin-lined vein (figure 6), we apply opening algorithm while structuring element B is an ellipse with the size of (9x9) pixels. 3.2.3. Effect of the dilation algorithm After consecutive opening and closing algorithm, some hair noises are removed. However, the vein’s lines are quite thick. To make those line thinner, we apply dilation algorithm while structuring element B is an ellipse with the size of (6x6) pixels (figure 6). Figure 6. Effect our morphological image processing procedure, including consecutive closing, opening and dilation algorithms on vein image of a human’s upper hand. 3.2.4. Effect of our new image processing procedures Figure 7. Vein image of a human’s upper hand after our new image processing procedures, showing significant hair noises (dot lines, in red) can be removed. After applying our new morphological image processing procedures, we can remove significant hair noises as shown in figure 7. Although the vein lines become thinner, all continuous vein lines in the images are maintained. Comparing with output images processed by conventional image processing procedure (figure 4), our result clearly shows the advantage of our new image processing procedures in removing hair noises while maintaining continuous vein lines. 3.3. Vein images from various part of human hands enhanced by our image processing procedures To verify effect of our new image processing approach, we have taken images Journal of Military Science and Technology, Special Issue, No.72A, 5 - 2021 73
6. Biomedical Physics from other parts of human hands including wrist (figure 8) and arm (figure 9). These images further confirm the advantage of our new image processing procedures in removing hair noises while enhancing vein lines and maintaining their borders with surrounding tissues. However, some discontinuous vein lines also are affected. With further optimization, our new image processing procedures can be widely applied to enhance vein images from various parts of the human body. Figure 8. Vein image of a human’s wrist after our new image processing procedures. Significant hair noises (dot lines, in red) can be removed while a discontinuous vein line (dash line, in blue) is also affected. Figure 9. Vein image of a human’s arm after our new image processing procedures. 4. CONCLUSIONS We have developed a new image processing approach employed morphological image processing to efficiently remove hair noises, and constructed a vein finder device to acquire vein images and compare our new procedures with conventional ones. We have studied the effects of the elements ellipse’ size in different morphological image processing steps and optimized them to achieve the best enhancement effect. We show that our new image processing procedures have overcome most conventional ones in removing hair noises, enhancing vein lines while maintaining important information of the vein images. Moreover, our approach is applicable in vein images from various part of the human hand. Thus, the principle of our approach can be applied widely to enhance images from other parts of the human body and in many other vein finder systems. Acknowledgement: This research is supported by the Vietnam’s Ministry of Science and Technology under Project ĐTĐL.CN-40/19. 74 P. V. Quan, , D. N. Thuan, “Application of morphological image human vein images.”
7. Research REFERENCES [1]. W. Wu, S. J. Elliott, S. Lin, S. Sun, and Y. Tang, in IET Biometrics (Institution of Engineering and Technology, 2020), pp. 1. [2]. K. H.-D. Nguyen, A. L.-T. Nguyen, and H. T.-T. Pham, in International Conference on the Development of Biomedical Engineering in Vietnam (Springer, 2017), pp. 559. [3]. S. Bhure and S. T. Hamde, in 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT)2019), pp. 1. [4]. L. Wang and G. Leedham, in 2006 IEEE International Conference on Video and Signal Based Surveillance (IEEE, 2006), pp. 52. [5]. T. Van Tran, H. S. Dau, D. T. Nguyen, S. Q. Huynh, and L. Q. Huynh, Science and Technology Development Journal 20, 91 (2017). [6]. E. Hemmer, A. Benayas, F. Légaré, and F. Vetrone, Nanoscale Horizons 1, 168 (2016). [7]. C.-T. Pan, M. D. Francisco, C.-K. Yen, S.-Y. Wang, and Y.-L. Shiue, Sensors 19, 3573 (2019). [8]. L. T. Tran and H. T.-T. Pham, in International Conference on the Development of Biomedical Engineering in Vietnam (Springer, 2018), pp. 383. [9]. C. Villaseđor-Mora, F. J. Sanchez-Marin, and M. E. Garay-Sevilla, Infrared Physics & Technology 51, 221 (2008). [10]. E. Dougherty, Mathematical morphology in image processing (CRC press, 2018). TĨM TẮT ỨNG DỤNG THUẬT TỐN XỬ LÝ ẢNH HÌNH THÁI ĐỂ LOẠI BỎ NHIỄU DO LƠNG GÂY RA NHẰM NÂNG CAO CHẤT LƯỢNG ẢNH TĨNH MẠCH TỪ KỸ THUẬT THU ẢNH HỒNG NGOẠI GẦN Gần đây, các thiết bị nhận ảnh tĩnh mạch đã được nghiên cứu rộng rãi và cĩ nhiều ứng dụng thực tế trong y tế và bảo mật. Một trong những hướng nghiên cứu chính phát triển thiết bị này là các thuật tốn xử lý ảnh giúp tăng cường ảnh tĩnh mạch. Một số thuật tốn xử lý ảnh như lọc trung vị và cân bằng hồnh đồ tương ứng thường được sử dụng và cho kết quả tốt. Tuy nhiên, trong một số trường hợp khi cĩ nhiều lơng ở vị trí chụp ảnh và gây ra nhiễu, thì các phương pháp này chưa thật hiệu quả trong loại bỏ nhiễu. Trong nghiên cứu này, chúng tơi đưa ra một cách tiếp cận mới sử dụng thêm thuật tốn xử lý ảnh hình thái để loại bỏ hiệu quả các nhiễu do lơng gây ra. Chúng tơi đã chế tạo thành cơng thiết bị thu ảnh tĩnh mạch và sử dụng chúng để tối ưu các bước xử lý ảnh của mình. Ảnh hưởng của kích thước, hình dạng của cấu trúc thành phần trong các bước xử lý ảnh hình thái khác nhau đã được khảo sát và tối ưu hĩa để đạt được hiệu quả tốt nhất. Cách tiếp cận của chúng tơi cĩ thể được áp dụng cho nhiều thiết bị nhận ảnh tĩnh mạch khác và nâng cao chất lượng hình ảnh tĩnh mạch chụp từ các vùng khác nhau trên cơ thể. Từ khĩa: Xử lý ảnh hình thái; Tăng cường ảnh; Ảnh tĩnh mạch người; Hồng ngoại gần. Received 31th August 2020 Revised 18th November 2020 Accepted 10th May 2021 Author affiliations: 1 Department of Optical Devices, Le Quy Don Technical University; 2 Institute of Materials Sciences (IMS), Vietnam Academy of Science and Technology. *Corresponding author: thuandn@ims.vast.ac.vn. Journal of Military Science and Technology, Special Issue, No.72A, 5 - 2021 75