A Study on the Shutter Time of a Surveillance Camera to Improve Speed Detection Accuracy of Vehicles on Highways and Inner-City Streets

Nội dung text: A Study on the Shutter Time of a Surveillance Camera to Improve Speed Detection Accuracy of Vehicles on Highways and Inner-City Streets

1. JST: Smart Systems and Devices Volume 31, Issue 2, September 2021, 043-050 A Study on the Shutter Time of a Surveillance Camera to Improve Speed Detection Accuracy of Vehicles on Highways and Inner-City Streets Nguyen Thi Thu Hien, Nguyen Viet Hung, Nguyen Tien Dzung* School of Electronics and Telecommunications, Hanoi University of Science and Technology, Hanoi, Vietnam *Email: dzung.nguyentien@hust.edu.vn Abstract Detection of vehicle speed based on image processing technology recently has been found in many applications over the world. However, the accuracy of those methods has not been investigated taking into account of physical characteristics of the surveillance camera. Based on the operation time of the camera's optical sensor system including shutter time (ST) and sensor operating time, the accuracy of vehicle speed detection on highways as well as inner city streets can be significantly improved. The operation time of the camera’s sensor is essential for determination of frames over time in a vehicle surveillance system. Therefore, control of the shutter time ST will help a camera-based speed detection system to achieve much better accuracy. Keywords: Vehicle speed detection, surveillance camera, image and video processing, shutter time. 1. Introduction1 method with an appearance-based method. However, the results are not achieving satisfaction since the An intelligent transportation system includes background is quite complicated. many functions for traffic management. A problem of vehicle speed detection utilized image processing Video image processing-based speed detection technologies has been studied for a long time as basic focuses on two main directions. The first one utilizes a studies [1] besides sensors-based solutions [2] as well surveillance camera for speed detection [4,11,14] as a combination between the two approaches. dealing only with the capture of a video stream and Traditionally, vehicle speed detection or surveillance detection of vehicle speed without considering the is obtained using radar technology, particularly, radar operating time of the camera. The second direction detector and radar gun. However, this method still has instead uses two cameras [5], which typically are several disadvantages such as the cosine error stereo or dual cameras to estimate the distance from a happening when the direction of the radar gun is not vehicle to the camera. Since the detected speed on the direct path of the incoming vehicle. From this accuracy is strongly dependent on the moving speed of study, it is known that vehicle detection is the main vehicles on highways and therefore on opening and thing to understand on traffic condition reasoning, a closing ST of the cameras, this paper will deal with starting point of understanding the traffic conditions investigation of the influence of ST on the speed based on the extraction of data from image processing. detection problem in a typical highway and mixed The improvement of image tracking will produce more traffic system likely Vietnam. In [6] the authors accurate data for further implementation in traffic already have studied geometrical modeling to provide conditions through reasoning, such as accidents or a solution to detect vehicle speed in urban areas in broken cars. According to [3], the authors apply these Vietnam, where vehicles move at an average range of methods on the virtual line based on various time- speed, but not applicable to ones moving at higher spatial images (TSIs) that are retrieved from multiple speeds. Therefore, this study may be utilized as a virtual detector (MVDL) lines of vehicle video frames. premise to expand widespread applications of modern This method may be effective in implementing the image processing techniques in vehicle speed autonomous system of transportation, but not well detection in highways as well as in inner city streets in capable of handling the traffic complexity conditions, Vietnam with a typical mixed traffic flow, where which is typical in Vietnam with a mixed flow of different means like cars, vans, buses, motorbikes, etc. motorbikes and cars in inner city streets. On the other both joining transportation. hand, Lin et. al [3] has been proposed some of the To demonstrate the performance of the proposed techniques in detecting various vehicles at the areas method, the rest of this paper is organized as follows. that cover with blind spot by unifying the edge-based Section 2 briefly introduces the system modelling of a ISSN 2734-9373 Received: August 19, 2020; accepted: January 27, 2021 43
2. JST: Smart Systems and Devices Volume 31, Issue 2, September 2021, 043-050 surveillance camera in traffic monitoring and the effect road traffic and what characteristics are involved in related ST considerations of the camera. Section 3 that has to be determined. As shown in Fig. 1, the discusses the experimental results and performance camera is set at the height of h above the road surface evaluation. Finally, Section 4 deals with the with its optical axis sloped at an angle δ from the road. conclusion and future works of this paper. The relation between camera lens angle and the view domain covered by camera can be determined using 2. System Modelling frequent geometrical equations. In short, the camera 2.1. Geometric Model of a Camera calibration parameters are set as follows: The essence of a camera is its lens system [7,13]. 1) Camera elevation angle range is α towards Fig. 1 demonstrates the position of an object according the vertical axis and α + δ is the maximum adjustable to the incident ray passing through O which is the elevation angle of this camera; center of the lens. The image obtained on the screen reflects the object thanks to the optical phenomena in 2) An obtained image I is at resolution is m × n the lens which is real and inverted to the object. The pixels; closer the object is to the viewfinder, the larger size it 3) The camera is located at the height h evaluated has. This viewfinder receives lighting source passing from the road surface; through a lens system of the camera, and then an image reflecting the object will be created on the screen at the focal length f. The pixel position of a reflected point from the object into the image plan can be evaluated from the object’s coordinates in 3D space as in [5]: X Y if= and jf= (1) Z Z where, X, Y, and Z are the pixel coordinates in the OXYZ space; f is the focal length of the camera lens system. Fig. 1. A model of a surveillance camera calibration The camera location must be set up over the mounted on roads. surface of the road with its optical axis inclined Assuming the camera's viewing angle along downward to the roadway to cover the road plane. Ox-axis direction intersects the road surface at point C, Since the proposed solution focuses on speed detection which is always set up as a center of the captured of vehicles on both highways and inner city streets image since point C corresponds to the center of the with a mixed traffic flow of motorcycles, cars, vans camera's image sensor. Point L is the closest position and other means, there is the need to identify each type on the road where the camera can capture. Taking into of means out of each other in multiple road lanes. account those parameters in this model setup, the Therefore, this paper utilizes the direction angle (DA) distance from the camera to the object can be of the first primary axis (FPA) for each coming vehicle essentially evaluated. detected in the video sequence and captured by the surveillance camera mounted on the roads [6]. The If O’ is assumed to be the center of the camera’s background subtraction is first performed for the image sensor, angle O'OA between O, O’ and a given captured frame sequence and then each vehicle is pixel A(i,j) on the image plan I of size m × n may be located and the DA of FPA is evaluated for decision determined as: making in identification task [8]. 22 mn The study in [6] modelled the location of the ij− +− 22 surveillance camera installation on the road as shown OO′ A = tan (2) in Fig. 1 below. Camera calibration is one of the f important aspects of the study. The vehicles’ location Utilizing the properties of circle, rectangles and in video images is 2-D (dimension), however, the trigonometry, all pixels in the image plan I will be vehicles in real world are 3-D, but because vehicles determined corresponding the angle range scanned by cannot leave the road surface, vehicles’ motion is also the camera. in 2-D which makes the transformation of image coordinates and vehicles’ coordinates a 2D-to-2D Since this problem focuses only on evaluation of mapping that can be precisely formulated. In this vehicle speed, a vehicle is assumed to move in a section, the calculation of the pattern function between straight direction, and therefore only the pixels along vehicle coordinates in the image and real-world the horizontal frame boundary are the target under coordinates is performed. How the video camera is consideration to determine the movement distance of installed when the video images are captured from the the vehicle followed by estimation of the vehicle 44
3. JST: Smart Systems and Devices Volume 31, Issue 2, September 2021, 043-050 speed. Let denote ∆p the size of one pixel on the image foreground or background and then is sorted based on sensor, its representation can be written as: the weights ωit, . The data herein that relate to the ith f . tan (δ ) Gaussian variable at time t in this combination and ∆=p (3) m the covariance function ∑it, of this Gauss variable has 2 the form σ 2 I . From the known angles α and δ, the position of k pixel A(i,j) on the resulting image I at the distance In order to improve the efficiency of vehicle from the centre of OO’ can be found as follows: identification through a surveillance camera system, m If pixel Ai(, j ) for i ≥ then: this paper utilizes the proposed method in [10] 2 combined with investigation of incidence angle of m each incoming vehicle and mapped into the predefined (ip−∆) =αδ +−2 database to identify the vehicles instead determination dh.tan arctan  (4) f of the vehicle sizes. Based on identification results, the  localization of the vehicles and their licence plates will m be detected and bounded by boxes, and the And if pixel Ai(, j ) for i < then: 2 corresponding centroids will be utilized in speed detection step on either highway with allowed vehicles ip∆ =αδ ++ or on inner city streets with any types of vehicles. dh.tan arctan  (5) f 2.2. Vehicle Identification In this important step, the background subtraction method is investigated to find the difference between the images or sequence video frames, followed by identification of moving objects in the video frames (a) afterward. Background subtraction is a widely used approach for detecting moving objects in videos from static cameras [8,9,12]. The rationale in the approach is that of detecting the moving objects from the difference between the current frame and a reference (b) frame, often called the background image or Fig. 2. A frame captured from surveillance camera and background model. As a basic, the background image its background extraction. a) On a highway. b) On a must be a representation of the scene with no moving street objects and must be kept regularly updated so as to adapt to the varying luminaries’ conditions and 2.3. A Video Frame geometry settings. Therefore, binarization should be The surveillance camera operates at 12 frames the very first step in the background and foreground per second (fps). A frame map example captured from separation process, which are road surface and the camera in this system is given in Fig. 3. vehicles, respectively as illustrated in Fig. 2. In [8,9], Stauffer and Grimson describe the 1 2 3 4 5 6 7 8 9 10 11 12 1 probability as observation of a pixel x at time t s t t t 12 wait 1 shutter wait 2 within a given image I is the average of a multivariable mixed Gaussian model that represents color values of red, green and blue respectively as Frames in a second presented in formula 6. Herein it is assumed that these 2 values are independent and have the same variance σ K : K Px()t =∑∑ωηit,, ( x t − µ it , ) (6) i=1,it Here, η denotes the probability density function, t t t and K the number of Gaussian variables in each wait 1 shutter wait 2 Gaussian distribution, and in fact, it is set to the value between 3 and 5. Each of these distributions describes only one of the objects belonging either to the Fig.3. Representation example of a frame map captured from the surveillance camera operating at 12 fps. 45
4. JST: Smart Systems and Devices Volume 31, Issue 2, September 2021, 043-050 The interval between two consecutive frames can horizontal to vertical view. In fact, each of these be easily determined as 1/12 second. While a vehicle parameters can be investigated to detect vehicle speed coming from a distance toward the surveillance in the traffic flow. camera, the vehicle displacement ∆s in the time interval ∆t is estimated by utilization of the consecutive frames captured within ∆t . The actual average speed v of the vehicle is then determined from the following formula: ∆S v = (7) ∆t According to the method presented in [6], only (a) the nearest pixels to the camera which belong to the vehicle region will be considered to estimate the centroid of the vehicle object. Fig. 4 demonstrates a frame extracted from a video sequence which is recorded on Phap Van – Cau Gie Expressway. Estimation of the displacement of the centroid for a time interval attached with the vehicle object appearing in the consequent frames will be utilized to approximate the vehicle speed. 2.4. Shutter Opening and Closing Time (b) Fig. 4. An example of a frame extracted from the This session deals with the investigation of the surveillance video sequence in database. a) Captured vehicle centroid to estimate the displacement of the on a highway. b) Captured on a street vehicle from the localized license plate as well as vehicle's region of interest (ROI). In addition, the traffic density is also considered in this study to verify the proposed solution. As shown in Fig. 4, the centroid of the license plate and that of vehicle object detected are almost duplicated. Therefore, the centroid positions are useful to be served in the speed measurement process, where vehicle’s centroid is suitable for highway and license-plate’s centroid is effective for streets, especially with crowded traffic because vehicles are partially hidden in the traffic crowd and not easy to be fully extracted and then localized. Fig. 5 shows the relationship between the centroid of the detected vehicle and bounded by a Fig. 5. Statistic of centroids of license plates and that rectangular box and that of the localized license plate. of vehicle objects detected Utilization of this data is helpful to investigate the vehicle-related information while it moves across the sensor quadrants as shown in Fig. 6. Of course, this matter is also partially dependent on the elevation angle of the mounted surveillance camera. If there is a mismatch of the centroid info observed in the sensor quadrants in Fig. 6, it means that the vehicle is approaching from afar toward the camera position. Whenever vehicles are viewed by the camera from a distance, the angled slope as described in [10] is investigated to classify vehicles in the very first captures if the rectangular box where the vehicle is surrounded by is detected horizontally. However, if a vehicle approaches from afar to a given distance, it may pass through the centre of the camera sensor Fig. 6. Magnified image sensor structure system, and the detected box may be rotated from the 46
5. JST: Smart Systems and Devices Volume 31, Issue 2, September 2021, 043-050 Let consider the effect of the opening and closing Table 1. Surveillance camera setup and its parameters shooter time (ST), i.e. exposure time on the stated Camera mounted height relative to problem of vehicle speed detection as follows. 6.15 m the road surface Highway system in Vietnam entirely allows speed up 78.7 to 120km / h at night, that is equivalent to about Elevation angle degrees 33.33 ms/ . However, in previous studies, especially 37.4 Vertical scanning angle in [6], this issue was mentioned but not resolved degrees because of the system implementation context in Focal length 4 mm typical urban areas rather than highways. Therefore, some typical surveillance cameras are unable to Resolution 1280 x 720 capture vehicles moving at high speed, hence the opening and closing ST time may significantly affect Table 2. Comparison of the results of the proposed the accuracy of the speed detected. method and the method in [5] (km/h) for highways Detected Looking back to Fig. 1, the surveillance camera Detected Detected speed by the used in the system to collect data operates at 12 fps at No Samples speed by speed by proposed a resolution of 1280 x 720 pixels. Normally, the method in [5] GPS opening and closing ST to capture high-speed method movement is typically designed at 1/8000s. 1 Samples 1 88.4 86.4 89.75 That means the travelled distance of a given vehicle detected per frame is around 33.33⁄12 m. In this 2 Samples 2 85.6 83.5 86.15 case, it is easy to see that the error in speed detection 3 Samples 3 90.3 89.3 90.74 without consideration of ST is equivalent to about 8.33% at speed of 120 km/h or 33.33 m/s. This error is 4 Samples 4 80.7 80.5 81.05 even greater for blurred frames in the acquired video sequence as well as processing steps in speed detection 5 Samples 5 86.4 85.3 86.77 workflow. If the ST is opened longer or at least long enough to capture a frame at a given speed, the image 6 Samples 6 90.7 88.6 91.05 will be blurred followed by such a called media trail. 7 Samples 7 89.5 87.4 90.18 This may lead to incorrect box detection covering the vehicle or its license plate. 8 Samples 8 83.4 82.3 83.66 3. Results and Evaluation 9 Samples 9 89.9 88.3 90.11 3.1 Simulation Scenario 10 Samples 10 87.6 87.1 87.85 The traffic data are acquired on highways and streets of an inner city and prepared for performance 3.3 Results and Performance Evaluation evaluation of the proposed method. The traffic data from the surveillance camera system implemented on Table 2 summarizes the detected speeds by Phap Van - Cau Gie Expressway and Quan Su Street applying the proposed method and that of the work in in Hanoi city are retrieved from the centralized [6]. Herein, 10 samples are taken from the traffic monitoring system. The simulation data is collected in databases including different vehicles on Phap Van – the morning time without rain and sunshine. The Cau Gie highway and moving at different speeds. surveillance camera’s setting parameters are given in Meanwhile as demonstrated in the third column of Table 1. Table 2, considering the detected speeds to that measured speed by the installed GPS in a vehicle 3.2 Simulation Results which appears in the captured video sequence, the The modeling in [6] is reutilized herein in the results imply the significant improvement in terms of proposed method, however with consideration of speed detection accuracy thanks to the controlled ST, opening and closing ST of the camera to enhance the which is small enough in this case. In addition, the accuracy of vehicle speed detection speed. Thanks to higher resolution of the acquired frames also the control of this ST of the surveillance camera, one contributes to better accuracy of seed detection can see that both vehicles and their license plates have process. been identified and localized on both highways as well as in inner streets. 47
6. JST: Smart Systems and Devices Volume 31, Issue 2, September 2021, 043-050 Fig. 7. Demonstration of detected vehicles with theirs license plates bounded by the red boxes from the consecutive frames in video sequence acquired on Phap Van – Cau Gie Expressway Fig. 8. Illustration of detected vehicles from a mixed traffic flow acquired on Quan Su street 48
7. JST: Smart Systems and Devices Volume 31, Issue 2, September 2021, 043-050 In analogy, Table 3 demonstrates the result The speed detection error in [6] is about 8.33% comparison in the speed detection process compared to that of the detected by GPS speed implemented by the proposed method, previous work mounted in the vehicle. The reason mainly comes from in [6] and GSP system inside vehicles that join in a the blurriness of the acquired video sequence of the mixed traffic flow on Quan Su Street. This surveillance cameras. Utilization of ST in this work performance proves that the proposed solution is helps to reduce the blurriness and therefore improving applicable to monitor vehicle speeds moving on the monitor of vehicle speeds. highways as well as streets. Table 3. Comparison of the results of the proposed The detected vehicles from the video sequence method and the method in [5] (km/h) for inner-city are demonstrated in Fig. 7 where a truck and bus are streets coming from a distance, then captured by the Detected Detected surveillance camera and its speed is detected from the Detected speed by the speed by measured centroid movement of the license plate No Sample speed by proposed method in throughout consecutive frames. GPS method [5] Fig. 8 illustrates another transportation scenario 1 Samples 1 28.01 27.84 28.06 in Quan Su Street in Hanoi, where a mixed traffic flow from different types of vehicles such as motorbikes, 2 Samples 2 25.64 24.43 25.83 cars is recorded by a surveillance camera. One can see that all vehicles in this traffic have been identified 3 Samples 3 24.57 23.68 24.68 including license plates of cars that have been tracked out. The centroid movement of tracked boxes in 4 Samples 4 41.35 39.99 41.69 consecutive frames has been effectively utilized to monitor the speed of each vehicle appearing in the 5 Samples 5 4.56 3.96 4.58 video sequence, because of the exact vehicle’s position 6 Samples 6 35.05 34.18 35.32 the duplicated centroids of vehicles and license plates detected. 7 Samples 7 37.63 36.91 37.66 These results again imply that the modeling in [6] 8 Samples 8 15.09 14.81 15.15 is not suitable for high-speed vehicles such as in highways if the opening and closing ST which have 9 Samples 9 12.23 11.72 12.32 been studied in this work are not taken into account. Fig. 9 illustrates the camera’s shutter, which can be 10 Samples 10 24.11 23.44 24.30 maximum opened to Xmax position for a given frame in a very short interval of time. Investigation of the 6. Conclusion opening and closing ST is helpful to overcome acquisition of blurred images for a given surveillance This paper focuses on modification of research camera, and then enhance the frame quality which work proposed in [6] to improve the speed detection leads to better performance in vehicle and plate accuracy on highways, which investigates the effect of localization and then speed detection. Therefore, the opening and closing ST of the system surveillance control of opening and closing ST will help determine camera. In addition, the proposed method shows better the exact moment the shutter starts to acquire frames performance utilizing centroid information of the and then localize more precise position of vehicle on vehicle's license plate instead of vehicle recognition. road. The future work will concentrate on determination of vehicle speed in dark conditions investigating vehicle’s headlights and taillights. Acknowledgments This work is supported and funded by Hanoi University of Science and Technology under project code T2018-PC-218. References [1]. D. Streller, K. Furstenberg, K. Dietmayer, Vehicle and object models for robust tracking in traffic scenes using laser range images, in proceedings of the IEEE 5th International Conference on Intelligent Fig. 9. Illustration of opening and closing ST for a Transportation Systems, pp. 118 – 123; 2002. given frame 49
8. JST: Smart Systems and Devices Volume 31, Issue 2, September 2021, 043-050 [2]. H. H. Takada, M. Momozawa, H. Kawano, A study of [9]. Massimo Piccardi, Background subtraction the measurement of velocity by image sensors, 2001 techniques: a review, IEEE International Conference IEEE Pacific Rim Conference on Communications, on Systems, Man and Cybernetics, pp. 3099-3104, Computers and Signal Processing (IEEE Cat. 2004. No.01CH37233); 07 August 2002; pp. 251 – 254; 2001. [10]. Nguyen Viet Hung, Nguyen Hoang Dung, Le Chung Tran, Thang Manh Hoang and Nguyen Tien Dzung; [3]. Jiexin Ren, Yao Wang, Yubing Han and Renli Zhang, Vehicle classification by estimation of the direction Information fusion of digital camera and radar, in angle in a mixed traffic flow, in proceedings of IEEE proceedings of 2019 IEEE MTT-S International Sixth International Conference on Communications Microwave Biomedical Conference (IMBioC), May and Electronics (ICCE), pp. 365 - 368, 2016. 2019. [11]. A. G. Rad, A. Dehghani, and M. Rehan Karim, Vehicle [4]. Bhaskar, P. K. Yong, S. P. Image processing based speed detection in video image sequences using CVS vehicle detection and tracking method, in proceedings method, International Journal of the Physical Sciences of 2014 International Conference on Computer and Vol. 5(17), pp. 2555-2563, 18 December 2010. Information Sciences (ICCOINS), July 2014. [12]. Luis Unzueta, Marcos Nieto, Andoni Cortés, Javier [5]. Lei Yang1, (Member, IEEE), Menglong Li, Xiaowei Barandiaran, Oihana Otaegui, and Pedro Sánchez, Song, (Member, IEEE), Zixiang Xiong, (Fellow, Adaptive multicue background subtraction for IEEE), Chunping Hou, And Boyang Qu, Vehicle speed robust vehicle counting and classification, IEEE measurement based on binocular stereovision system, Transactions on Intelligent Transportation Systems, IEEE Access; Vol. 4, 2016; pp. 1-14. vol. 13, Issue 2, June 2012. [6]. Nguyen Viet Hung, Nguyen Thi Thao, Do Huy Khoi, [13]. 13 A. Fernández-Caballero Francisco J.Gómez, Juan Nguyen Tien Dzung (2017), Modeling of a vehicle López-López, Road-traffic monitoring by knowledge- speed detection method based on image processing driven static and dynamic image analysis, Science techniques, Journal of Science and Technology, Thai Direct, Expert Systems with Applications, Vol. 35, Nguyen University, ISSN 1859-2171, Vol. 169, No 9, Issue 3, October 2008, Pages 701-719. pp. 39-44, 2017. (In Vietnamese) [7]. David A. Forsyth and Jean Ponce, Computer Vision a [14]. 14 Huansheng Song, Haoxiang Liang, Huaiyu Li, Zhe Modern Approach, University of Illinois at Urbana- Dai & Xu Yun, Vision-based vehicle detection and Champaign, Jean Ponce, Ecole Normale Superieure, counting system using deep learning in highway Paris, Second edition, 2012. scenes, European Transport Research Review, Vol. 11, No. 51, Dec. 30, 2019. [8]. C. Stauffer and W. E. L. Grimson, Adaptive background mixture models for real-time tracking, in proceedings IEEE CVPR 1999, pp. 246-252, June 1999. 50