Effect of Inertial Measurement Unit on Creating High-Definition 3D Map for Autonomous Vehicle

Nội dung text: Effect of Inertial Measurement Unit on Creating High-Definition 3D Map for Autonomous Vehicle

1. JST: Smart Systems and Devices Volume 31, Issue 1, May 2021, 033-040 Effect of Inertial Measurement Unit on Creating High-Definition 3D Map for Autonomous Vehicle Nang Xuan Ho1,2*, Anh Son Le1,2 1 Phenikaa Research and Technology Institute, Phenikaa Group, Hanoi, Vietnam 2 Faculty of Vehicle and Energy Engineering, Phenikaa University, Hanoi, Vietnam * Email: nang.hoxuan@phenikaa-uni.edu.vn Abstract One of the most famous technologies for the autonomous vehicle was using a scan matching algorithm, in which a high-definition 3D map created by the LIDAR sensor plays a significantly important role in localizing and path planning. Within this manuscript, a novel way of finding the effect from the Inertial Measurement Unit (IMU) on creating a high-definition 3D map from the LIDAR sensor was investigated. The collection data system was first ever created and collected in Vietnam. The results show that the normal distributions transform shows very good performance for creating the HD 3D map with have IMU sensor. On the other hand, without IMU the accuracy and the robustness of the creating map were reduced especially in the non- flat area. This manuscript will start the evolution of preparation for autonomous vehicles in Vietnam as well as contribute to the autonomous vehicle research society in the world. Keywords: HD map, autonomous vehicle, IMU. 1. Introduction* extract the point cloud by adding the global position system (GPS) information (Fig.2) and 2/ using the According to the World Health Organization, a LIDAR (Fig.3). traffic accident is a critical problem for the world, it takes approximately 1.35 million people to die each The point cloud is a set of points (x, y, z) in a year and it is still increasing [1]. One of the ideas is space created with many purposes such as use in an autonomous vehicle, which can significantly shaping, architecture, and especially applied to the reduce traffic accidents in the future. survey of traffic works or using used to convert to vector maps for future autonomous vehicles. In the According to several researchers, to be ready for case of using the camera for creating a point cloud an autonomous vehicle, we have to prepare the map, the series of pictures will be taken with the facility included drive-by-wire vehicles, maps, and global positioning system (GPS) information. Each sensors (Fig.1). After that, the sensing data, picture will be processed to be a set of the vector perception, planning, control, localization should be point, and then by comparing the nearest set of vector completed to make a full system of the autonomous points of the next picture, the combination will vehicle [2-8]. Here we only focus on the creation of a combine two sets to become one and creating a new high definition (HD) 3D map. point cloud map. HD map for AV is not a new technology in the In the case of using LIDAR, all of the world, but in Vietnam, there is still no HD map information from LIDAR will be collected. The scan database or HD map companies. The application of matching algorithm will be applied for comparing 3D LIDAR for making point clouds is limited. There with the previous scan and then combine them are only some companies and groups that use together. photogrammetry techniques to make 3D point clouds, mainly for geology and architecture conservation Furthermore, the HD map is not only using for purposes. To realize AV in Vietnam shortly, it is vehicle navigation but also using for the vehicle to necessary to develop both HD map databases and understand the surrounding environment such as lane, tools/devices. traffic signal location, road type, by adding the information to the HD map (Fig.4).As recommended There are two kinds of techniques for creating a by the previous researcher [10], we have selected the point cloud map [9]: 1/ Using camera (image) to second technology by using LIDAR 3D to create the HD 3D map and also investigate the effect of IMU on creating an HD map in Vietnam, which does not exist ISSN 2734-9373 before. Received: January 01, 2021; accepted: May 12, 2021 33
2. JST: Smart Systems and Devices Volume 31, Issue 1, May 2021, 033-040 Fig.1. A typical autonomous vehicle system overview Fig. 2. Point Cloud map created by image Fig. 3. Point Cloud map created by LIDAR 2. Materials and Methods 2.1. The Mathematic Calculation for Creating an HD 3D Map For creating the HD 3D map, Biber et al. introduced the Normal Distributions Transform (NDT) scan matching method by using point cloud and divided it into a uniform grid [11]. Each voxel (cell) uses the mean and distribution of the sub-point cloud assigned to it. After that, this algorithm was developed by Takeuchi et al to enhance the scan- matching by adding the dual resolutions of NDT [12]. In this manuscript, we applied the same technique with Takeuchi, the equation shows below: Fig. 4. Adding information for HD map 34
3. JST: Smart Systems and Devices Volume 31, Issue 1, May 2021, 033-040 1 M k = − −1 = tnew t Hg (5) pxk ∑ ki (1) M = k i 1 where g and H are the partial differentials and M k second-order partial differential of the optimizing 1 T = xpxp −− (2) ∑∑k ( ki k)( ki k ) function f. M k i=1 Basically, without IMU, the t parameter will be T while xi = ( xyzi,, ii) with i = 1:M; estimated through equation (5), which is not considered about the z-axis (the normal global Denote R as the rotation matrix and t′ as the positioning system doesn’t have z-axis information). translation vector, the x' can be calculated by: Furthermore, with the high frequency from IMU, the t i parameter will be updated more often. The processing ' data was shown in Fig.5. xii= Rx + t′ (3) 2.2. Data Acquisition System and Test Area The pose translation and rotation parameters to be estimated are • Data acquisition system For data logging (Fig.6), a hard-ware system t= ( ttttx,,, y z roll , t pitch , t yaw ) was created by:1/an SUV vehicle; 2/ The LIDAR T −1 sensor that has a measurement range up to 100m with N −−'' − ( xpii) ∑ ( xp ii) E( Xt, ) = ∑exp i (4) a range accuracy of ± 3cm and a vertical field of view i 2 from +15˚ to -15˚. The rotation rate can be adjusted from 5 to 20 Hz; 3/ Real-time kinematic (RTK): The E(X,t) represents the matching or the well- GNSS system namely Piksi-Multi evaluation kit with aligned. high accuracy; and 4/ IMU namely Xsens Mti-G-710 A high value of E(X,t) means both the input with gyroscope range +/- 450 degrees/s with bias cloud and the reference map are well aligned. stability 10 degrees/hour. Newton’s nonlinear function optimization is utilized All of the signals from IMU, LIDAR, RTK will to find t such that E(X,t) is maximized. Therefore, we be processed through a high specification computer minimize the function f(t) = −E(X, t). by using Robot Operation System (ROS). All of the The parameter t will be updated using: mathematic for scan matching was done in the ROS. Fig. 5. Data process on ROS 35
4. JST: Smart Systems and Devices Volume 31, Issue 1, May 2021, 033-040 Fig. 6. Data acquisition system. • Testing area and strategy In this study, we collected data at Phenikaa University (Fig.7), which is located in Hanoi, Vietnam with two scenarios: 1/ Inside the campus of Phenikaa University; and 2/ The campus and the road in front of the campus with the overpass. 3. Results 3.1. Data Collection The data collected show very good performance with clear data (Fig.8), especially for the LIDAR signal. All of the data will be input to the ROS and process the NDT algorithm for scan matching and then create the HD map with two cases: with and without IMU by removing the IMU topic. Fig. 7. Testing area. Fig. 8. Creating the map. 36
5. JST: Smart Systems and Devices Volume 31, Issue 1, May 2021, 033-040 3.2. Map creation inside the Phenikaa University (Fig.9). There is not much vibration when the vehicle moves with low speed for mapping creation. The NDT shows very good performance with a high density of point cloud in both case with and without IMU. However, while zoom some parts in detail, the map creation is shown some error while matching the scan point in case of without IMU (Fig.10). In Fig.10, the overview of the HD map shows almost the same due to the flat area, and the vehicle moves slowly. However, at some corners, there is some vibration exit and it may make the scan matching work not so well. In this case of local closed-loop street mapping, both methods with and without using IMU data were able to deliver a usable Fig. 9. A view of Phenikaa University. point cloud of the area, as shown in Fig.10. However, Phenikaa University is located in a flat area, as we zoomed up to the detail of each point cloud south-west of Hanoi. The University just rebuilding data, matching errors were found on the building’s all of the facilities so the road condition is good walls in the case of not using IMU data. Fig. 10. Map creation inside the campus. 37
6. JST: Smart Systems and Devices Volume 31, Issue 1, May 2021, 033-040 3.3. Map Creation Outside the Phenikaa University this data, although the top-view of the main road part seems to be similar to the 2D-map shape, there were In this case, a long straight main road with no still matching errors on the building’s wall, which is closed-loop was chosen. There is an overpass in one similar to the first strategy case when closely looking ending of the testing area, which offers a large- into detail of the point cloud. Besides, the side view altitude change in point cloud data. Moreover, there of the point cloud shows that the main road is bent are not many marks on the overpass, which makes it during the point matching process, which makes the more challenging for the 3D matching without using height of the road changed incorrectly when IMU. Fig.11 shows very clearly in detail the effect of comparing with terrestrial data. On the other hand, IMU, as we zoomed up to the detail of each point the NDT-matching method also failed to generate the cloud data, matching errors were found on the side detail of the overpass part of the main road where the view, overpass in the case of not using IMU data. logging vehicle’s altitude is suddenly changed. In Furthermore, a clear and shaped point cloud was conclusion, IMU is found to be an essential delivered in the case of using IMU data during point secondary sensor that provides stable performance for cloud generation. On the contrary, in the case of not a mapping system. using IMU data, only unusable data was formed. In Fig. 11. Map creation inside the campus. 38
7. JST: Smart Systems and Devices Volume 31, Issue 1, May 2021, 033-040 4. Conclusion and Outlook Theory Appl., vol. 93, no. 3–4, pp. 533–546, 2019, The revolution for transportation namely autonomous vehicles can make a significant change [4] L. Caltagirone, M. Bellone, L. Svensson, and M. such as reducing accidents and congestion will be real Wahde, LIDAR–camera fusion for road detection using fully convolutional neural networks, Rob. soon by overpassing many technical challenges [13]. Auton. Syst., vol. 111, pp. 125–131, 2019, One of the key for autonomous driving in the city is HD maps, which is used for accurate localization [14]. Here, a novel creating the HD maps as well as [5] L. Wang, Y. Zhang, and J. Wang, Map-Based the effect of the IMU sensor on the making HD map Localization Method for Autonomous Vehicles Using 3D-LIDAR., IFAC-PapersOnLine, vol. 50, no. 1, pp. in Vietnam was shown. Basically, IMU, which is an 276–281, 2017, electronic device that measures the orientation based on accelerometers, plays a very important role in navigation such as using with global position system [6] L. Wang, Y. Zhang, and J. Wang, Map-Based to calculate EKF [15]–[17]. At this time, the result Localization Method for Autonomous Vehicles Using also indicated that IMU has a strong influence on 3D-LIDAR, IFAC-PapersOnLine, vol. 50, no. 1, pp. 276–281, 2017, creating HD maps. By collecting the information of x,y,z axis, the IMU can help the scan matching algorithm remove the noise as well as enhance the [7] S. D. Pendleton et al., Perception, planning, control, accuracy by giving exactly the altitude, which is and coordination for autonomous vehicles, Machines, necessary for the vehicle while running in a non-flat vol. 5, no. 1, pp. 1–54, 2017, area. [8] E. Javanmardi, Y. Gu, M. Javanmardi, and S. Kamijo, To make autonomous level 5 happened shortly, Autonomous vehicle self-localization based on we still have to overcome a lot of challenges not only abstract map and multi-channel LIDAR in urban area, inside the vehicle but also the facility preparation. IATSS Res., vol. 43, no. 1, pp. 1–13, 2019, However, with the revolution of the sensors, the computer, the algorithm, we believed that the [9] R. Liu, J. Wang, and B. Zhang, High Definition Map autonomous vehicle will be a key point for changing for Automated Driving: Overview and Analysis, J. the society in near future. Navig., vol. 73, no. 2, pp. 324–341, 2020, The map plays a very important role in the autonomous vehicle especially for path planning and [10] A. Carballo et al., Characterization of Multiple 3D localization. In this manuscript, the effect of IMU on LIDARs for Localization and Mapping using Normal the creating map has been shown. The results Distributions Transform, no. April, 2020, [Online]. indicated that without IMU, the map creation may Available: have a lot of effects such as cloud noise, incorrect [11] P. Biber, The Normal Distributions Transform: A potion, burry. It may help other researchers have a New Approach to Laser Scan Matching, IEEE Int. deep looking into the design system and move Conf. Intell. Robot. Syst., vol. 3, no. November 2003, forward to make autonomous vehicles happened. pp. 2743–2748, 2003, Acknowledgments [12] E. Takeuchi and T. Tsubouchi, A 3-D scan matching This work was partly supported by Phenikaa using improved 3-D normal distributions transform University. for mobile robotic mapping, in IEEE International Conference on Intelligent Robots and Systems, 2006, References [1] Road traffic injuries, 2020 [Online]. Avaiable: [13] M. Martínez-Díaz, F. Soriguera, and I. Pérez, Autonomous driving: A bird’s eye view, IET Intell. sheets/detail/road-traffic- Transp. Syst., vol. 13, no. 4, pp. 563–579, 2019. injuries#:~:text=Approximately 1.35 million people die,result of road traffic crashes.&text=More than half of all,pedestrians%2C cyclists%2C and motorcyclists. [14] H. G. Seif and X. Hu, Autonomous Driving in the iCity—HD Maps as a Key Challenge of the [2] Á. Arcos-García, J. A. Álvarez-García, and L. M. Automotive Industry, Engineering, vol. 2, no. 2, pp. Soria-Morillo, Evaluation of deep neural networks for 159–162, 2016. traffic sign detection systems, Neurocomputing, vol. 316, pp. 332–344, 2018, [15] J. Kim and S. Lee, A vehicular positioning with GPS/IMU using adaptive control of filter noise [3] H. Sobreira et al., Map-Matching Algorithms for covariance, ICT Express, vol. 2, no. 1, pp. 41–46, Robot Self-Localization: A Comparison Between 2016 Perfect Match, Iterative Closest Point and Normal Distributions Transform, J. Intell. Robot. Syst. 39
8. JST: Smart Systems and Devices Volume 31, Issue 1, May 2021, 033-040 [16] A. Ndjeng Ndjeng, D. Gruyer, S. Glaser, and A. Intelligent vehicle localization in urban environments Lambert, Low cost IMU-Odometer-GPS ego using EKF-based visual odometry and GPS fusion, localization for unusual maneuvers, Inf. Fusion, vol. IFAC Proc. Vol., vol. 44, no. 1 PART 1, pp. 13776– 12, no. 4, pp. 264–274, 2011. 13781, 2011, [17] L. Wei, C. Cappelle, Y. Ruichek, and F. Zann, 40