Deep Learning Based Dynamic Object Filtering for Semantic Information Assisted Visual LiDAR SLAM in Dynamic Scenes
DOI:
https://doi.org/10.67541/jdc2612Keywords:
Visual-LiDAR SLAM; Dynamic object filtering; Evidence arbitration; Semantic modulation; Tightly coupled optimizationAbstract
To address the structural mismatch between semantic priors and motion evidence in tightly coupled visual-LiDAR SLAM, this paper proposes a dynamic object filtering architecture centered on evidence arbitration. Semantic segmentation outputs are transformed from binary rejection masks into continuous spatial modulation coefficients ranging from 0 to 1. These coefficients guide the generation of two complementary streams of motion evidence: visual optical flow residuals and multi-frame LiDAR occupancy consistency. Log-odds fusion then produces cross-modal confidence and its uncertainty. Using semantic instances as arbitration units, the method resolves conflicts within a sliding window of 6-8 frames by jointly considering mean confidence, temporal stability, and spatial concentration, with an upper decision delay bound of 3 frames. Finally, dynamic probabilities are embedded in the factor graph as information matrix modulation factors, replacing direct front-end rejection. Experiments on Semantic KITTI, KITTI raw, and Urban Nav show that, while maintaining a dynamic-point residual rate no higher than that of the hard-filtering baseline, the method increases the static semantic constraint retention rate from 0.42 to 0.78. Decision accuracy reaches 0.86 and 0.83 for conflict modes (b) and (c), respectively, and trajectory errors during transition intervals are substantially lower than those of the comparison variants. A breakdown of per-frame processing time identifies semantic segmentation as the main bottleneck. The system meets real-time constraints when the segmentation frame rate is at least 8 Hz, and the dynamic instance density does not exceed 15 instances per frame.
References
[1] Wang, H., Hu, Q., Li, B., Geng, Y., Su, C., Zhu, S., ... & Chen, W. (2026). RTS-SLAM: A Trajectory Consistency-Driven Multi-Constraint Dynamic Feature-Rejection Method for Visual SLAM in Dynamic Environments. Sensors, 26(9), 2846. https://doi.org/10.3390/s26092846
[2] Polukhin, A., Stirenko, S., Leier, M., Jervan, G., Rokovyi, O., Alienin, O., ... & Gordienko, Y. (2026). Depth-assisted sparse visual odometry for UAV-relevant synthetic RGB-D evaluation: A controlled geometric-backend ablation. Robotics, 15(7), 128. https://doi.org/10.3390/robotics15070128
[3] Zheng, S., Li, J., Wang, G., Ni, M., Palit, A., Montana, G., & Guan, Y. (2026). Person parametric physics-informed representation for mmwave-based human pose estimation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 10(2), 1-34. https://doi.org/10.1145/3810219
[4] Zhang, Y. X., Gui, J., Yu, B., Cong, X., Gong, X., Tao, W., & Tao, D. (2026). Deep learning-based point cloud registration: A comprehensive survey and taxonomy. International Journal of Computer Vision, 134(4), 168. https://doi.org/10.1007/s11263-025-02723-w
[5] Cheng, X., Liu, X., Li, J., & Zhou, W. (2024). Deep learning-based point cloud registration: a comprehensive investigation. International Journal of Remote Sensing, 45(10), 3412-3442. https://doi.org/10.1080/01431161.2024.2343434
[6] Liu, M., Zou, Q., Long, J., Wang, Y., Lin, M., & Wang, F. (2025). SSE-SLAM: Semantic Visual SLAM Based on RGB-D Camera for High-Accuracy Pose Measurement in Dynamic Environments. IEEE Transactions on Instrumentation and Measurement. https://doi.org/10.1109/tim.2025.3621768
[7] Zhou, T., Porikli, F., Crandall, D. J., Van Gool, L., & Wang, W. (2022). A survey on deep learning technique for video segmentation. IEEE transactions on pattern analysis and machine intelligence, 45(6), 7099-7122. https://doi.org/10.1109/TPAMI.2022.3225573
[8] Li, X., Wang, L., Luo, H., & Li, G. (2026). RI-DVP: A Physics–Geometry Dual-Driven Framework for Static Map Construction in Sparse LiDAR Scenarios. Remote Sensing, 18(5), 821. https://doi.org/10.3390/rs18050821
[9] Chen, Z., Zhang, F., Zhu, C., Zhang, H., Chen, L., Zhu, H., & Yu, F. (2026). Enhancing PointPillars for 3-D Small Object Detection: A Multi-Dimensional Feature Fusion Approach. IEEE Sensors Journal. https://doi.org/10.1109/JSEN.2026.3680151
[10] Kojukhov, A., Levin, I., & Bovshover, A. (2025). From Subsumption to Semantic Mediation: A Generative Orchestration Architecture for Autonomous Systems. Algorithms, 18(12), 773. https://doi.org/10.3390/a18120773
[11] Sanket, N. J., Singh, C. D., Fermüller, C., & Aloimonos, Y. (2023). Ajna: Generalized deep uncertainty for minimal perception on parsimonious robots. Science Robotics, 8(81), eadd5139. https://doi.org/10.1126/scirobotics.add5139
[12] Sun, Z., Xu, W., Shi, L., Cen, Y., Jin, Y., & Li, Y. (2026). UAGM: Uncertainty-Aware Geometric Modeling for Multi-Scenario 3-D Object Detection in Autonomous Vehicles. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/tits.2026.3685623
[13] Fang, Z., Zhou, S., Cui, Y., & Scherer, S. (2020). 3d-siamrpn: An end-to-end learning method for real-time 3d single object tracking using raw point cloud. IEEE Sensors Journal, 21(4), 4995-5011. https://doi.org/10.1109/jsen.2020.3033034
[14] Che, R., Hu, H., Wang, N., & Li, Z. (2026). Dense SLAM using 3D Gaussian Splatting with IMU and Geometric Constraints. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2026.3662499
[15] Tan, Z., Zhang, Z., Lyu, Y., & Pan, Q. (2026). EDOG-Map: Evidential Dynamic Occupancy Grid Mapping for Consumer-Grade UAV Navigation in Dynamic Environments. IEEE Transactions on Consumer Electronics. https://doi.org/10.1109/tce.2026.3725136
[16] Hirt, P. R., Holtkamp, J., Hoegner, L., Xu, Y., & Stilla, U. (2022). Occlusion detection of traffic signs by voxel-based ray tracing using highly detailed models and MLS point clouds of vegetation. International Journal of Applied Earth Observation and Geoinformation, 114, 103017. https://doi.org/10.1016/j.jag.2022.103017
[17] Chakraborty, D., & Dey, E. K. (2024). Segmentation of LiDAR point cloud data in urban areas using adaptive neighborhood selection technique. Plos one, 19(7), e0307138. https://doi.org/10.1371/journal.pone.0307138
[18] Zhou, B., Jiang, D., Dai, Y., & Hayat, T. (2023). Threshold dynamics and probability density function of a stochastic avian influenza epidemic model with nonlinear incidence rate and psychological effect. Journal of Nonlinear Science, 33(2), 29. https://doi.org/10.1007/s00332-022-09885-8
[19] Shen, S., Ito, T., & Hirose, T. (2025). Spatio-temporal gradient flow for efficient motion estimation in sparse point clouds. Journal of Robotics and Mechatronics, 37(6), 1327-1342. https://doi.org/10.20965/jrm.2025.p1327
[20] Wang, H., Hu, Q., Li, B., Geng, Y., Su, C., Zhu, S., ... & Chen, W. (2026). RTS-SLAM: A Trajectory Consistency-Driven Multi-Constraint Dynamic Feature-Rejection Method for Visual SLAM in Dynamic Environments. Sensors, 26(9), 2846. https://doi.org/10.3390/s26092846
[21] Morris, J., Wang, Y., Kliniewski, M., & Ila, V. (2025). Dynosam: Open-source smoothing and mapping framework for dynamic slam. IEEE Transactions on Robotics. https://doi.org/10.1109/tro.2025.3641813
[22] Li, Z., Jiang, C., Sun, Y., Su, H., & He, L. (2025). Precise feature removal method based on semantic and geometric dual masks in dynamic SLAM. Applied Sciences, 15(13), 7095. https://doi.org/10.3390/app15137095
[23] Xu, W., Liu, S., Chen, R., Du, S., & Wang, Y. (2026). Robust Multi-Modal Factor Graph Optimization for Distributed Collaborative LiDAR–Visual–Inertial SLAM. Applied Sciences, 16(10), 4677. https://doi.org/10.3390/app16104677
[24] Chen, J., Jia, K., & Wei, Z. (2025). Small but Mighty: A Lightweight Feature Enhancement Strategy for LiDAR Odometry in Challenging Environments. Remote Sensing, 17(15), 2656. https://doi.org/10.3390/rs17152656
[25] Gopalakrishnan, V., Dey, N., & Golland, P. (2026). PolyPose: Deformable 2D/3D Registration via Polyrigid Transformations. Advances in Neural Information Processing Systems, 38, 60633-60659. https://doi.org/10.48550/arXiv.2505.19256
[26] Zheng, J., Li, Y., Zhao, J., Chen, Y., Xu, J., & Xu, S. (2026). Residual Reweighting with Scan Normalization for LiDAR-Inertial Odometry in Degenerate Environments. IEEE Sensors Journal. https://doi.org/10.1109/JSEN.2026.3714871
[27] Deng, Z., Li, C., Lin, W., & Sun, L. (2026, June). GMM-LIO: Adaptive and Robust LiDAR-Inertial Odometry with Gaussian Mixture Model Voxel Map. In 2026 IEEE International Conference on Robotics and Automation (ICRA) (pp. 21276-21283). IEEE. https://doi.org/10.1109/icra57385.2026.11697299
[28] Liao, Y., Xie, J., & Geiger, A. (2022). Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3), 3292-3310. https://doi.org/10.1109/tpami.2022.3179507
Published
Data Availability Statement
The data that support the findings of this study are available upon request from the corresponding authors, X.Z.
Issue
Section
License
Copyright (c) 2026 Xianlong Zhao (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
