Skip to main content
SHARE
Publication

Edge AI-Enhanced Traffic Monitoring and Anomaly Detection Using Multimodal Large Language Models

by Ryan Peruski, Abhilasha J Saroj, Wenjun Zhou, Seddik Djouadi, Charles Cao
Publication Type
Conference Paper
Book Title
International Conference on Transportation and Development 2025
Publication Date
Page Numbers
429 to 438
Publisher Location
Virginia, United States of America
Conference Name
ASCE International Conference on Transportation and Development 2025 (ICTD)
Conference Location
Glendale, Arizona, United States of America
Conference Sponsor
Several - ASCE Foundation, ARA
Conference Date
-

This paper addresses the challenge of traffic monitoring and incident detection in remote areas, utilizing multimodal large language models (LLMs) deployed on edge AI devices. The key novelty of the LLM is to convert real-time video streams into descriptive texts, enabling low-bandwidth transmissions and reliable detection of anomalies and incidents in environments of intermittent connectivity. The model is developed based on fine-tuning open-source LLMs and extending it with multi-modal capabilities to analyze video frames. Our work also involves deploying this model on edge devices such as Nvidia IGX Orin and is planned to be tested in realistic environments in future work. The methodology includes data set curation, iterative model fine-tuning and compression, and hardware-based optimization. This approach aims to enhance traffic safety and response speed in remote areas, marking a significant advancement in the application of AI for traffic monitoring and safety management.