PEDESTRIANQA
Project Title : PEDESTRIANQA: A Benchmark for Vision-Language Models on Pedestrian Intention and Trajectory Prediction
Authors: Naman Mishra, Shankar Gangisetty, C. V. Jawahar
PedestrianQA is a large-scale video-based dataset that frames pedestrian intention and trajectory prediction as question-answering tasks with structured rationales. By leveraging vision–language models, it enables learning from visual dynamics, traffic context, and agent interactions while providing explainable predictions. Experiments on PIE, JAAD, TITAN, and IDD-PeD demonstrate improved intention recognition, trajectory forecasting, and rationale generation, highlighting VLMs as a unified framework for pedestrian behavior modeling.
MOTOR
Project Title : MOTOR: A Multimodal Dataset for Two-Wheeler Rider Behavior Understanding
Authors: Varun Paturkar, Shankar Gangisetty, C. V. Jawahar
MOTOR is the first large-scale multimodal dataset for two-wheeler rider behavior analysis in dense, unstructured traffic, comprising 2,500 sequences (25+ hours) from 16 riders. The dataset integrates synchronized multi-view videos, eye-gaze, audio, GPS, and inertial telemetry, along with rich annotations of rider behavior, traffic context, maneuvers, and legality labels. Benchmark experiments show that combining RGB, gaze, and telemetry data significantly improves rider behavior recognition and legality classification, providing a foundation for safety-critical two-wheeler research.
eXplainable Driving Action Anticipation Dataset (DAAD-X)
Project Title : Towards Safer and Understandable Driver Intention Predictions
Authors: Mukilan Karuppasamy, Shankar Gangisetty, Shyam Nanadan Rai, Carlo Masone, C V Jawahar
This work introduces DAAD-X, an explainable multimodal dataset for driver intention prediction with human-understandable reasoning using gaze and ego-vehicle data. It also proposes a Video Concept Bottleneck Model (VCBM) to generate interpretable predictions, showing transformer-based models are more explainable than CNNs
IDD-PeD
Project Title : Pedestrian Intention and Trajectory Prediction in Unstructured Traffic Using IDD-PeD
Authors: Ruthvik Bokkasam, Shankar Gangisetty, A. H. Abdul Hafez, C V Jawahar
This work introduces an Indian pedestrian behavior dataset designed for unstructured driving conditions, capturing challenges like occlusions, lighting variations, and vehicle–pedestrian interactions. It shows that existing intention and trajectory prediction models perform significantly worse, highlighting the need for more robust approaches.
IDD-VPR
Project Title : Visual Place Recognition in Unstructured Driving Environments
Authors: Utkarsh Rai, Shankar Gangisetty, A. H. Abdul Hafez, Anbumani Subramanian, C V Jawahar
This work introduces a new Indian Visual Place Recognition (VPR) dataset designed for unstructured driving environments with challenges like occlusions, traffic variation, and lighting changes. It also provides an annotation tool and shows that existing methods perform significantly worse, highlighting the need for more robust VPR models
Driving Action Anticipation Dataset (DAAD)
Project Title : Early Anticipation of Driving Maneuvers
Authors: Abdul Wasi, Shankar Gangisetty, Shyam Nanadan, CV Jawahar
This work introduces Anticipating Driving Maneuvers (ADM) to predict driver actions before they begin, addressing the limitation of existing methods that only detect maneuvers after onset. It also presents the DAAD dataset and a transformer-based model to improve early maneuver anticipation using multi-view and multimodal data.