Accurate 3D trajectory data is crucial for advancing autonomous driving. Yet, traditional datasets are usually captured by fixed sensors mounted on a car and are susceptible to occlusion. Additionally, such an approach can precisely reconstruct the dynamic environment in the close vicinity of the measurement vehicle only, while neglecting objects that are further away.
In this paper, we introduce the DeepScenario Open 3D Dataset (DSC3D), a high-quality, occlusion-free dataset of 6 degrees of freedom bounding box trajectories acquired through a novel monocular camera drone tracking pipeline. Our dataset includes more than 175,000 trajectories of 14 types of traffic participants and significantly exceeds existing datasets in terms of diversity and scale, containing many unprecedented scenarios such as complex vehicle-pedestrian interaction on highly populated urban streets and comprehensive parking maneuvers from entry to exit. DSC3D dataset was captured in five various locations in Europe and the United States and include: a parking lot, a crowded inner-city, a steep urban intersection, a federal highway, and a suburban intersection. Our 3D trajectory dataset aims to enhance autonomous driving systems by providing detailed environmental 3D representations, which could lead to improved obstacle interactions and safety. We demonstrate its utility across multiple applications including motion prediction, motion planning, scenario mining, and generative reactive traffic agents. Our interactive online visualization platform and the complete dataset are publicly available at app.deepscenario.com, facilitating research in motion prediction, behavior modeling, and safety validation.
DSC3D counts about 15 hours of data across five different
locations in Germany and the United States, with a total of
177,151 unique trajectories. The dataset covers diverse scene
types including parking area (SIFI), inner-city environment (MUC), interesting non-signalized
intersections (STR, SFO), and federal highway (BER), capturing a wider range of
scenarios compared to specialized datasets that focus on
single environment types.
The DSC3D dataset presents the largest classification framework
of drone-based datasets including 14 separate categories.
The main categories consist of pedestrians (140,227),
bicycles (17,736), cars (13,241), scooters (1,475), motorcycles (1,054),
animals (677), trucks (475), buses (191), and
other (2,075). This detailed classification system allows researchers
to perform in-depth analysis of interactions among
various traffic participants.
@inproceedings{dsc3d,
title = {Highly Accurate and Diverse Traffic Data: The DeepScenario Open 3D Dataset},
author = {Dhaouadi,Oussema and Meier, Johannes and Wahl, Luca and Kaiser, Jacques and Scalerandi, Luca and Wandelburg, Nick and Zhuo, Zhuolun and Berinpanathan, Nijanthan and Banzhaf, Holger and Cremers, Daniel},
booktitle = {2025 IEEE Intelligent Vehicles Symposium},
year = {2025},
organization = {IEEE}
}