Skip to the content.

:wave: Welcome to the Monocular Depth Estimation Challenge Workshop organized at :wave: wacv

image_0026 image_0254 image_0698 depth_0026 depth_0254 depth_0698

Monocular depth estimation (MDE) is an important low-level vision task, with application in fields such as augmented reality, robotics and autonomous vehicles. Recently, there has been an increased interest in self-supervised systems capable of predicting the 3D scene structure without requiring ground-truth LiDAR training data. Automotive data has accelerated the development of these systems, thanks to the vast quantities of data, the ubiquity of stereo camera rigs and the mostly-static world. However, the evaluation process has also remained focused on only the automotive domain and has been largely unchanged since its inception, relying on simple metrics and sparse LiDAR data.

This workshop seeks to answer the following questions:

  1. How well do networks generalize beyond their training distribution relative to humans?
  2. What metrics provide the most insight into the model’s performance? What is the relative weight of simple cues, e.g. height in the image, in networks and humans?
  3. How do the predictions made by the models differ from how humans perceive depth? Are the failure modes the same?

The workshop will therefore consist of two parts: invited keynote talks discussing current developments in MDE and a challenge organized around a novel benchmarking procedure using the SYNS dataset.

:newspaper: News


:hourglass_flowing_sand: Important Dates


:calendar: Schedule

Time (HST) Event
TBA TBA

:microphone: Keynote Speakers

NOTE: More speakers TBA!

James Elder
James Elder
Professor
York University
Oisin Mac Aodha
Oisin Mac Aodha
Assistant Professor
University of Edinburgh

:checkered_flag: Challenge

Teams submitting to the challenge will also be required to submit a description of their method. As part of the WACV Proceedings, we will publish a paper summarizing the results of the challenge, including a description of each method. All challenge participants surpassing the performance of the Garg baseline [13.7211] (by jspenmar) will be added as authors in this paper. Top performers will additionally be invited to present their method at the workshop. This presentation can be either in-person or virtually.

[GitHub Code] — [Challenge Website]

The challenge focuses on evaluating novel MDE techniques on the SYNS-Patches dataset proposed in this benchmark. This dataset provides a challenging variety of urban and natural scenes, including forests, agricultural settings, residential streets, industrial estates, lecture theatres, offices and more. Furthermore, the high-quality dense ground-truth LiDAR allows for the computation of more informative evaluation metrics, such as those focused on depth discontinuities.

image_0551 image_0893 image_1114 depth_0551 depth_0893 depth_1114

The challenge is hosted on CodaLab. We have provided a GitHub repository containing training and evaluation code for multiple recent SotA approaches to MDE. These will serve as a competitive baseline for the challenge and as a starting point for participants. The challenge leaderboards use the withheld validation and test sets for SYNS-Patches. We additionally encourage evaluation on the public Kitti Eigen-Benchmark dataset.

Submissions will be evaluated on a variety of metrics:

  1. Pointcloud reconstruction: F-Score
  2. Image-based depth: MAE, RMSE, AbsRel
  3. Depth discontinuities: F-Score, Accuracy, Completeness

Challenge winners will be determined based on the pointcloud-based F-Score performance.


:construction_worker: Organizers

Jaime Spencer
Jaime Spencer
Research Fellow
University of Surrey
Stella Quian
Stella Qian
Research Fellow
Aston University
Chris Russell
Chris Russell
Senior Applied Scientist
Amazon
Simon Hadfield
Simon Hadfield
Senior Lecturer
University of Surrey
Erich Graf
Erich Graf
Associate Professor
University of Southampton
Andrew Schofield
Andrew Schofield
Professor
Aston University
Richard Bowden
Richard Bowden
Professor
University of Surrey