Aseem Behl

Ph.D. Student

Autonomous Vision

Office: S2.017
Max-Planck-Ring 4
72076 Tübingen
Germany

My research is in the area of Computer Vision and Machine Learning. Currently, I am working on problems related to estimation of 3D motion fields from stereo image sequences.

Website: http://aseembehl.github.io/

Google Scholar: https://scholar.google.de/citations?user=UEivs9sAAAAJ&hl=en

computer vision machine learning

Global Localization and Affordance Learning

Intelligent systems not only require the relative motion of objects around them \cite{Liu2017IROS}\cite{Liu2018IROS}, but typically also a precise global location \cite{Camposeco2017ICCV} with respect to a map, i.e. for planning or navigation tasks. With LOST \cite{Brubaker2016PAMI}, we have demonstrated that localization solel...

Andreas Geiger Joel Janai Fatma Güney Aseem Behl Laura Sevilla Yiyi Liao Benjamin Coors Moritz Menze Jun Xie

Object Scene Flow

To recover 3D motion \cite{Menze2018JPRS}, we have developed a state-of-the-art 3D scene flow estimation technique which exploits recognition (bounding boxes, instance segmentation and object coordinates) to support the challenging matching task \cite{Behl2017ICCV}.

While great progress has been made in recent years, large...

Moritz Menze Aseem Behl Andreas Geiger

9 results (View BibTeX file of all listed publications)

2020

Label Efficient Visual Abstractions for Autonomous Driving

Behl, A., Chitta, K., Prakash, A., Ohn-Bar, E., Geiger, A.

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, October 2020 (conference)

Abstract

It is well known that semantic segmentation can be used as an effective intermediate representation for learning driving policies. However, the task of street scene semantic segmentation requires expensive annotations. Furthermore, segmentation algorithms are often trained irrespective of the actual driving task, using auxiliary image-space loss functions which are not guaranteed to maximize driving metrics such as safety or distance traveled per intervention. In this work, we seek to quantify the impact of reducing segmentation annotation costs on learned behavior cloning agents. We analyze several segmentation-based intermediate representations. We use these visual abstractions to systematically study the trade-off between annotation efficiency and driving performance, ie, the types of classes labeled, the number of image samples used to learn the visual abstraction model, and their granularity (eg, object masks vs. 2D bounding boxes). Our analysis uncovers several practical insights into how segmentation-based visual abstractions can be exploited in a more label efficient manner. Surprisingly, we find that state-of-the-art driving performance can be achieved with orders of magnitude reduction in annotation cost. Beyond label efficiency, we find several additional training benefits when leveraging visual abstractions, such as a significant reduction in the variance of the learned policy when compared to state-of-the-art end-to-end driving models.

pdf slides video Project Page [BibTex]

2020

Behl, A., Chitta, K., Prakash, A., Ohn-Bar, E., Geiger, A. Label Efficient Visual Abstractions for Autonomous Driving IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, October 2020 (conference)

pdf slides video Project Page [BibTex]

Exploring Data Aggregation in Policy Learning for Vision-based Urban Autonomous Driving

Prakash, A., Behl, A., Ohn-Bar, E., Chitta, K., Geiger, A.

In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020, 2020 (inproceedings)

Abstract

Data aggregation techniques can significantly improve vision-based policy learning within a training environment, e.g., learning to drive in a specific simulation condition. However, as on-policy data is sequentially sampled and added in an iterative manner, the policy can specialize and overfit to the training conditions. For real-world applications, it is useful for the learned policy to generalize to novel scenarios that differ from the training conditions. To improve policy learning while maintaining robustness when training end-to-end driving policies, we perform an extensive analysis of data aggregation techniques in the CARLA environment. We demonstrate how the majority of them have poor generalization performance, and develop a novel approach with empirically better generalization performance compared to existing techniques. Our two key ideas are (1) to sample critical states from the collected on-policy data based on the utility they provide to the learned policy in terms of driving behavior, and (2) to incorporate a replay buffer which progressively focuses on the high uncertainty regions of the policy's state distribution. We evaluate the proposed approach on the CARLA NoCrash benchmark, focusing on the most challenging driving scenarios with dense pedestrian and vehicle traffic. Our approach improves driving success rate by 16% over state-of-the-art, achieving 87% of the expert performance while also reducing the collision rate by an order of magnitude without the use of any additional modality, auxiliary tasks, architectural modifications or reward from the environment.

pdf suppmat Video 2 Project Page Slides Video 1 [BibTex]

Prakash, A., Behl, A., Ohn-Bar, E., Chitta, K., Geiger, A. Exploring Data Aggregation in Policy Learning for Vision-based Urban Autonomous Driving In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020, 2020 (inproceedings)

pdf suppmat Video 2 Project Page Slides Video 1 [BibTex]

Learning Situational Driving

Ohn-Bar, E., Prakash, A., Behl, A., Chitta, K., Geiger, A.

In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020, 2020 (inproceedings)

Abstract

Human drivers have a remarkable ability to drive in diverse visual conditions and situations, e.g., from maneuvering in rainy, limited visibility conditions with no lane markings to turning in a busy intersection while yielding to pedestrians. In contrast, we find that state-of-the-art sensorimotor driving models struggle when encountering diverse settings with varying relationships between observation and action. To generalize when making decisions across diverse conditions, humans leverage multiple types of situation-specific reasoning and learning strategies. Motivated by this observation, we develop a framework for learning a situational driving policy that effectively captures reasoning under varying types of scenarios. Our key idea is to learn a mixture model with a set of policies that can capture multiple driving modes. We first optimize the mixture model through behavior cloning, and show it to result in significant gains in terms of driving performance in diverse conditions. We then refine the model by directly optimizing for the driving task itself, i.e., supervised with the navigation task reward. Our method is more scalable than methods assuming access to privileged information, e.g., perception labels, as it only assumes demonstration and reward-based supervision. We achieve over 98% success rate on the CARLA driving benchmark as well as state-of-the-art performance on a newly introduced generalization benchmark.

pdf suppmat Video 2 Project Page Video 1 Slides [BibTex]

Ohn-Bar, E., Prakash, A., Behl, A., Chitta, K., Geiger, A. Learning Situational Driving In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020, 2020 (inproceedings)

pdf suppmat Video 2 Project Page Video 1 Slides [BibTex]

Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art

Janai, J., Güney, F., Behl, A., Geiger, A.

Arxiv, Foundations and Trends in Computer Graphics and Vision, 2020 (book)

Abstract

Recent years have witnessed enormous progress in AI-related fields such as computer vision, machine learning, and autonomous vehicles. As with any rapidly growing field, it becomes increasingly difficult to stay up-to-date or enter the field as a beginner. While several survey papers on particular sub-problems have appeared, no comprehensive survey on problems, datasets, and methods in computer vision for autonomous vehicles has been published. This monograph attempts to narrow this gap by providing a survey on the state-of-the-art datasets and techniques. Our survey includes both the historically most relevant literature as well as the current state of the art on several specific topics, including recognition, reconstruction, motion estimation, tracking, scene understanding, and end-to-end learning for autonomous driving. Towards this goal, we analyze the performance of the state of the art on several challenging benchmarking datasets, including KITTI, MOT, and Cityscapes. Besides, we discuss open problems and current research challenges. To ease accessibility and accommodate missing references, we also provide a website that allows navigating topics as well as methods and provides additional information.

pdf Project Page link Project Page [BibTex]

Janai, J., Güney, F., Behl, A., Geiger, A. Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art Arxiv, Foundations and Trends in Computer Graphics and Vision, 2020 (book)

pdf Project Page link Project Page [BibTex]

2019

PointFlowNet: Learning Representations for Rigid Motion Estimation from Point Clouds

Behl, A., Paschalidou, D., Donne, S., Geiger, A.

In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2019, June 2019 (inproceedings)

Abstract

Despite significant progress in image-based 3D scene flow estimation, the performance of such approaches has not yet reached the fidelity required by many applications. Simultaneously, these applications are often not restricted to image-based estimation: laser scanners provide a popular alternative to traditional cameras, for example in the context of self-driving cars, as they directly yield a 3D point cloud. In this paper, we propose to estimate 3D motion from such unstructured point clouds using a deep neural network. In a single forward pass, our model jointly predicts 3D scene flow as well as the 3D bounding box and rigid body motion of objects in the scene. While the prospect of estimating 3D scene flow from unstructured point clouds is promising, it is also a challenging task. We show that the traditional global representation of rigid body motion prohibits inference by CNNs, and propose a translation equivariant representation to circumvent this problem. For training our deep network, a large dataset is required. Because of this, we augment real scans from KITTI with virtual objects, realistically modeling occlusions and simulating sensor noise. A thorough comparison with classic and learning-based techniques highlights the robustness of the proposed approach.

pdf suppmat Project Page Poster Video [BibTex]

2019

Behl, A., Paschalidou, D., Donne, S., Geiger, A. PointFlowNet: Learning Representations for Rigid Motion Estimation from Point Clouds In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2019, June 2019 (inproceedings)

pdf suppmat Project Page Poster Video [BibTex]

2017

Bounding Boxes, Segmentations and Object Coordinates: How Important is Recognition for 3D Scene Flow Estimation in Autonomous Driving Scenarios?

Behl, A., Jafari, O. H., Mustikovela, S. K., Alhaija, H. A., Rother, C., Geiger, A.

In Proceedings IEEE International Conference on Computer Vision (ICCV), IEEE, Piscataway, NJ, USA, IEEE International Conference on Computer Vision (ICCV), October 2017 (inproceedings)

Abstract

Existing methods for 3D scene flow estimation often fail in the presence of large displacement or local ambiguities, e.g., at texture-less or reflective surfaces. However, these challenges are omnipresent in dynamic road scenes, which is the focus of this work. Our main contribution is to overcome these 3D motion estimation problems by exploiting recognition. In particular, we investigate the importance of recognition granularity, from coarse 2D bounding box estimates over 2D instance segmentations to fine-grained 3D object part predictions. We compute these cues using CNNs trained on a newly annotated dataset of stereo images and integrate them into a CRF-based model for robust 3D scene flow estimation - an approach we term Instance Scene Flow. We analyze the importance of each recognition cue in an ablation study and observe that the instance segmentation cue is by far strongest, in our setting. We demonstrate the effectiveness of our method on the challenging KITTI 2015 scene flow benchmark where we achieve state-of-the-art performance at the time of submission.

pdf suppmat Poster Project Page [BibTex]

2017

Behl, A., Jafari, O. H., Mustikovela, S. K., Alhaija, H. A., Rother, C., Geiger, A. Bounding Boxes, Segmentations and Object Coordinates: How Important is Recognition for 3D Scene Flow Estimation in Autonomous Driving Scenarios? In Proceedings IEEE International Conference on Computer Vision (ICCV), IEEE, Piscataway, NJ, USA, IEEE International Conference on Computer Vision (ICCV), October 2017 (inproceedings)

pdf suppmat Poster Project Page [BibTex]

2015

Optimizing Average Precision using Weakly Supervised Data

Behl, A., Mohapatra, P., Jawahar, C. V., Kumar, M. P.

IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 2015 (article)

[BibTex]

2015

Behl, A., Mohapatra, P., Jawahar, C. V., Kumar, M. P. Optimizing Average Precision using Weakly Supervised Data IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 2015 (article)

[BibTex]

2014

Optimizing Average Precision using Weakly Supervised Data

Behl, A., Jawahar, C. V., Kumar, M. P.

IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2014, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), June 2014 (conference)

[BibTex]

2014

Behl, A., Jawahar, C. V., Kumar, M. P. Optimizing Average Precision using Weakly Supervised Data IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2014, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), June 2014 (conference)

[BibTex]

Learning to Rank using High-Order Information

Dokania, P. K., Behl, A., Jawahar, C. V., Kumar, M. P.

International Conference on Computer Vision, 2014 (conference)

[BibTex]

Dokania, P. K., Behl, A., Jawahar, C. V., Kumar, M. P. Learning to Rank using High-Order Information International Conference on Computer Vision, 2014 (conference)

[BibTex]

Aseem Behl

Global Localization and Affordance Learning

Object Scene Flow

2020

Label Efficient Visual Abstractions for Autonomous Driving

2020

Exploring Data Aggregation in Policy Learning for Vision-based Urban Autonomous Driving

Learning Situational Driving

Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art

2019

PointFlowNet: Learning Representations for Rigid Motion Estimation from Point Clouds

2019

2017

Bounding Boxes, Segmentations and Object Coordinates: How Important is Recognition for 3D Scene Flow Estimation in Autonomous Driving Scenarios?

2017

2015

Optimizing Average Precision using Weakly Supervised Data

2015

2014

Optimizing Average Precision using Weakly Supervised Data

2014

Learning to Rank using High-Order Information

Latest News

Links

Contact Us