People

Antonin Vobecky
ELLIS PhD student

Weakly-Supervised Multi-Modal Learning for Scene Understanding
The objective of the research is to improve the accuracy of machine learning models under the conditions of low amounts of annotated training/testing data as well as the lack of correctly defined data distributions in the context of autonomous driving applications.

Supervisor: Josef Sivic (Czech Technical University)
Industry supervisor: Patrik Perez (Valeo.ai)

PhD Duration: 01 October 2019 – 01 September 2023

Diana Sungatullina
ELLIS PhD student

Learning to solve multiple-view geometry
Multi-view geometry is an important field in computer vision and robotics, which provides an understanding to the foundations of the subject. The development of this field has, however, not yet been greatly influenced by the recent advances in machine learning and geometrical machine learning in particular.

Supervisor: Tomas Pajdla (Czech Technical University)
Co-supervisor: Konrad Schindler (ETH Zürich)

PhD Duration: 01 February 2022 – 01 February 2026
Exchange Duration: 01 February 2024 – 01 August 2024

Erik Derner
ELLIS Postdoc

Towards Responsible and Ethical Large Language Models
The objective of this project is to contribute to the development of safe, secure, reliable, trustworthy, fair, accessible, and user-friendly language models. The research will include a case study focused on the Valencian language, emphasizing the societal and cultural importance of supporting regional languages.

Supervisor: Nuria Oliver (ELLIS Alicante Unit Foundation | Institute of Humanity-centric AI)
Co-supervisor: Robert Babuska (Delft University of Technology & Czech Technical University)

PostDoc Duration: 01 September 2023 – 31 August 2025
Exchange Duration: 15 December 2023 – 31 January 2024 | 15 December 2024 – 31 January 2025

Gabriele Trivigno
ELLIS PhD student

Attention-guided cross domain visual geo-localization
Photo geolocation is a challenging task since many photos offer only few cues about their location. For instance, an image of a beach could be taken on many coasts across the world. he goal of this PhD is to study the problem of visual geo-localization across visual domains. We will leverage over the intrinsic spatial connotation of place images and combine attention mechanisms with modern domain adaptation algorithms, in order to obtain perceptual representations that can be used for visual place recognition, as well as for content based image retrieval, able to close the domain gap differently on different parts of the images.

Supervisor: Barbara Caputo (Politecnico di Torino & Italian Institute of Technology)
Co-supervisor: Torsten Sattler (Czech Technical University)

PhD duration: 01 November 2021 -01 November 2024
Exchange Duration: 01 February 2024 – Ongoing | 01 August 2024 – Ongoing

Jonas Kulhanek
ELLIS PhD student

Towards A Unified 3D Scene Representation
3D scene representations, or 3D maps, are an essential component of a wide range of intelligent systems, such as self-driving cars, robots, or virtual reality. A fundamental limitation of the current approaches is, however, that they are designed for a specific sensor setup which makes them difficult to share between applications. This could be enabled by building a database of 3D geometry parts which can be queried efficiently. We hope that our 3D scene representation will bridge the barriers between different modalities and will enable large-scale applications of systems which would otherwise require difficult-to-obtain data.

Supervisor: Torsten Sattler (Czech Technical University)
Co-supervisor: Marc Pollefeys (ETH Zürich & Microsoft)

PhD Duration: 01 September 2021 – 31 August 2025
Exchange Duration: 01 January 2024 – 30 June 2024

Vladimir Petrik
ELLIS Postdoc

Learning Robotic Manipulation from Instructional Videos
The objective of the project is to enable robots to learn new manipulation skills from instructional videos available online. We will study how instructional videos could be used to overcome the sparse reward problem in reinforcement learning. The sparse reward complicates reinforcement learning by assigning the reward only after the task completion. Therefore, if the task is not completed successfully, there is no gradient guiding the reinforcement learning agent towards the goal. State-of-the-art approaches use human demonstrations to initialize the learning procedure. We would like to replace the human demonstrations with instructional videos that are already available, e.g. on Youtube, for the desired task. Although the task is the same, the key challenge is that the environment depicted in the video is not modeled in the simulation exactly. Instead, we will study how to modify the learning procedure to overcome differences in the simulator, real robotic testbed, and the video instructions.

Supervisor: Josef Sivic (Czech Technical University)
Co-supervisor: Ivan Laptev (Inria)

PhD Duration: 01 April 2019 – 31 March 2020
Affiliation: Czech Technical University in Prague

Zehao Yu
ELLIS PhD student

Holistic 3D Scene Understanding for Self-Driving Cars
Holistic 3D scene understanding plays a critical role in self-driving cars. It involves several sub-tasks, such as geometric layout estimation, object detection, recognition, and tracking. While each sub-task may be solved independently, it would be beneficial to utilize the complementary nature of different sub-task and model the 3D scene in a holistic way. This project will focus on research problems to achieve this goal, such as representing the 3D scene compactly and finding a unified representation that can model scene geometry, sematic, and dynamics, and effectively and efficiently infer such 3D scene representation sensor inputs.

Supervisor: Andreas Geiger (University of Tübingen & Max Planck Institute for Intelligent Systems)
Co-supervisor: Torsten Sattler (Czech Technical University)

PhD Duration: 01 September 2021 – 30 June 2025
Exchange Duration: 01 December 2022 – 31 May 2023