• ECCE
  • School Edu
  • Higher Edu
  • Edu Tech
  • Skills
  • Jobs
  • Advertise
  • Login
[t4b-ticker]
  • Home
  • Latest
  • Spotlight
  • Perspective
  • Interview
  • Videos
  • Events
  • Shop
  • Student Kiosk
No Result
View All Result
  • Home
  • Latest
  • Spotlight
  • Perspective
  • Interview
  • Videos
  • Events
  • Shop
  • Student Kiosk
No Result
View All Result
Education21
Home Spotlight

When Modern AI Meets Classical Robotics

Prof. Madhava Krishna , Professor and Head of the Robotics Research Centre and the Kohli Center for Intelligent Systems (KCIS) at IIIT Hyderbad, briefly explains the way IIITH’s self-driving car works and what is novel about it

education by education
November 25, 2024
in Spotlight
0
When Modern AI Meets Classical Robotics
0
SHARES
60
VIEWS
Share on FacebookShare on Twitter

IIIT Hyderabad’s Self Driving Car is an electric vehicle that performs point to point Autonomous Driving with collision avoidance capabilities over a wide area. Equipped with 3D LIDAR, Depth Cameras, GPS systems and AHRS (Attitude and Heading Reference System which essentially means sensors on three axes to estimate it’s orientation in space), the car can also accept Open Set Natural Language commands and follow those commands to reach a desired destination. SLAM-based point cloud mapping is used to map the campus environment and a LIDAR-guided real-time state estimation allows for localization while driving. Trajectory optimization frameworks a-la Model Predictive Control provides for real-time rollout of optimal trajectories. Such trajectories can be initialised with data-driven models for faster inference time optimization. A number of publications in high-profile venues and highly competitive conferences decorate the research landscape of Autonomous Driving research at the International Institute of Information Technology, Hyderabad, Gachibowli Campus.

Open Set Navigation

To better understand open set navigation commands, let’s first consider how humans often navigate – with minimal map usage, relying on contextual cues and verbal instructions. Many navigation directions are exchanged based on recognizing specific environmental landmarks or features, for example, “Take right next to the white building” or “Drop off near the entrance”. In a parallel context, autonomous driving agents need precise knowledge of their pose (or localization) within the environment for effective navigation. These are typically achieved using high-resolution GPS or pre-built High-Definition (HD) Maps like Point Cloud, which are compute and memory-intensive

Alternatively, many efforts utilise open-source topological maps (GPS-like Maps) for geolocalization, like OpenStreetMaps (OSM), as a lightweight solution. However, they are metrically inaccurate (~6-8 meters) and suffer localization errors when navigating to arbitrary destinations within the map. Moreover, some environmental landmarks, such as open parking spaces, may not be marked in OSM maps due to their dynamic nature. Thus, at IIITH, we are interested in exploring a feasible and scalable method to localise using solely real-world landmarks, akin to human navigation.

IIITH’s Efforts

We intend to address it by exploiting foundational models that have a generic semantic understanding of the world which can be distilled for downstream localization and navigation tasks. This has been achieved by augmenting open-source topological maps (like OSM) with language landmarks, such as “a bench”, “an underpass”, “football field”, etc. which resemble the cognitive localization process employed by humans. These enable an “open-vocabulary” nature that allows navigation to places for which the model is not explicitly trained, leading to a zero-shot generalisation to new scenes. The RRC Autonomous Driving group- AutoDP at IIITH demonstrates an attempt to integrate classical methods with post-modern solutions in open-world understanding, bringing the best of both worlds to solve the ever-challenging task of precise localization and navigation with real-world deployment through the in-house developed prototype.

Differential Planning

Mapping, localization, and planning form the key components in the Autonomous Navigation Stack. While both the modular pipeline and end-to-end architectures have been the traditional driving paradigms, the integration of language modality is slowly becoming a defacto approach to enhance the explainability of autonomous driving systems. A natural extension of these systems in the vision-language context is their ability to follow navigation instructions given in natural language – for example, “Take a right turn and stop near the food stall.” The primary objective is to ensure reliable collision-free planning. Traditionally, upstream predictions and perception are customised for improving the downstream tasks, which is typical in current Vision-Language-Action models and other existing end-to-end architectures. However, prediction and perception components are often tuned with their own objectives, rather than the overall navigation goal. In such a pipeline, the planning module heavily depends on the perception abilities of these models, making them vulnerable to prediction errors. Thus, end-to-end training with downstream planning tasks becomes crucial, ensuring feasibility even with arbitrary predictions from upstream perception and prediction components.

Our USP: NLP+VLM
To achieve this capability, IIITH has developed a lightweight vision-language model that combines visual scene understanding with natural language processing. The model processes the vehicle’s perspective view alongside encoded language commands to predict goal locations in one-shot. However, these predictions can sometimes conflict with real-world constraints. For example, when instructed to “park behind the red car,” the system might suggest a location in a non-road area or overlapping with the red car itself. To overcome this challenge, we augment the perception module with a custom planner within a neural network framework. This requires the planner to be differentiable, enabling gradient flow throughout the entire architecture during training which eventually improves both prediction accuracy and the planning quality. This end-to-end training approach with a differentiable planner serves as the key sauce of our work.

  • Facebook
  • Twitter
  • Instagram
  • LinkedIn
  • RSS Feed
Previous Post

Rice University’s expansion into India with Rice Global India underscores India’s growing influence

Next Post

The University of Tasmania invites applications for Master of Marine and Antarctic Science

education

education

Related Posts

Maharashtra organizes Maharashtra Bal Sansad 2026 on World Children’s Day in Pune
School Edu

Maharashtra organizes Maharashtra Bal Sansad 2026 on World Children’s Day in Pune

by education
November 20, 2026
Sat Paul Mittal National Awards 2026 honour four individuals and institutions for service to humanity
Events

Sat Paul Mittal National Awards 2026 honour four individuals and institutions for service to humanity

by education
November 12, 2026
National Education Day: More Questions Than Celebrations as Ad Hoc Governance Puts Education System in Limbo
Perspective

National Education Day: More Questions Than Celebrations as Ad Hoc Governance Puts Education System in Limbo

by education
November 11, 2026
Small Schools, Big Questions: Former Education Secretary Sanjay Kumar reveals India’s Schooling Challenge
School Edu

Small Schools, Big Questions: Former Education Secretary Sanjay Kumar reveals India’s Schooling Challenge

by education
November 1, 2026
Capacity-Building Programme for Medical and Allied Healthcare Professionals in Drug De-addiction Services Launched
Spotlight

Capacity-Building Programme for Medical and Allied Healthcare Professionals in Drug De-addiction Services Launched

by education
October 27, 2026
Next Post

The University of Tasmania invites applications for Master of Marine and Antarctic Science

  • Facebook
  • Twitter
  • Instagram
  • LinkedIn
  • RSS Feed

Useful Announcements

  • All
  • Useful Announcements

Shiv Nadar School student releases documentary inspired by his autistic brother

November 22, 2026

NGO Brain Bristle announces audition of autistic and neurotypical actors aged 16 to 22 for its new play

November 22, 2026

TIET to host the Annual Techno-Cultural Festival ‘Saturnalia 2026’ from November 21–23

November 20, 2026

Eurogrip Tyres Invites Students and Creators to Participate in Torq Challenge 4.0 and Creator Challenge

November 18, 2026

DIDAC India & DIDAC Skills 2026 being held at the Bangalore International Exhibition Centre (BIEC), Bengaluru from 25–27 November

November 18, 2026
EY report on India’s Music Economy calls for stronger creator education and transparency

Accenture LearnVantage and Google Cloud to Bring AI Learning to More Than 1,000 Schools and Colleges in India

November 17, 2026

Target Learning Ventures Launches ‘Bhavna vs Boards’ Campaign

November 17, 2026

Former CJI, Dr. Justice D. Y. Chandrachud to deliver Memorial Lecture at SAI International Education Group

November 17, 2026

Sheffield Hallam University invites applications for MSc Physiotherapy for January 2027 intake

November 16, 2026

Ashoka University Opens 2027 Undergraduate Admissions, Offers 500 Scholarships

November 16, 2026

Download current issue Not available

https://online.pubhtml5.com/jlyo/bxvr/

Monthly Magazine : Feb 2024

Interactive (Quizzes/Surveys)

Start Monthly Quiz
Education21

An initiative in continuation of Curriculum Magazine, Education21.in, is a platform for New India that aspires to be a valuable innovator, partner and collaborator for a just and sustainable world. Continuing with our steady and 360 degree coverage of education system and skills landscape, we are here more oriented towards learner community.

Useful Links

  • About us
  • Team
  • Contact
  • Editorial opportunities
  • Subscriptions
  • Job Opportunities
  • Features

Important Links

  • Blog
  • Archives
  • People
  • Careerwise
  • Resources
  • Downloadable
  • Old issues

Newsletter

  • Facebook
  • Twitter
  • Instagram
  • LinkedIn
  • RSS Feed

Copyright © 2020 All rights reserved. Education21.

No Result
View All Result
  • Home
  • Latest
  • Spotlight
  • Perspective
  • Interview
  • Videos
  • Events
  • Shop
  • Student Kiosk

Copyright © 2020 All rights reserved. Education21.

Welcome Back!

Login to your account below

Forgotten Password?

Create New Account!

Fill the forms bellow to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In
Cleantalk Pixel