Learning
This project aims to expand the capabilities of robots by designing and implementing strong visual learning systems that will help robots understand the environment around them. It addresses important challenges in deep learning such as effective transfer learning, the role of probabilistic graphical models in deep learning, and efficient training and inference algorithms.
Team Members
Chunhua Shen
University of Adelaide, Australia
Chunhua Shen is a Professor at School of Computer Science, University of Adelaide. He is also an adjunct Professor of Data Science and AI at Monash University.
Prior to that, he was with the computer vision program at NICTA (National ICT Australia), Canberra Research Laboratory for about six years. His research interests are in the intersection of computer vision and statistical machine learning. He studied at Nanjing University, at Australian National University, and received his PhD degree from the University of Adelaide. From 2012 to 2016, he held an Australian Research Council Future Fellowship. He is Associate Editor (AE) of the Pattern Recognition journal, IEEE Transactions on Circuits and Systems for Video Technology; and served as AEs for a few journals including IEEE Transactions on Neural Networks and Learning Systems.
Tom Drummond
Monash University, Australia
Professor Drummond is a Chief Investigator based at Monash. He studied a BA in mathematics at the University of Cambridge. In 1989 he emigrated to Australia and worked for CSIRO in Melbourne for four years before moving to Perth for his PhD in Computer Science at Curtin University. In 1998 he returned to Cambridge as a post-doctoral Research Associate and in 1991 was appointed as a University Lecturer. In 2010 he returned to Melbourne and took up a Professorship at Monash University. His research is principally in the field of real-time computer vision (ie processing of information from a video camera in a computer in real-time typically at frame rate), machine learning and robust methods. These have applications in augmented reality, robotics, assistive technologies for visually impaired users as well as medical imaging.
Feras Dayoub
Queensland University of Technology (QUT), Australia
Feras is interested in the reliable deployment of machine learning and computer vision on mobile robots in challenging environments. Autonomous Underwater Vehicles (AUV) as the computer vision lead on the COTSBot project. Unmanned Aerial Vehicles (UAV) as part of a project on assisted autonomy during the inspection of power infrastructure. Mobile service robots as a research fellow on an Australian research council discovery project on lifelong robotic navigation using visual perception.
Niko Sünderhauf
Queensland University of Technology (QUT), Australia
Dr Niko Suenderhauf is a Chief Investigator of the Australian Centre for Robotic Vision, and a Senior Lecturer at Queensland University of Technology (QUT) in Brisbane, Australia (a Senior Lecturer is roughly equivalent to a junior Associate Professor in the US system).
Niko conducts research in robotic vision, at the intersection of robotics, computer vision, and machine learning. His research interests focus on scene understanding and how robots can learn to perform complex tasks that require navigation and interaction with objects, the environment, and with humans.
Stephen Gould
Australian National University (ANU), Australia
Stephen Gould is a Professor in the Research School of Computer Science in the College of Engineering and Computer Science at the Australian National University. He received his BSc degree in mathematics and computer science and BE degree in electrical engineering from the University of Sydney in 1994 and 1996, respectively. He received his MS degree in electrical engineering from Stanford University in 1998. He then worked in industry for a number of years where he co-founded Sensory Networks, which sold to Intel in 2013. In 2005 he returned to PhD studies and earned his PhD degree from Stanford University in 2010. His research interests include computer and robotic vision, machine learning, probabilistic graphical models, deep learning and optimization.
In 2017, Steve spent a year in Seattle leading a team of computer vision researchers and engineers at Amazon before returning to Australia in 2018.
Bohan Zhuang
University of Adelaide, Australia
Bohan joined the Centre as a Research Fellow after completing his PhD at the University of Adelaide in March 2018, supervised by Chief Investigators Chunhua Shen and Ian Reid. His research interest is in compressing and accelerating deep neural networks for resource constraint devices. And he also focuses on a wide span of applications in Computer Vision. He completed his bachelor degree in Electrical Engineering in July, 2014 at Dalian University of Technology, China. During his undergraduate study, Bohan worked with Prof. Huchuan Lu.
Michele ‘Mike’ Sasdelli
University of Adelaide, Australia
Michele Sasdelli’s original background is in physics. He has studied and worked in five countries both in academic and industry environments. He worked as a Postdoctoral Researcher at the Astrophysics Research Institute in Liverpool, focusing on deep learning applications. He was a research scientist at Cortexica Vision Systems, an AI company in London working on deep learning based algorithms for computer vision.
His interests lie in fundamental machine learning questions for computer vision and astrophysics. He is a science enthusiast and firmly believes in cross-feeding between different research fields. He joined the Centre in 2018 as a Research Fellow at the University of Adelaide working in learning theory for deep learning.
Ben Harwood
Monash University, Australia
Ben completed undergraduate studies in Computer Science and Mechatronics Engineering at Monash University before commencing a PhD in Computer Systems Engineering. Ben’s research was principally focused on efficient algorithms for processing high dimensional big data. This research has applications across a broad range of tasks from extracting information out of video streams, to finding meaningful connections between statements in a large corpus.
Ben has submitted his PhD for examination in 2018 and is now a Research Fellow with the Centre.
Benjamin Meyer
Monash University, Australia
Ben completed his PhD in August 2019 at the Australian Centre for Robotic Vision at Monash University, under the supervision of Tom Drummond. He is continuing his work at the centre as a research fellow. Ben’s research focuses primarily on deep learning, with particular interest in the problems of metric learning, novelty detection, open set recognition, active learning and generative models. He is now a Research Fellow in the Centre.
Yan Zuo
Monash University, Australia
Yan completed his PhD titled “Advances in Decision Forests and Ferns with Applications in Deep Representation Learning for Computer Vision” in 2019 under the supervision of Chief Investigator Professor Tom Drummond. During his PhD, his research focused on a family of learning algorithms categorised as ensemble learning methods. His work involved investigating methods for incorporating decision forests and decision ferns within deep learning frameworks and applying them to computer vision. These applications include a range of tasks including image classification, image segmentation, image synthesis and video prediction. He then joined the Centre as a Research Fellow in June 2019 and his current research is focused on solving problems in machine perceptron using learning approaches such as Generative Adversarial Networks and Reinforcement Learning for navigation and mapping within the Learning project of the Centre.
Dylan Campbell
Australian National University (ANU), Australia
Dylan joined the Centre as a Research Fellow at the Australian National University (ANU) in August 2018. Previously, he was a PhD student at ANU and Data61/CSIRO, where he worked on geometric vision problems, and a research assistant in the Cyber-Physical Systems group of Data61/CSIRO, where he worked on Resource Constrained Vision. Dylan received a BE in Mechatronic Engineering from the University of New South Wales. He has broad research interests within computer vision and robotics, including geometric vision and human-centred vision. In particular, he has investigated geometric sensor data alignment problems, such as camera localisation, simultaneous localisation and mapping, and structure from motion.
He is currently looking at the problems of recognising, modelling, and predicting human actions, poses and human-object interactions with a view to facilitate robot-human interaction as part of a Centre project.
Itzik Ben Shabat
Australian National University (ANU), Australia
Itzik joined the Centre as a Research Fellow at the Australian National University (ANU) node in July 2019. Previously, he was a Ph.D. student at Technion Israel Institute of Technology where he worked on “Classification, segmentation, and geometric analysis of 3D point clouds using deep learning” under the supervision of Prof. Anath Fischer and Michael Lindenbaum.
Itzik completed his Bsc. Cum Laude in 2008 (Mechanical Engineering, Technion) and his Msc. Summa Cum Laude in 2015 under the supervision of Prof. Anath Fischer (Mechanical Engineering, Technion). His research interests lie in the intersection of robotic perception, 3D computer vision, and geometric analysis, usually using 3D point cloud data.
Thalaiyasingam Ajanthan
Australian National University (ANU), Australia
Ajanthan joined the Centre as a Research Fellow in January 2019. Prior to this, he was a Postdoctoral Research Fellow at the Torr Vision Group at the University of Oxford from June 2017. Ajanthan obtained his PhD from the Australian National University in May 2017 and he was primarily supervised by Prof. Richard Hartley. During his PhD he was also a member of the Analytics group at Data61, CSIRO, Canberra. He holds a Bachelor’s degree in Electronic and Telecommunication Engineering from the University of Moratuwa, Sri Lanka. Ajanthan has broad interests in Graphical Models, Optimization Algorithms and Machine Learning.
He is part of the Centre’s Learning research project team.
Libo Sun
University of Adelaide, Australia
Libo joined the Centre as a PhD researcher in 2019 at the University of Adelaide, under the supervision of Professor Chunhua Shen. Before this he worked as a project officer and research associate at Nanyang Technological University, Singapore. His current research interests are mainly in the field of robotics, computer vision, and deep learning. In his free time, Libo enjoys hiking and travelling and has visited many places of interest including the Terracotta Warriors, The Great Wall of China, and Angkor Wat in Cambodia.
Rafael Felix Alves
University of Adelaide, Australia
Rafael joined the Centre as a Ph.D. candidate mid-year in 2016 under the supervision of Prof. Gustavo Carneiro and co-supervision of Prof. Ian Reid at The University of Adelaide. His research project aims to bring contributions to the fields of zero-shot learning using generative models. Rafael received his M.Sc. degree in Computer Engineering from the Universidade Presbiteriana Mackenzie (Brazil – 2015), and B.Sc. on Information Systems, Universidade Estadual de Montes Claros (Brazil – 2011).
His research interests include deep learning, zero-shot learning, open-set recognition, generative models, uncertainty, and optimization.
Vladimir Nekrasov
University of Adelaide, Australia
Vladimir joined the Centre as a PhD researcher in 2017. His thesis title is “Per-pixel image classification” supervised by Prof. Ian Reid and Prof. Chunhua Shen.
He received his specialist degree (BS and MS combined degree in Russia) in Mechanics and Applied Mathematics from Lomonosov Moscow State University, Russia in 2015. After that, he worked as a researcher on deep learning and started his PhD in computer science at UNIST, South Korea until March 2017 before moving to Australia.
Vladimir’s current research interests lie in deep learning: in particular, in its applications aimed to tackle computer vision tasks, such as dense per-pixel image classification, and its theoretical understanding.
Ming Cai
University of Adelaide, Australia
I am currently a PhD researcher supervised by Professor Ian Reid and Professor Chunhua Shen, with the focus on using deep learning for computer vision, especially, vision-based geometry tasks, such as 6D pose estimation for a camera and/or objects in the instance-level (or category-level).
Adrian Johnston
University of Adelaide, Australia
Adrian completed his undergraduate degree in Computer Science at the University of Adelaide before joining a research group for twelve months, where he worked on software engineering tools for the Defense Science Technology Group. He then returned to complete his honours degree in Computer Science, graduating with first class honours. Prior to beginning his PhD, he worked as a software engineer for the Australian Institute for Machine Learning (AIML). He commenced PhD studies in 2015 under the supervision of Professor Gustavo Carneiro. Adrian’s PhD research is focused on 3D object reconstruction using Deep Learning. He is currently, completing his PhD part-time while working for LifeWhisperer developing Computer Vision and AI techniques for IVF. He will complete his PhD in early 2020.
Max Xiaoqin Wang
Monash University, Australia
Max is currently a Data Scientist at AGL Energy in Melbourne, Australia. He completed his PhD at Monash University in December 2015 and was supervised by Associate Investigator Ahmet Sekercioglu and Chief Investigator Tom Drummond. His research interests include robotic vision, machine learning and distributed systems.
Gil Avraham
Monash University, Australia
Gil Avraham grew up in Israel and studied his Bachelor’s & Master’s in Electrical Engineering while serving at a technological unit in the Israel defense force throughout 2002-2012.
In 2013 he joined RTC-Vision, a company which specializes in providing computer vision solutions, there he worked as a computer vision engineer. In 2017 started his PhD at Monash University.
Luis Guerra Fernandez
Monash University, Australia
Luis Guerra completed a Bachelor of Electronics Engineering (2013) and a Masters of Science in Digital Signal Processing (2015) in Chihuahua, Mexico. During his undergraduate studies, he completed an internship at the Advanced Materials Research Center where he won a national research contest. Afterwards he entered the automotive industry as an embedded software developer for a year and then held a research position as Computer Vision Engineer in ADAS for another year. Luis started his PhD with the Centre at Monash University at the end of 2017. His research interests are focused on efficient heterogeneous deep learning implementations for embedded systems and constrained devices. He is supervised by Chief Investigator Professor Tom Drummond.
Alan Tianyu Zhu
Monash University, Australia
Alan completed a double degree in 2017 with a Bachelor of Engineering and Bachelor of Science at Monash University, graduating with first class honours. He is currently supervised by Chief Investigator Tom Drummond in the field of Computer Vision and Artificial Intelligence. His current research goal is to investigate how to use the advantage of attention mechanism and distance metric to improve the performance of metric learning applications.
Yunyan Xing
Monash University
Yunyan completed her Bachelor of Electrical and Computer System Enginnering degree at Monash University in 2017, graduating with first class honours. She started her PhD at Monash University in 2018 and is supervised by Prof. Tom Drummond. Her current research focuses on utilising deep reinforcement learning to improve video prediction.
William Hooper
Queensland University of Technology (QUT), Australia
William Hooper graduated from QUT in 2017 with First Class Honours in Bachelor of Electrical Engineering and Bachelor of Mathematics. During his degree he completed a Vacation Research position, learning how to build 3D realistic household interiors and modifying them to allow appropriate use in robotics applications. William is currently pursuing a PhD in the Centre, with a research focus on bringing his interest in mathematics and statistics to real-world robotics applications. His current research interests include; probabilistic generative models, causal reasoning and algorithmic complexity.
Dimity Miller
Queensland University of Technology (QUT), Australia
Dimity joined the Centre in 2018 after graduating from QUT in 2017 with First Class Honours in a Bachelor of Mechatronics Engineering. Dimity is currently completing her PhD on how to obtain uncertainty and robustness in deep learning for robotic vision. She is particularly interested in the reliability of deep learning in open-set conditions, where object classes that were not present in the training data are encountered.
Serena Mou
Queensland University of Technology (QUT), Australia
Serena started her PhD with the Centre in 2018 after being introduced to research by her supervisor, Centre CI Dr. Niko Sünderhauf, when she completed a Vacation Research Experience Scheme (VRES) project in 2016/2017. This led to an Honours project supervised by Dr. Sünderhauf “Learning to Navigate with Reinforcement Learning”. Along with her interests in robotics, she also has a passion toward the protection and conservation of the environment. Surveying is important for understanding and protecting ecosystems but is repetitive, time consuming and expensive. By using her knowledge of deep learning and its associated frameworks, robotic task design, and semantic representation she is focusing on efficient detection of flora and fauna from aerial surveillance.
Maruf Rahman
Queensland University of Technology (QUT), Australia
Maruf completed his bachelor’s and master’s degree from Department of Computer Science and Engineering, University of Dhaka, Bangladesh in 2012 and 2014 respectively. During his master’s degree, he started working in software industry and has gained five years of industry experience before starting his PhD with our centre in 2018. He is interested in the application of deep learning in robotic vision and is supervised by Research Fellow Dr Feras Dayoub and Centre Director Prof. Peter Corke. His research topic is to apply unsupervised machine learning in a mobile robot to detect novel objects.
Jordan Erskine
Queensland University of Technology (QUT), Australia
Jordan graduated from QUT in 2017 with first class honours in a Bachelor of Mechatronic Engineering. While there, he worked with QUT’s team for the Amazon Picking Challenge, as well as working on a CSIRO project involving developing autonomous surveying with UAVs. He started his PhD in 2018 and is supervised by Centre Research Affiliate and QUT Research Fellow Chris Lehnert, Research Fellow Juxi Leitner and Centre Director Peter Corke. His field of study involves developing and improving generalisable robotic manipulation skills.
Brendan Tidd
Queensland University of Technology (QUT), Australia
Brendan graduated from QUT in 2017 with a Bachelor of Engineering majoring in Mechatronics, achieving first class honors. In 2016 Brendan developed a tethered underwater vessel as part of QUT’s entry to the Robotx Maritime challenge and travelled with the team to Hawaii. TeamQUT competed against 13 universities from around the world, and took out second place.
Brendan joined the Centre in late 2017 under supervision of Dr Juxi Leitner and Distinguished Professor Peter Corke, with Dr Nicholas Hudson from Data61 at CSIRO joining his supervisory team in 2018. Brendan’s work uses visual perception with reinforcement learning to control dynamic legged robots in challenging environments. This topic lead him to work with CSIRO as part of a team funded by DARPA in their latest robotics competition, a subterranean challenge, where a team of robots (including tracked, legged and flying robots) commanded by a single operator must enter an underground scenario and localise target artefacts.
Tong Zhang
Australian National University (ANU), Australia
Tong Zhang joined the Centre as a PhD researcher at the Australian National University (ANU) in 2018. He obtained his masters degree from New York University in 2014 and his bachelor degree from Beihang University in 2011. He is mainly working on subspace clustering, weakly-supervised learning and generative models.
Medhani Menikdiwela
Australian National University (ANU), Australia
I’m doing my research under the supervison of Hondong Li and Chuong Nguyen. My research interest are Deeplearning, computer vision, Robotics and control systems. I did Bachelor of science and technology specialized in Mechatronics, Uva Wellassa University,Sri Lanka(2012). And followed master of science in robotics and control system at University of Moratuwa, Sri Lanka(2014). I have several conference publications relevant to haptic and vibration suppression of bilateral control systems. I’m currently working on object classification and detection by using deeplearning.
Amir Rahimi
Australian National University
Amir joined the Centre as PhD researcher at the Australian National University (ANU) in 2018. He is working under the supervision of Chief Investigators Richard Hartley and Stephen Gould. He has a B.Sc. degree in Software Engineering, and a M.Sc. degree in Artificial Intelligence both from University of Tehran, Iran. Throughout his career, he carried out an internship as a research assistant at Technical University of Dresden under the supervision of Prof. Carsten Rother and Dr. Bogdan Savchynskyy. His research interests are object detection, probabilistic graphical models and deep learning in general.
Cuong Nguyen
University of Adelaide, Australia
Cuong Nguyen is a PhD researcher at The University of Adelaide. He received his Bachelor degree from Portland State University, USA in 2012, and his MPhil degree from The University of Adelaide in 2018. He is interested in creating intelligent machines that can quickly learn from only a few examples. He joined the Centre in 2020 to research in meta-learning, a challenging field that builds algorithms for robots to quickly adapt to new tasks or new environments with a limited number of training examples. He is supervised by Centre Chief Investigator Gustavo Carneiro and is working in the Centre’s Learning Project.
Jing Zhang
Australian National University (ANU), Australia
Jing Zhang is currently a Ph.D student at the College of Engineering and Computer Science, Australian National University. She received the B.E. degree and M.E degree in signal and information processing from Northwestern Polytechnical University, Xian, China, in 2007 and 2010 respectively. She was a visiting student at ANU from Dec. 2014 to Dec. 2016 with the support of the China Scholarship Council. Her research interests include saliency detection, edge detection and semantic segmentation. She won the Best Student Paper Prize at DICTA 2017 and the Best Deep/Machine Learning Paper Prize at APSIPA ASC 2017.
Kartik Gupta
Australian National University (ANU), Australia
Kartik joined the Centre as an associated PhD researcher at the Australian National University (ANU) in 2018. He is currently working on object pose estimation and few-shot learning. Before joining ANU, Kartik worked in the computer vision industry for around 1.5 years. Kartik completed his MS (by Research) degree in computer science from Indian Institute of Technology (IIT) Mandi in 2017, working mainly in the area of computer vision. He also worked with Prof. Darius Burschka as a DAAD research scholar at Technical University of Munich (TU Munich) during his Masters. In 2018, he began his research at Australian National University and Data61 CSIRO under the supervision of Prof. Richard Hartley. His research interests are solving complex computer vision problems with least amount of data annotations.
Sadegh Aliakbarian
Australian National University (ANU), Australia
Sadegh is an Associated PhD researcher at our Australian National University node and a researcher at Smart Vision Systems, Data61, CSIRO. He is working on vision-based action anticipation. His method for action anticipation is particularly crucial in scenarios where one needs to react before the action is finalised, such as to avoid hitting pedestrians with an autonomous car. Currently, he is working on action anticipation in driving scenarios to predict human centric actions (e.g., driver maneuver, violating traffic rules, front car intention, pedestrian intention, and accidents with cars and pedestrians) before they actually begin to happen. Prior to starting his PhD, he was a researcher at NICTA. Before joining NICTA, Sadegh worked in the Computer Vision industry for more than two years. He also has a Bachelor of Science with honors in Computer Software Engineering.
Sadegh is also interested in other computer vision applications such as weakly-supervised semantic segmentation, usage of synthetic data in computer vision, and sequence learning.
Project Aim
This project aims to expand the capabilities of robots by designing and implementing strong visual learning systems that will help robots understand the environment around them. It addresses important challenges in deep learning such as effective transfer learning, the role of probabilistic graphical models in deep learning, and efficient training and inference algorithms. Answering these questions will allow us to design and implement strong visual learning systems that will help robots to understand the environment around them. Robotic vision is the key to success because it will enable robots to work competently in unstructured environments, where other sensors cannot work as effectively.
Key Results
In 2019, the project team developed influential learning algorithms that are likely to be used for visually guided robotic navigation systems. For example, the RefineNet algorithm (TPAMI 2019 publication) has become an important benchmark in the field, with its original paper (published in CVPR 2017) being cited almost 1,000 times.
The project team had five publications in the most prestigious computer science conference – the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). According to Google Scholar, CVPR is currently the second most prestigious publication outlet in the fields of Engineering and Computer Science, with an h5 index of 240. This places the Centre among the best research groups in the world in computer vision and machine learning.
Other papers published at NeurIPS and ICML (respectively 8th and 19th most prestigious publication outlets in Engineering and Computer Science) are helping to shape the next generation of machine learning algorithms. In particular, metric learning methods that are useful for building universal feature extractors that can be used by robots in previously unseen environments, and also on efficient deep learning models that can be deployed in robots with hardware constraints.
The team helped organise challenges in probabilistic object detection and uncertainty in deep learning for robotics. These challenges are defining benchmarks for the next generation of visual detectors for robots that will need to consider detection uncertainties.
Activity Plan for 2020
Generalised Zero shot learning
- Uncertainty for discovering/classifying new objects
- Collaboration with self-driving car demo
- Adapt metric learning for (G)ZSL (no need to build a classifier with a fixed number of outputs)
- Anomaly/novelty detection – failure prediction and performance measure
Semantic segmentation
- Deployment of the efficient models on real-world robots
- Uncertainty combined with ZSL for semantic segmentation
Sound processing for human-robot interaction
- Collaboration on Social Robotics Demo on automatic speech processing
Effective Deep Learning Training
- Bayesian learning using noisy training sets
- Bayesian meta-learning and meta-learning regularisation
- Model compression (binary weights, information bottleneck to estimate model compressibility)
Reinforcement learning
- Auto-encoder, attention mechanism
- Integrating semantics and world model knowledge
- Vision for walking, high level planning
- Prediction nets for RL
- Collaboration with the Manipulation Demo using the hierarchical skill net for manipulation – solve new tasks using a skill net