Watch: How connected and autonomous vehicles can manage data

The world of autonomous vehicles promises great change for how we live and work, with opportunity and disruption across a variety of contexts.

However, in order to fulfil its promise, autonomous vehicle providers will need to process the sheer wealth of data that will be produced. Without a firm grip on the data lifecycle, autonomous vehicles may fail to get started.

SmartCitiesWorld recently sat down with Michael Ger, Managing Director, Automotive and Manufacturing Solutions at Cloudera, to discuss the role of data in the connected and autonomous vehicle market. Below is an edited transcript of the video discussion.

SmartCitiesWorld: Cloudera has described the shift to autonomous vehicles as less of an automotive engineering challenge and more of a data analytics and machine learning one. Why is that?

Michael Ger, Cloudera: As we move towards autonomy, teaching a car to drive is a really data intensive proposition. That is the part that is really new and interestingly the automotive industry is struggling even to find skills in this area.

If you look at autonomous driving, it’s probably the mother of all machine learning use cases in terms of complexity. Because when you think about the sheer number of conditions that a self-driving car has to operate under - different weather conditions, in different countries, different local requirements - you are teaching those cores to operate in a failsafe manner across all those different types of conditions.

SCW: What is the autonomous data loop?

Ger, Cloudera: Imagine if you have all this data and want to train a car to behave a certain way going through an intersection. I need to be able to retrieve those types of frames...and create a machine learning model around that. You ingest data, store it, make it searchable, then once that data is in one place, you build out machine learning algorithms.

Then you have to test those models because you can’t just deploy them back. How do you test them? You test them in a simulation mode, against all those millions of miles of data that you collected. And then once those machine learning models are validated, we actually push that model back out into the car.

SCW: Given the relatively early stage that machine learning is at, how can it scale up to meet the requirements of autonomous vehicles?

Ger, Cloudera: I think the automotive industry has done a great job at getting to where it is right now; we’ve come a long way relatively quickly. But if you look at chapter one of autonomous driving, data management platforms were very proprietary, they were very closed. In parallel you’ve had commercial software vendors optimising pieces of that lifestyle with open and commercially supportable types of capabilities.

No one vendor can can manage that entire end to end lifecycle So it’s a combination of using commercial capabilities that are used across industries, lots of best practices and coming up with an appropriate and relevant partner ecosystem.