Data streaming: More bandwidth is needed before self-driving cars can fully function with autonomy

The variety of computing platforms and data make it difficult to fully integrate an autonomous car system in which all vehicles communicate with each other, expert says.

Image: iStock/metamorworks

More about artificial intelligence

Imagine a fully automated traffic grid that spans an entire city ecosystem—a grid where cars communicate with each other in real time, continuously processing and streaming data. You could do work on your iPhone or tablet as your car drives to work, and grandma wouldn’t have to worry about driving herself to the doctor anymore.

SEE: Managing the multicloud (ZDNet/TechRepublic special feature) | Download the free PDF version (TechRepublic)

This is the vision of the self-driving car—but to make self-driving cars a widespread reality, real-time data processing and analytics inside and outside of each car and improvements to bandwidth, cloud super-structures, road asset digitization, and safety and regulatory guidance must happen first.

So let’s level-set for where we are today. “Within the automotive industry, there are two areas we’re currently seeing the highest demand for streaming analytics,” said Przemek Tomczak, SVP of Internet of Things (IoT) and utilities at Kx Systems, which provides high-speed processing and analysis of real-time, streaming and historical data. “One demand is in research and development, including for testing entire vehicles and their parts in wind-tunnels, where analytics is performed in real-time during each test. The other is in manufacturing processes, in detecting deviations and controlling processes based on streams of data coming from sensors in the plant.”

SEE: Special feature: Autonomous vehicles and the enterprise (free PDF) (TechRepublic)

Examples of recent use cases include work by Aston Martin Red Bull and MIT Motorsport in the design and test of racing cars in wind tunnels and on test tracks. In the Aston-Martin case, more than 100 sensors on a car captured, integrated, and analyzed vast quantities of data to provide real-time actionable insights that improved racing performance.

This work is getting results, but moving up from there requires additional work, beginning with improving interoperability and security across different subsystems inside each vehicle.

“For data streaming with cars, we want to combine, enrich and analyze data streams that exist in a heterogeneous computing environment,” Tomczak said. “This variety of computing platforms and data, combined with the velocity of large data volumes per vehicle, makes it extremely difficult to push all generated data to the cloud for processing and analytics.”

Tomczak noted that industry consortiums have formed to help improve cooperation and standards around connectivity, interoperability, and security, but that other challenges still remain.

SEE: Yellowstone National Park plans test of self-driving shuttles to ease traffic jams (TechRepublic)

“One challenge is that you can’t guarantee high-quality communications with sufficient bandwidth and low latency all of the time,” he said. “To that end, a lot of the data processing has to be performed in a vehicle with limited computational resources available. Consequently, taking technologies that have traditionally worked in large-scale data centers and cloud platforms, does not lend itself to be scaled down while meeting demanding analytics on high-velocity and volume data streams.”

It’s a key point. For large-scale traffic grids to operate and for cars to intercommunicate, the cloud will certainly be needed, as will sufficient bandwidth with low latency. “The entire data pipeline and lifecycle of data through capture, processing, storage, and analysis must be optimized,” Tomczak said.

SEE: Self-driving cars: Will buyers come if you just build them? Maybe (TechRepublic)

Some key takeaways from this:

Historical data from legacy sources must be mixable with real-time streaming data for cars to interoperate with each other in an autonomous and self-sufficient mode.
Bandwidth, latency, and cloud technologies must also be built out further if they are to be a part of traffic grid architecture for self-driving vehicles.
Enough intelligence must be built into a real-time analytics process to handle thousands of scenarios that can happen in traffic.

“Streaming analytics for industrial and consumer IoT applications needs to be able to handle disruptions in connectivity and communications, late and/or out-of-order data, and classical data retrievals for more complex calculations and aggregations,” Tomczak said. “With the demands for faster insights and decision-making, the velocity and volume of sensors and sensor data, organizations are having to look at different approaches for addressing these challenges.”