The Smart Car Revolution (or the Automotive Industry’s Big Data Problem)

The smart car revolution has just begun. This week Tesla announced their high-volume, lowest-cost model Tesla 3 and 300,000 customers have already ordered it in advance. Smart cars (that is electric, autonomous and connected automobiles) like the ones Tesla manufactures, are what customer are willing to pay for. But how is it possible for Tesla to disrupt the well-established and mature market of car manufacturing? The reason is that the majority of traditional car manufacturers didn’t do their “big data homework” by delegating this to their suppliers. As a result, suppliers became big data companies leaving behind the car manufacturers themselves.

In this post, I will explain how car manufacturers can overcome their current limitations in order to become big data companies producing smart cars like Tesla. These insights are the results of my experiences at Volkswagen, where I was an IT architect for mobile online services.

Smart cars

In the near future, cars will become more than just mechanical objects bringing customers safely and quickly from point A to B. Currently, the focus is on mechanical improvements to safety features such as airbags or reducing fuel consumption and emissions. Soon, however, the focus will switch to the electrical and IT aspects of cars. The automobile will become a rolling computer, providing added value via autonomous driving and an infotainment system which makes the resulting leisure time more productive and enjoyable. Thanks to improvements in electric motor technology, the complexity regarding manufacturing has decreased significantly. This means that enterprises such as Google or Apple—companies that develop sophisticated computer, smart phone and infotainment platforms—will be more involved in important aspects of car production than traditional car manufacturers such as Volkswagen. As often stated, based on valuable information stemming from big data analytics, future cars will be electric, autonomous, connected, on the whole smart.

You can regularly read in newspapers about the progress car manufacturers and their suppliers are making regarding the first three points. There is not much conversation, however, regarding the coming smart car revolution. It is obvious that the industry has recognized the risk of losing future market share by failing to evolve into big data enterprises based on recent acquisitions, investments and partnerships they have made, and their hiring spree of data scientists. BMW, Audi, and Mercedes, for example, just bought Nokia’s mapping service, and BMW has undertaken a massive recruitment of data scientists on LinkedIn. Nevertheless, time is running out. If car manufacturers don’t start to take the reins now, they will miss out on the smart car revolution of tomorrow.

Structural limitations of the automotive industry

Let's start by examining the automotive industry's structural challenges. In order to become big data companies producing smart cars, car manufacturers must change their perspective and the way they manufacture cars. The truth is that car manufacturers have evolved over time into systems integrators and contract manufacturers. For each particular car model, they provide their suppliers a master specification that addresses assumed market needs and follows government regulations such as requirements regarding safety, emissions and so on. They expect their suppliers to develop and provide the components and subsystems that fulfill the specification's requirements. The supplied components are finally assembled into a vehicle based on an extremely optimized and efficient process.

This approach results in much of the innovation and knowledge regarding how to address these requirements remaining with the individual suppliers, rather than being passed on to the car manufacturers. The graphics card company NVIDIA (better known for designing graphics processing units for the gaming market), for example, has become one of the leaders in data processing and data mining for self-driving cars with Level 4 autonomy (watch this great video about how they utilize a deep neural network from minutes 1:40 to 3:00). Level 4 autonomy means that the vehicle performs all safety-critical functions for the entire trip, with the driver not expected to control the car at any time. Similarly, the user experience of each such subsystem, e.g., infotainment (which is still a relatively unknown source of excellent margins), is managed by the supplier rather than the manufacturer. Consider, for instance, the control Tesla has regarding the user experience of their infotainment system in comparison to Volkswagen, which integrates an infotainment system from Bosch. As confirmed here, Tesla keeps much of the design and implementation in-house.

If car producers want to understand the components they integrate and perform big data analytics on the data these components provide, they must first understand their data, an essential prerequisite. More specifically, they have to understand how the data is produced and filtered. They also must develop such expertise regardless of the level of autonomy offered in their cars. Car manufacturers must get their hands dirty, and involve themselves in the production of all components that produce valuable data. Audi's, BMW's and Daimler's recent purchase of "NOKIA here" underlines this fact. Building up data expertise won’t be easy because car manufacturers must first understand the difference between customer-centric mobility and traditional manufacturing-centric car production, involving welding, molding and assembling components.

Customer-centric mobility is based on data. Generating individual navigation routes, for example, can only be accomplished when the information gathered by one car about, for example, traffic jams is shared with all others. That is why car manufacturers will have to accept that they are entering the big data business. They must be able to create valuable information, e.g., generating predictive models for engine failure or build systems that utilize big data for generating real-time map data for autonomous driving and transmit them to their fleet. Additionally, they have to integrate third-party data such as weather data to offer innovative services around electronic vehicles such as real-time range calculations based on current weather and conditions.

Nowadays, cars possess hundreds of embedded microprocessors. Such a car produces approximately 20 Megabytes of data per day, which equals 15 minutes of music compressed as an MP3 file. Compared to companies such as Google, Uber, or Tesla, however, traditional car manufacturers are not able to leverage this data. Indeed, only a small percentage is accessible via the car’s internal bus system, called CAN bus. Regarding online services, only a fraction of it is actually used, for instance, for mobile online services such as sending vehicle status report to local shops, thereby helping mechanics to diagnose problems more quickly.

Now imagine for a second how much data will be generated by autonomous, connected cars. Such cars will probably have many more sensors such as radar or lidar—two methods used to measure distances with microwaves and laser lights, respectively— and camera sensors for exploring their environment. All of these sensors will constantly generate data.

The vehicle’s infotainment system is another component which consumes huge amounts of data like map updates, entertainment content, etc. Such cars will produce (and consume) GBytes of data per day, which is at least 1,000 minutes of music in MP3 format, a significant increase in comparison to the 20 Mbytes mentioned earlier. On top of this data, you have to account for the data produced by car-to-car communication required by autonomous cars for exchanging environmental information. Car manufacturers must therefore become proficient with big data as quickly as possible. As already stated, some of them have taken steps to gain data expertise primarily through hiring, acquisitions, as well as investments in and partnerships with relevant companies. Some recent cases include Delphi’s purchase of the Carnegie Mellon spinoff Ottomatika, General Motors’ acquisition of Sidecar, and BMW’s investment in the smartphone analytics company Zendrive. Developing such expertise, however, will not be easy for today's car manufacturers, as the big data and information industry is very different from the manufacturing business.

To cope with the aforementioned problems, car manufacturers have to take action. The following three points are crucial for them:
a) Defining and executing a big data strategy
b) Restructuring the car’s system architecture to meet the requirements of big data
c) Pave the way for data management and sharing

Defining and executing a big data strategy

As laid out in my previous post, large enterprises such as car manufacturers have to set up a big data strategy to align their business and big data objectives. New players in the automotive industry, such as Google, Tesla or Uber, as well as start-ups like Faraday Future, Atieva, Renovo Motors and Divergent Technologies, have noticed how important big data is and developed such a strategy. Furthermore, they’ve made it a crucial part of their overall company strategy. Some companies in the field of autonomous driving specifically have begun to define what data is needed and how often predictions have to be updated for autonomous driving. Additionally, they have begun investigating which pieces of data are needed to model the vehicle's performance or the driver's mood, or how they can achieve a 360-degrees view of the vehicle and its surroundings. They have also examined which data-driven applications will be developed internally and which will be bought from external partners. Most of them work with data science organizations to further adapt their big data strategies to their individual use cases.

Car manufacturers must likewise devise a big data strategy and set up a system containing all in-vehicle and backend hardware and software, as well as the associated data-driven applications. They have to be in control of this system to be able to understand all data flows and adapt them to their needs. There is no other way that automakers can gain the overall knowledge they need to execute a successful big data strategy.

To understand how IT can evolve from a passive service provider into a division that actively drives innovation, car manufacturers should look at other industries where IT plays a significant role in the value-added chain, such as telecommunications, aerospace, logistics, and maybe even financial services.

Restructuring the car’s system architecture

More and more microprocessors have found their way into our cars in recent years. A state of the art car may have hundreds of microprocessors. Since car manufacturers build their current car architecture by writing specifications for every component and selecting the best supplier for each of them, the end result is a patchwork without central architecture. And even worse, the system architecture of a car model is not brought to the following generation. For every new generation, the same process of writing specifications and selecting new suppliers must be started from scratch. Additionally, components are sized in such way as to fulfill the today’s requirements while ignoring potential future developments, which may arise due to system updates, bug fixes or big data use cases. Tesla, for example, brought an autopilot mode to their cars via a remote system update, which is only possible if you size the car's components so they may be adapted to tomorrow's use cases. Furthermore, car manufacturers have to overcome the problem of developing the car architecture in distinct departments (or silos) and not holistically. Nowadays the in-vehicle architecture is designed separately from the backend architecture. The backend architecture is mostly considered another component. And even stranger, the internal IT department as well as external suppliers give a quote for these requirements. So, another patchwork in the backend is created. Welcome to what I term "death by complexity."

Car manufacturers can solve this problem by creating a unified in-vehicle and backend architecture which serves their vehicles' needs. They must take into consideration both the in-vehicle data, as well as data that is created and stored outside the vehicle, in data centers managed by their suppliers and partners. The in-vehicle components should be remotely updateable and sized to provide enough processing power for tomorrow's use cases. The architecture should be driven by the parts of the value chain which the automakers want to own. Before being able to establish and control such an architecture, car manufacturers will probably have to partner with start-ups like Renovo Motors. By doing so, they will be able to ensure that their vehicles are properly equipped and have access to all necessary data. Renovo Motors has already partnered with some manufacturers to create initial versions of such a unified architecture; I may give a more detailed view on the requirements of such an architecture in an upcoming post.

Paving the way for data management and sharing

By offering mobile online services, car manufacturers will for the first time come into direct contact with their customers. In the past, cars they produced were shipped to dealers who acted as the point of contact with customers. This meant that dealers held all personal data regarding their customers. The aftermarket remains a very strong source of income. This is the reason why car manufacturers are usually unwilling to share collected data, which may contain information about their customers. Additionally, the larger a car manufacturer is, the lower their readiness to assume risk. Sharing data can always result in problems when the data is not processed by the data receiver according to governmental regulations and legal norms. Following these rules, however, complicates data processing, a double-edged sword.

Therefore, suppliers follow the same data sharing strategy to minimize risk and uncertainty. The result is that nobody shares data with each other, although rich data value chains and use cases could be created which would benefit everyone. Data privacy is a particularly sensitive topic. Volkswagen, for example, is able to capture my GPS position when I start my car's engine in the morning and determine where I live. This problem can only be solved by pseudonymization or creating data management platforms (have a look at this patent which I filed during my time at Volkswagen), where customers can configure which data they want to share with car manufacturers, used applications and so on.

It is very unlikely that individual car manufacturers will capture all the data they need for the complex mobility solutions of tomorrow on their own. For security reasons regarding autonomous driving, for example, they will have to exchange information on where traffic jams occur with other manufacturers. Additionally, Google, Apple and many start-ups collect data from infotainment systems which they offer car manufacturers. Currently, car manufacturers negotiate with each other regarding patents they own. The future currency won’t be patents, but data. Otherwise, I think car manufacturers will be excluded from the valuable big data use cases of tomorrow. The question is whether auto manufacturers want to become the Foxconn of car manufacturing and merely observe the data passing through their cars or take control of the data chain. By taking control, they could start to leverage the data for new products and valuable services for them and their customers.


Leveraging big data is a huge challenge for the today's car manufacturers since the big data business is very different from the manufacturing business. Car manufacturers must be in control of their big data strategy, create unified architectures from the ground up which combine the vehicle and the backend, and allow for data management to succeed in the information industry. In order to build rich data value chains (like Tesla) they have to partner with start-ups or other suppliers in the automotive value chain.

Dr. Michael Nolting

Read more posts by this author.

Subscribe to Big Data - Automotive - Analytics

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!