The purpose of Tesla AI Day is to get the world excited about what Tesla is doing in artificial intelligence beyond cars.
AI day is also a recruiting event for prospective engineers as the company ramps up hiring.
Key Takeaways from Tesla AI Day 2021
Make useful AI that people love, and is unequivocally good. – Elon Musk
- Vertical integration is a common theme in the presentation. For both software, hardware, neural net training, and more. This means that Tesla builds and designs a large percentage of their technology in house.
- The company is able to auto label data sets as well as create simulation data sets with unlimited scenarios for training the neural network.
- DOJO is Tesla’s supercomputer designed for one purpose – training neural networks. It will be in use and available next year.
- The neural net architecture resembles the visual cortex of an animal.
- They will build a humanoid robot (see Tesla Bot at right)
- TLDR: skynet is born? Hopefully not. Although Elon said that human-level superintelligence is certainly possible, both the car and the Tesla Bot are examples of building “narrow AI” to avoid AI being misaligned with humans.
FSD beta version 9
FSD (Full-self-driving) is the autonomous system that is deployed to all cars, which customers can purchase for around $10,000.
The often debated fact that Tesla does not use LIDAR, as do many other autonomy oriented companies like GM cruise or Google Waymo, means that its cars use only cameras to gather data about the surroundings and navigate the world. Although the company mentioned plans to upgrade the cameras, the current cameras are still more than good enough.
The philosophy behind this decision is that roads were built for human eyes to see and navigate. Therefore, the cars should be able to gather sufficient data to navigate autonomously using only cameras.
Elon jokingly stated that because of this, someone could technically wear a T-shirt with a stop sign on it, and the car would stop. But ultimately, the company seems confident that cameras will be sufficient.
“It’s clearly headed to way better than human. Without a question.” – Elon Musk
Note: Tesla cars are not yet fully autonomous. Drivers still need to keep their attention and focus on the road at all times. 
Tesla still has yet to reach High Driving Automation level of autonomy (known as Level 4 Autonomy)
FSD driver-assist benefits:
- Navigate on Autopilot
- Auto Lane Change
- Traffic Light and Stop Control
Neural Net Architecture
There are 8 cameras surrounding the video that capture images of the real world. Tesla’s system uses these images to create a 3D reconstruction of the scene in “vector space”.
Using these images and vector space rendering, the system makes predictions about what the car may encounter a few moments into the future, allowing the car to drive itself safely and without running into anything.
As the neural network improves as it is trained on more and more data, Tesla is slowly building a brain-like neural net that resembles the visual cortex of an animal.
The presenters mentioned that everything Tesla is building is fundamentally country agnostic. Although they are optimizing the neural net models for the US at this point, they will be able to extrapolate to other countries as well in the future.
The ability to plan allows the car to predict and make changes about what other cars are doing on the road in real time.
Predictive and planning themed capabilities aside, the upper limit of the neural network has enough power to remember all of the roads and highways on planet Earth.
The presentation spent a significant amount of time diving into the specifics of Tesla’s Neural Net Architecture. For specifics, watch the replay of the 2021 AI day livestream.
Training Neural Networks – Data Required
Every time a human driver gets inside a Tesla, they are helping to train the neural network. Although this may make an incremental improvement, this is not enough training data.
These networks have hundreds of millions of parameters – it is incredibly important to get as many data sets as possible to create 3D renderings in vector space.
Millions of labels are needed, and each piece of data is essentially just a small video clip. Associated with each clip, you have the actual image/video data, odometer information, GPS coordinates, and more.
Tesla needs millions of vector space data sets to train these neural networks. In the spirit of vertical integration, there was formerly a team at Tesla that tediously labeled all of that data. But manual labeling proved to be too slow – there is a better way.
Auto Labeling Data Sets
Tesla developed an auto labeling system, allowing them to generate extremely large training data sets much faster for training the neural network. The auto labeling mechanism is extremely important.
“Without auto labeling, we would not be able to solve the self driving problem.” – Elon Musk
In addition to real-world data sets from camera footage, Tesla also is creating simulations of traffic scenarios.
It is like a video game, where Tesla Autopilot is the player. Simulation is helpful when data is difficult to source, difficult to label, or is in a closed loop.
Algorithms are able to create the simulation scenarios. These algorithms analyze where the system is failing, and then create more data around the failure points to allow the neural network to learn, improve, and handle those scenarios better in the future.
Elon specifically discouraged the use of machine learning because it is extremely difficult, and largely not the right solution for most use cases.
Project Dojo, Tesla’s Supercomputer
Dojo is the name of the neural network training system. Like a training Dojo.
Given all the data and simulations required, there is a demand for speed and capacity in AI neural network training. This is where Dojo comes in.
Currently, it is difficult to scale up bandwidth and reduce latencies, because processors have not been traditionally designed for training neural nets.
This is why Tesla invented the DPU.
DPU – Dojo processing unit. Whereas CPUs and GPUs are not designed to train neural networks, the DPU is designed to train neural networks.
The goal is to achieve the best possible AI training performance, supporting larger complex models while being power efficient and cost effective. Elon said it will be available next year.
This effectively enhances the AI software system, improving FSD.
Dojo leverages a distributed compute architecture.
It is apparently capable of an exaFLOP, which mean is can do a super high number of calculations per second, way more than your average computer. It is a supercomputer, after all…
Tesla will also make Dojo available to other companies that want to train their own neural networks, effectively building a platform for improving neural networks. This feels like an optimum opportunity to apply the “as-a-service” business model to the world of artificial intelligence and neural network training. By licensing out the use of Dojo, Tesla may be able to create yet another revenue stream for the company.
Thay have reportedly innovated in these chips in a way that means there are no roadblocks to extremely high bandwidth.
The software stack is completely vertically integrated. They build everything in house.
As the Dojo computer and neural network data sets improve the neural network, it is likely that the company will deploy the improved brain-like software upgrades via their over-the-air software updates.
Hardware and Computer Chips
One of the biggest goals is minimize latency, and maximize frame rate. These metrics may be familiar to you if you are involved in video games & graphics.
There is a computer in the car that runs the neural network that has been trained by the massive data sets discussed above.
Allegedly, the computer chip for Tesla’s Full Self-Driving system are produced by Samsung. 
The importance of computer chips to Tesla cannot be overstated. Various computer chips are used in all areas of the vehicle – even in the typically non-tech intensive parts of a car: computerized airbags, seat belts, doors and door handles, etc.
Given the reliance on them, the computer chip and semiconductor shortage is an issue across the globe is certainly a hurdle to resolve. 
Tesla Bot – known endearingly by Elon as “Optimus subprime” – a 5 foot 8 inch, 125 pound humanoid robot.
Given that the Tesla car is already essentially a robot, the Tesla Bot will simply use all the same technologies, in a device with a shape like that of a human.
It will make use of all the same tools that Tesla has in the car… such as 8 cameras, FSD computer, etc.
Elon was unfortunately reluctant to share any specific use cases, other than stating vaguely that it will do boring, repetitive, and dangerous tasks that humans do not want to do.
There are still many unknowns. Will the Tesla Bot have features similar to Siri or Amazon Alexa / Echo?
In addition to being a large automaker, Tesla is showing that they are very much a robotics, artificial intelligence, and software company.
Disclaimer: TSLA shareholder
Sources and references
- Criticism of Tesla (wikipedia article)
- Tesla.com AI day
Get the Future of Technology letter each month. Sign up below.