How does OpenPilot work?

OpenPilot works by taking the radar data integrated with many models (Honda and Toyota especially) and combining it with the camera built into the EON’s phone-based-hardware and frequently updated software to determine what acceleration, braking and steering events are required.

The actions are initiated/deactivated by the vehicle’s assist buttons and braking safeguards and control commands are sent through the vehicle’s BUS, emulating as if the vehicle’s OEM-based driver assistance program was sending the commands.

What is OpenPilot?

OpenPilot by Comma AI is a driver assistance application. The goal is to use Artificial Intelligence, Machine Learning and large datasets of video to come up with an advanced driver aid that makes driving safer and enjoyable.

What is the Hardware for OpenPilot?

OpenPilot has gone through several iterations during its evolution.  It began as a product called “NEO” which combined a 3D printed case with various boards and parts and utilized a OnePlus 3 phone for processing.

The phone hardware was upgraded to use a OnePlus 3T and the various boards and connectors were integrated into a product called “Panda” which contained the components to interact with the OBD functions and “Giraffe” which is a Make/Model specific piece of hardware to handle the various hardware differences across Makes and Models.

Eventually, the Panda was upgraded to a “Grey (High Precision GPS)” device with an external antenna and a high precision GPS chipset.

On October 2018, the Comma being selling a “Gold” version of the EON (called appropriately “EON Gold”) which is based on a LeEco LePlus3 (LEX727) Phone as an alternative to the aging OnePlus 3T phone.

Wait, OpenPilot runs on a cellphone?

Absolutely. Cellphone technology has improved leaps and bounds over the years and the quad core CPU in the phone is more than enough to handle the task. In addition, the GPU on the phone handles the video processing and the stripped-down Android Operating System is designed for maximum efficiency and is focused on the single task of driving the vehicle.

Think of the advantages of this approach. All of the processing and camera input and display are all integrated into a single thin unit. There are no external cameras to install or dedicated video screens or laptops required. It’s all there, centralized.

Are there any safeguards?

Absolutely. The OBD-based Panda is designed to be an emergency fail-safe in case the EON should experience critical failure. The Panda features Misra-c2012 compliance, tested against cppcheck coverage.

The Panda ensures you can override the automation at any time!

Do all OpenPilot based vehicles work identically?

Unfortunately, no. Because OpenPilot is an aftermarket replacement, it relies on the rules and safeguards built by the vehicle manufacturers.

If you look at the official list of supported Models, you will see a table with various values. The columns show the supported Make, Models and trim levels that have the hardware required to make OpenPilot available, but includes columns for “Lateral”, “Longitudial”, “No Accel Below”, “No Steer Below” as well as “Giraffe” which indicates which Giraffe should be purchased from Comma.AI (or sourced from other members).

“Lateral” means steering. You would expect ALL Production-Ready vehicles to be able to steer systematically by OpenPilot and all of them do.

“Longitudal” is braking. There are currently 3 different possiblities:

a) “Yes”. This means that OpenPilot has access to the radar data and has full control over braking.

b) “Stock”. This means that OpenPilot does NOT have access to the radar data and/or cannot control braking. This means that the vehicle’s OEM assist system is in full control of the braking and OpenPilot is only used for steering. This isn’t ideal as OpenPilot usually does a better job in keeping distance and braking, but it sometimes give you the ability to control distance using OEM distance controls (ie: 1 bar to 4 bars distance).

c) “Yes” (note #2) – Primarily for Toyotas, this indicates that in order for OpenPilot to have access to distance and braking controls, the Driver Support Unit (DSU) must be removed.

“No Accel Below” is the speed in which the vehicle disengages assisted control. This is more common in earlier editions of the OEM systems. This is not ideal as it doesn’t allow you to use OpenPilot in stop-and-go traffic and can cause unanticipating manual braking. The is often overcome by an unofficial solution called “Comma Pedal” which tricks the vehicle into believing that braking is being done manually as opposed to systematically.

“No Steer Below” relates to the speed in below which OpenPilot cannot be operated under. This was also an early version safeguard by vehicle manufacturers wanting to ensure assisted driving doesn’t occur below highway speeds. Many Models have a “0 mph” value which is great for everybody.

Does OpenPilot utilize the GPU for processing?

Not currently. OpenPilot only utilizes the multi-core elements of the CPU and only uses the GPU for the display elements including displaying the video as well as the overlaying lane logic.

How does the AI Modeling work?

You can find more in depth analysis by looking at Comma’s 2k19 dataset page as well as the White Paper published describing the data and procedures.

What is comma's history?

Biggest history from comma has been to make things affordable so anyone can buy it. You can originally or even now buy the full kit around $1000.

  • Comma team first tested OP on RC car that had CAN
  • Comma tried to sell comma one where comma team faced legal issue due to reason of being not safe.
  • Comma open source the one project which was later called NEO where comma just didn’t opensource the NEO OS used by 3t but they open sourced the NEO board where ilx was start and then civic came along so did other cars which was basically DIY where you had to build your own wiring harness.
  • Comma released EON an dashcam kit running 3t phone with better cooling solution where they started selling as dashcam kit running chffrplus which is basically openpilot that doesn’t drive but is only used for dashcam however OP can be used on it. EON had two editions first was 64gb and second was midnight black 128gb. Along with it they released panda’s whites were first after that grey’s came along which were newer versions NEO boards but can finally be plugged into OBD2 port or comma giraffe where you can turn your regular car into self driving.
  • After running out of stock from EON or 3t in general then they launched EON gold which is leeco gold.
  • Comma is planning to make EON even cheaper then EON gold where they have already tested EON on rpi with external camera.
What's the best large SUV supported by Openpilot?

The Toyota Highlander and Honda Pilot are both supported.

Why not all cars are supported or ported with openpilot?

Well each car is unique with it’s own CAN data. Please note openpilot is advanced LKAS where your existing car needs LKAS in order to be supported however not all cars are plug and play since each car’s signal to real hardware is different and hard to trace in general but not impossible. If your car does have LKAS but not yet supported, Feel free to look at tools like cabana so you can see how CAN works in general. If you would like to get basic understanding on how to port a car at following link.

https://medium.com/@comma_ai/how-to-write-a-car-port-for-openpilot-7ce0785eda84

What is best way to improve lateral control?

Well use the Openpilot Dashboard so you monitor the lag with your car in general and make the modeifications based on it for best OP experince.

How do I code a program to move my mouse in X, Y

You can do this using an arduino acting as a mouse.

The mouse functions enable 32u4 or SAMD micro based boards to control cursor movement on a connected computer through their micro’s native USB port. When updating the cursor position, it is always relative to the cursor’s previous location.

The mouse can be moved using the Mouse.move() function. Keep in mind that you must first call Mouse.begin()

Here is an example of this function in action.

#include <Mouse.h>

const int xAxis = A1;         //analog sensor for X axis
const int yAxis = A2;         // analog sensor for Y axis

int range = 12;               // output range of X or Y movement
int responseDelay = 2;       // response delay of the mouse, in ms
int threshold = range/4;      // resting threshold
int center = range/2;         // resting position value
int minima[] = {
  1023, 1023};                // actual analogRead minima for {x, y}
int maxima[] = {
  0,0};                       // actual analogRead maxima for {x, y}
int axis[] = {
  xAxis, yAxis};              // pin numbers for {x, y}
int mouseReading[2];          // final mouse readings for {x, y}


void setup() {
 Mouse.begin();
}

void loop() {

// read and scale the two axes:
  int xReading = readAxis(0);
  int yReading = readAxis(1);

// move the mouse:
    Mouse.move(xReading, yReading, 0);
    delay(responseDelay);
}

/
  reads an axis (0 or 1 for x or y) and scales the
  analog input range to a range from 0 to <range>
/

int readAxis(int axisNumber) {
  int distance = 0;    // distance from center of the output range

  // read the analog input:
  int reading = analogRead(axis[axisNumber]);

// of the current reading exceeds the max or min for this axis,
// reset the max or min:
  if (reading < minima[axisNumber]) {
    minima[axisNumber] = reading;
  }
  if (reading > maxima[axisNumber]) {
    maxima[axisNumber] = reading;
  }

  // map the reading from the analog input range to the output range:
  reading = map(reading, minima[axisNumber], maxima[axisNumber], 0, range);

 // if the output reading is outside from the
 // rest position threshold,  use it:
  if (abs(reading - center) > threshold) {
    distance = (reading - center);
  }

  // the Y axis needs to be inverted in order to
  // map the movemment correctly:
  if (axisNumber == 1) {
    distance = -distance;
  }

  // return the distance for this axis:
  return distance;
}
How to tell what version of NEOS you're running?

/cat VERSION via ssh

Do I need the lane departure button activated for Openpilot to work??

No – OP takes over that. It is rendered useless.

What's the best electric car supported by Openpilot?

Both the Tesla Model S and Chevy Volt are supported by Openpilot.

Can the camera on the EON be disconnected so that the EON doesn't have to be removed from the windshield all the time?

Yes, but it involves building a rugged cable to a camera sensor that isn’t designed for remote operation. Issues like signal attenuation, EMI shielding, and other factors need to be worked through. If you’re the adventurous type, give it a try and report your results on Discord!