Dear Community:
The Openpilot Community needs your help to continue to keep's lights on and support the path to Workbench v0.2. With only a one person crew and without commercial funding it's not always financially feasible to fund these services out of pocket. Please consider becoming a Patreon supporter of the Openpilot Community project and receive exclusive perks and benefits!
Become a patreon Learn more Maybe Later

Mar 01·3 min read

Traffic Light and Sign Detection Guide

Updated! with paper found at Git

Comma released a teaser while back on twitter with traffic light recongnation.

The teaser was there but it didn't go anywhere, matter of fact, OP itself did have code for traffic lights but it doesn't do much yet. One of the major reasons is that OP wasn't ready. In feb 2019 comma finally upstreamed last piece of puzzle for Toyotas with pedal interceptors (big shout out to kevin for his hard work). Last year, 2018, has been the year of pedal interceptors for all cars including teslas which solved the problem of complete Stop and Go which has been available for Honda vehicles but not the rest of the supported cars. Now with the Stop and Go problem was solved, I needed another project to work on so I though of traffic light recognition.

Recently 0.5.9 was released which is close to 0.6.0 which will hopefully have HD maps and BIG model which should help with smooth drives. With that, it's time to bring a new community developed gimmick. The new gimmick comes with big challenges like anything else, although traffic lights can be easily be recognized by a computer vision model, the major issue is determining where the car need to stop after seeing the light or sign. HD maps which are coming soon will be able to help determin the stopping point, but we will need to find another way to do this in the meantime.

0.5.7 was released last year and had pre-map features using the OSM (Open Street Maps) API, where OP can finally slow down at curves by making API calls. I did some digging to see if OSM has traffic light data as well and it turns out they do:

Traffic Lights

Traffic Signs

Since OP already has the code for slowing we will be using similar logic for stopping and resuming. The following is a to-do list I have come up with.

  1. We need a new model for traffic. Currently there are 2 models running on OP 1) Vision model 2) Driver Monitoring model 3) will be for monitoring traffic which will be named trafficd.

Trafficd will require some access to visiond and will be running parallel to visiond so that there no conflict. Another main reason is that OP's model is not open source although, visiond is finally open source and can be used for trafficd libraries. Trafficd will be a community trained model so we have more access to it. With help from some contacts I had with comma team (big shout out to Willem) for helping out on the project to bring assents for model which is now officially located at:

  1. Testing stage
    • stage 1-trafficd basics (if traffic light or sign is present stop right away)
    • stage 2-come up with a math to stop before intersection where the generic intersection will be needed.
    • stage 3-use OSM for stopping for traffic lights and signs (solve the math problem)
    • stage 4- use HD maps if it ever happens for more localization solution.

Although there was a ghetto way to run the model using rpi but it will be best to have model running on EON. Comma is not going to have traffic lights anytime soon so this will be beginning to something awesome.

Following is list of devs who has been part of the project so far :)

  1. mad genius who brought us auto lane change who i call albert einstein @BogGyver
  2. the king of gimmicks from Germany @arneschwarck