Dear Community:
The Openpilot Community needs your help to continue to keep's lights on and support the path to Workbench v0.2. With only a one person crew and without commercial funding it's not always financially feasible to fund these services out of pocket. Please consider becoming a Patreon supporter of the Openpilot Community project and receive exclusive perks and benefits!
Become a patreon Learn more Maybe Later

Mar 01·4 min read

Can OP detect traffic lights or signs?

Updated! with paper found at Git

Comma released a teaser while back on twitter with traffic lights recongnation.

Although the teaser was there but it didn't go anywhere matter of fact OP itself did have code for traffic lights but it doesn't do much yet. One of the major reason is OP wasn't ready. In feb 2019 comma finally upstreamed last piece of puzzle for toyota with pedal Big shout out to kevin for his hard work. Last year 2018 has been year of pedal for all cars including tesla running OP which solved the problem with Stop and Go although Honda's did had it but not rest of the community. Now since the SNG problem was solved i needed another project to work on so i though of traffic lights recognition.

Recently 0.5.9 was released which is close to 0.6.0 which will have HD maps well hopefully with BIG model which should help with smooth drivers overall so it was time to bring new gimmick. The new gimmick comes with big challenges like anything else of course so following i will describe the major problem with traffic lights or signs although traffic lights can be easily be recognized by model but the major issue is where does the car need to stop after seeing the light or sign since HD maps are at the corner which will help with more of accurate location for stopping meanwhile.

Back last year of 2018 0.5.7 which released had pre map using OSM API where OP can finally slow down at curves by making API calls so did some digging to see if OSM has some traffic lights as well which can be used and turns out they do:

traffic lights

traffic signs

Since OP already has the code for slowing we will be using similar logic for stopping and resuming. Following is to do list i have come up with.

  1. We need new model traffic. Currently there are 2 models running on OP 1) we all know this one :) 2) driver monitor 3) will be for traffic which will be called trafficd.

trafficd will need limited access to visiond which will be running parallel with visiond so this way there no conflict another main reason is the op model is not open source although visiond(finally open source) which can be used for trafficd libraries. trafficd will be community trained model so we have more access to it.With help of some contacts i had with comma team big shout out to willem for helping out on the project to bring assents for model which is now officially located at:

  1. testing stage
    • stage 1-trafficd basics( if traffic light or sign is present stop right away)
    • stage 2-Come up with a math to stop before intersection where the generic intersection will be needed.
    • stage 3-use OSM for stopping for traffic lights and signs(solve the math problem)
    • stage 4- use HD maps if it ever happens for more localization solution.

Although there was ghetto way to run the model using rpi but it will be best to have model running on EON. Comma is not going to have traffic lights anytime soon so this will be beginning to something awesome.

Following is list of devs who has been part of the project so far :)

  1. mad genius who brought us auto lane change and now traffic lights who i call albert einstein @BogGyver
  2. the king of gimmicks from Germany @arneschwarck


Can't we just use existing model for traffic events?
Although the message structure is there for trafficevents but they are not being sent.

big shout out to jfruxT who finally created channel topic called traffic sign detection on discord:)