2. Description of the algorithm

2.1 Image processing part

The goal of the image processing is to detect lanes in real-time. And it consists of the following steps.

Step 1: Find edges with Canny Edge Detection

Step 2: Divide the image into five sections

After dividing the image into five sections, we are only interested in image processing in the second and third sections counting from the top. As seen from below, detecting lane boundaries and calculating midlines from these two sections is enough for us to distinguish between straight and curved road.

Step 3: Find straight lines with Hough Transform

Step 4: Identify lane boundaries out of straight lines

Here we used clustering and merging technique. First, we clusterize lines that represent same lane boundaries in the real world. Then we merge lines in the same cluster into one single line. Finally, we find the nearest two lines relative to our camera as lane boundaries.

Step 5: Calculate the midline from two lane boundaries

Step 6: Extract additional information from midlines

First, we can calculate the angle between two midlines from adjacent sections, as seen from image on the left. Later we will use a "state system" to determine if we are in straight or curved road state based on the angle between two midlines.

Second, we can calculate the angle between the midline and the vertical line. This is useful in lane centering because it tells us whether our car is pointing straight.

2.2 The State System

We use the image processing algorithm to determine how our car should behave given the circumstances. We split this behavior into four distinct states.

Diagram of the state system:

State One: About to enter a curve

If we are currently in a straight road, but the camera detects that the angle between the two midlines is greater than 55 degrees in the upcoming segment, then we know the car is about to enter a curve. In this case, we want the car to start turning. However, since the midlines are actually looking ahead of the car, we want to make sure the car only turns a little bit in order to position itself in the center of lane when it begins to turn around the curved road. If we start turning to the same degree as in the body of the curve, the car will drive off the road.

State Two: In a curved road

If the camera continues to detect that the angle between the two midlines is greater than 55 degrees in the upcoming segment, then we know the car is currently in a curved road segment. When the car is in a curved road, we want to make direction adjustments based on the angle between the two midlines we find during image processing. These midlines give a good estimation of the shape of the road, and therefore, adjusting the cars direction based on the degree difference allows the car to smoothly take the turn.

State Three: About to exit a curve

If we are currently in a curve, but the camera detects that the angle between the two midlines is less than 15 degrees in the upcoming segment, then we know the car is about to exit the curve. In this case, we want the car to continue acting the way it does in the body of the curve. However, we also want to make sure that we turn more in order to maintain a good position at the end of the curve.

State Four: In a straight road

If our camera continues to detect an angle difference less than 15 degrees between our two midlines, our car switches to a "straight road" state. In this state, the car executes its lane centering functionality. This keeps the car in a good position for any other processes we want to execute, namely lane overtaking.