Neural Networks in Racing Games
Agents navigating the track
A neural network consists of layers of neurons. Data is passed to the input layer, processed through hidden layers of neurons and then returned through the output layer.
After studying earlier cases of using neural networks in racing games, I decided that my inputs would be how the cars perceive the world - specifically their distance to track limits. The outputs would be what controlled the cars, i.e. driving instructions in the form of throttle, brake and steering commands.
To acheive my goal of having the agents complete a lap I divided my problem into smaller tasks:
Input - Raycast
My plan was to have each agent cast five rays forward in 45 degree increments and use these rays to calculate distances to the track limits.
Since I chose to use our in-house engine Tga2D, I needed to implement some utilities to accomplish this. I created a line segment class and intersection tests for ray vs. line segment. I also imported systems and utilities from my own libraries and earlier AI projects.
After doing this, I tested the code using an input-controlled car casting a ray that intersected with my line segments.
Verifying raycast intersections
Input - Track
The next step was to construct a proper race track with track limits for the agents to navigate. I built the track in Tiled Map Editor using free to use assets. I exported the track and constructed track limits by placing line segments along its edges. The track limits are stored in json-format per track in separate folders, where I also store checkpoints.
To measure track progress, I implemented a line based checkpoint system. Track progress was used to measure performance.
The race track with checkpoints and track limits.
Output - AI Controller
My hope was to be able to compete against the AI controlled cars when the project was finished. For that reason I generalized the actor code to accept an input-controller or an AI-controller.
The AI-controller has the neural network that processes inputs and creates driving Instructions.
Evolution - Genetic Algorithm
To evolve the highest performing agents I used a genetic algorithm.
It generates a new population when the previous generation dies or does not improve in a set amount of time.
I configured the algorithm to select the four best agents and generate new offspring by cross breeding and mutating their neural network data. A copy of the best agent is always carried forward unchanged to avoid losing progress from one generation to the next.
Evolution - Training
When all the pieces were in place it was time to begin training the agents. It took a while for the agents to evolve into speed demons. Therefore, I added a time scale slider to fast forward their development.
It was quite fun watching the neural network agents improve with each generation. And some agents not really improving that much…
To avoid having to train the AI on every startup, I added a save/load feature. For each agent and track a genome is saved so that the opponents are ready to race when I select a track.
Making it more of a game
I was thrilled when the cars started to drive around the track. As mentioned earlier, I enjoy old racing games so I was having a ton of fun. I wanted to add some things to make it more game-like, instead of just an AI navigating a track.
My first step was to add lap records and track position to keep track of my performance versus the AI agents.
I also wanted to add headlights to the cars for some extra flair. In Tga2D there is a point-light class that adds lighting to surrounding sprites based on distance. However, circular headlights looked a bit odd so I modifed the pixel shader and renderer to accept a direction and angle. This way I could make the light sources appear as light cones.
To make the scene more dynamic I added a simple ”day night cycle” cycling the ambient light. The agents turn their headlights on when it gets dark and turns them off again when it gets brighter.
I also added a particle system so that the cars spawn a little explosion when they’re destroyed.
On one of my tracks I added some decorative oil spills and yellow arrows on the asphalt. One possible improvement is to add functionality to these objects. The current neural network only uses distances to track limits as inputs so the decorations have no effect on the cars.
It would be interesting to see how the agents would act if I added this functionality and modified them to be aware of obstacles.
I am very happy with the results of my specialization. I learned about neural networks and genetic algorithms and how they can be applied in real world applications.
The agents can complete laps and after a few generations they are quite fast, it's difficult to keep up!