After taking multiple online courses on machine learning, I set out to create my own ML project. This project started because of a disagreement I had with my girlfriend about what was the optimal strategy when playing Pacman.

My main goal was to be able to train and run Pacman on my 12" MacBook. The computer didn't have fans, nor a dedicated GPU, so training a model quickly and having efficient inference was paramount. My initial approach was to split the game board up into 28x36 tiles consisting of 8x8 tiles. I chose this method to reduce the number of features that would in feed in to my model. By determining each tile individually, I was able to reduce the number of inputs from 193,000 (224 * 288 * 3) to 1008.

I wrote a macOS app to launch Pacman on an emulator (OpenEmu), and take a screenshot of the game window 11 times, per second. After processing the image and resizing it, I run a tiling algorithm to split the 224x288 image into 1008 8x8 images. From there, the 1008 images would be classified into 14 different classes. The inference was all done in C++ using TensorFlow. From there, I would pass my 1008 classified tiles to my RL agent. Currently, I have an overall F1 score of 0.96, and I'm currently in the process of designing the RL agent. However I'm leaning towards a DQN using experience replay.

One benefit of using window sliding was that I was easily able to create an OCR model. The OCR model required half as many images to train as the tile classifier model, and has an overall F1 score of 0.99.

In retrospect, I would have used a ConvNet to reduce my feature space instead of tiling, but doing so taught me a lot about vision and building efficient infrastructure around my ML code. I also learned an extremely painful lesson during training: Have consistent labels. As I started classifying, my internal rules for what was a wall, what was a ghost, and what was just a mainly blank tile changed frequently. With the OCR model I ran into problems when classfiying 0's or O's. Reclassfiying the tiles by hand took a lot of time, and was a constant source of frustration. I look forward to finishing my agent, and using the new strategies to beat my friends at Pacman.