A few weeks ago I got the idea to try to automatically bid on Pancakeswap’s Prediction game. So I decided to try to build a Pancakeswap Prediction bot to take on the task. I did this just to learn about how you can automate browser interactions and hoped to make a few dollars in the process. It was mostly just a challenge to see if I could do it since I haven’t been doing a lot of coding lately.
What is Pancakeswap Prediction?
Pancakeswap is a DEX or decentralized exchange. The main use of the site is to buy, sell, and stake crypto. You can learn about my thoughts on Pancakeswap here.
Pancakeswap Prediction is a “game” on Pancakeswap in which you bet on the price of BNB going up or down over 5-minute windows. You choose “up” or “down” and if the price ends up being higher (up) or lower(down) in 5 minutes you win. It’s essentially gambling with about a 50/50 chance you pick the correct outcome.
Building the Pancakeswap Prediction Bot
The first step I took was some basic Google searching to see if anything existed for this solution. What I mostly found were scam sites trying to get you to either purchase something or give them your secret key to your wallet (which you never want to do!!).
I did stumble upon one post on Medium titled “Make an AI powered bot for PancakeSwap Prediction, part 1“. It looked like Pierre trained a rudimentary machine learning model, scraped the data from Pancakeswap Prediction, and tried to determine if the outcome of the next round would be Up or Down. This code was not complete. It did not seem to bid on actual games, but just keep track of what the outcome would have been. It seems like the author abandoned it because he wasn’t able to make a profit. Nevertheless, this was a great starting point for me. Thanks, Pierre!
There were a few key learnings that really guided me in the right direction
- It used data from the Binance API. This gave me a starting point for retrieving past data as Pancakeswap does not publish past results. The author used the results of the last 5 rounds to predict the next round.
- It showed me how to train a machine learning model to predict the next outcome. I have never really trained ML models in Python, so this was very helpful. I ended up using this pretty much exactly but also developed some alternative bidding algorithms.
- It introduced me to using Selenium to scrape data from a web browser. In the past, I have used Python/PHP/Ruby to scrape data using their HTTP libraries, but have never had a full browser client to interact with the site. This is also important as I would have to interact with my Metamask wallet to make the actual bids.
Building the Pancakeswap Prediction Bot
As I mentioned, I used Pierre’s bot code as a base starting point. I don’t think that I plan on publishing the full code because it needs refactor badly and I don’t think I want to release a slew of bots on the game. I do know that there are other bots running though. However, I can talk about the process.
This first step was to set up Selenium. To do this you need drivers for your web browser. I decided to use Firefox for this project even though I use Chrome as my main browser. With Selenium, you can open a browser and configure things like plugins, etc and save the state of the browser. This state can then be used with the application launches the browser. This is important because I needed to set up a Metamask wallet in the browser first so that I could interact with Pancakeswap.
I also needed to get a Binance API key to pull historical data for the ML model. Plus I needed to store my Metamask password somewhere secure so that I could unlock the wallet to make bids. This was not the secret key, but a password to unlock Metamask. I stored it in an environmental variable so that it could be read by the application.
I started off by just trying to get the original code to work. With some slight refactoring, I had the bot read the results of the games and determine if it should bid using Pierre’s model.
Interacting with Pancakeswap in the Browser
At this point, the application was read-only. I was able to scrape data from the page, but not interact with it. I did a lot of reading about Selenium and found out that you can interact with pages using it. Selenium is designed for automated testing of web apps, but can also serve the purpose I was looking for.
Selenium works by identifying elements using CSS selectors or XPath. I have a lot of past experience with XPath from my time doing enterprise search engines that were based on XML at the time. I found it the most natural for me to help identify elements.
The entire process was extremely iterative. I would have to use the browser console to identify elements by XPath, and perform the action I intended – such as a click. Then I would run the app and monitor what happened. The good thing about Selenium is that you can run the browser in view so that you can see the interaction live.
Data Analysis and Bidding Models
At the same time, I worked with some Jupyter Notebooks to do some analysis of past performance to see if my bidding models would work. Jupyter Notebooks is a tool used by data analysts to examine data. My goal was to try to see if my models would be profitable based on historical data.
I grabbed the historical data from the Binance API. Trained a machine learning model using 80% of the data, but kept 20% as a test set. I would use this test set to first get the prediction from the mode (up or down) and then get the actual outcome of each 5-minute window. Then I could see if I would have won or lost.
I went through a bunch of different iterations of the notebook, but initially made a mistake that led me to believe the application would be more profitable than possible.
I also built out different graphs to show if other bidding algorithms would work better. For example, I compared the ML model prediction versus just always betting “Down”. I took into consideration the odds. Pancakeswap determines the odds based on the amount of money betting up versus down minus the 3% fee that the platform takes. So up could have odds of two, which would mean you make a profit of 100%. However, often times the odds were lower, like 1.5. In that case, you make a profit of 50% if you win, but you lose 100% if you lose. I started testing algorithms that would bid only if the odds were above a certain level to see how they performed.
Unfortunately, I seem to have lost the notebook so I can’t share any screenshots of the graphs.
Testing the Bot
At this point, I had thought that I had enough knowledge to successfully run the bot for profit, but I wouldn’t really know until I actually tried it. I ran the bot on several different days and used different algorithms to see what would happen. Some days I would profit, while other days I would not.
I ran some test bidding against real games. I stored the results in an SQLite database so that I could run my algorithms against real data from Pancakeswap versus the data I gathered from Binance API.
It also took a lot of babysitting to handle all of the exceptions that occurred in the app. In order to really troubleshoot these, I had to actually bid on games because that interaction tended to cause the most problems. I ended up coding the app to capture exceptions and take a screenshot and a dump of the browser source code to help me debug issues. After about a week of trial and error, I had the application stable enough to run on its own with minimal supervision.
Running the Pancakeswap Prediction Bot
So, I had run my data analysis and determined that this could potentially be profitable and I had the bot stable enough to run on its own overnight without me babysitting. I ran some short tests over a couple of hours and had mixed results. I determined that I needed to run this over a longer period of time to avoid short-term variance in price action.
I added some funds to my Metamask wallet for the bot to use and then I set the bot up and let it run for a few days. I also had a mechanism for harvesting profit so that I could keep the bot running without using up all of the money in my wallet.
Then indented to evaluate the results. I kind of forgot the prediction bot was running for a few days because I was busy with other things at work.
Evaluating the Results of the Pancakeswap Bot
After a few days, I decided to see how my bot was doing. At various points during my short tests, I may have won over 60% of the games. However, over the long term, I ended up losing much more money than I intended. I tried some different bidding algorithms but continued to lose money.
Surprisingly, the bidding algorithm that seemed to work the best was always bidding down if the odds were over a certain number (I think it was 1.8). This was likely just due to the state of the crypto market. Over the past year crypto has been getting annihilated, so betting down was a good strategy until the market corrected.
I think I had some fundamental flaws that I overlooked in my initial data analysis notebooks. For one, some of the algorithms I was using to measure profit had some mathematical flaws that inflated the results to show more profit than they should. The graphs were up and to the right, but that didn’t match what happened when I actually ran the bot.
Actually running the bot was more cyclic. I would be profitable for hours, then have a big downswing, then recover some, another downswing, etc.
Realistically, if the bot performed better than 50%, I had thought I could make a profit. I didn’t think about how the odds come into play. When you lose, you lose 100%. When you win, you most likely win less than 100% profit. Over time, even at a 50% win rate, you still lose. I think my model was even underperforming the 50% goal anyway. I suspect you would need to win 60-70% to be porofitable.
So in the end, I lost way more money than I should have wagered, but I did get some value in return for that investment. I learned some new skills that I can apply in the future in both my career and fun side projects.
See my Pancakeswap Prediction Bot in Action
I guess I should post some proof that I was actually able to achieve my goal of automatically bidding on these games. I had some problems with Selenium breaking when trying to use the native screen capture tools in macOS. Here is a video I was finally able to record.
Summary
I might revisit this at some point, but I don’t think that it can really be profitable. If I was highly profitable, I would even be sharing this with you. However, I just find it fun to try things like this. I might turn it back on and bid extremely low amounts to see if I can tweak the algorithms to make it more likely to be profitable, but I suspect that won’t do much. However, I took a look at some of the best-performing users on Pancakeswap and some of the top winners do appear to be bots.
My guess here is that bidding “up” during an extreme bull run, or “down” during a bear run, is a better option over the machine learning model. I don’t think that the past performance of rounds is really a good indicator of the outcome of the next round – which is what the ML model uses. However, maybe I’m wrong. Who knows. Either way, it was a fun exercise.
