Once upon a time, when I was an aspiring young astrophysicist, I took a course on neural networks. The maths was simple, the idea simple, the student simple. It all made sense, but I never thought anything more of it. Filled a gap in my day for a few weeks. Another little bit of theory to add to my mind, and never do anything about.
Wake up you sleepy head
Put on some clothes, get out of bed
Put another log on the fire for me
I’ve made some breakfast and coffee
Look out my window and what do I see
Fast forward let’s say…a year or two. If I was Down With the Kids, I’d say something like ‘Machine Learning, so hot right now’. But I’m only aware of the meme, I’ve never seen the source material. In a shock to no-one. Never one to be left out, I thought I’d have a poke around with The Latest Tech and see what it is all about.
A crack in the sky
and a hand reaching down to me
All the nightmares came today
And it looks as though they’re
here to stay
First reaction? Seriously, has there been no advance in neural net theory in twenty-fivefive years? (In a parenthesised aside: full disclosure: much like QCD and General Relativity, I grok the maths of neural nets. But understanding it at some deep inner-soul level? Nope.) Anyhow, prevailing wisdom indicates installing some python libs and playing around. Why not? Sounds fun. Except…python. And libs. Dependencies that don’t install, won’t work out of the box and a general chore. (And in another parenthesised aside: so the world uses the same set of python libs for this? All they are doing is…fiddling config numbers and rerunning training scripts? Really? For this this get the big bucks? Isn’t that already just baked-in bias from the off?)
What are we coming to
No room for me, no fun for you
I think about a world to come
Where the books were found by the Golden ones
Written in pain, written in awe
Frustrating. Irritating. And not going anywhere. But as ever, I hit on an idea. Why don’t I write it from scratch? That way, maybe I’ll finally get some insight into what it means. The classic example is a simple neural net that mimics XOR. So I sat down, in and of a Friday night, with a text book and wrote myself a neural network. From scratch. In golang. Because, why not? (And in a third parenthesised aside: nope, I still have no idea what it does and how it does it. The very definition of a back-propogated black box.)
By a puzzled man who questioned
What we came here for
All the strangers came today
And it looks as though they’re here to stay
Anyhow, there it is. Working. But there must be more to it than that. AIs are built on this stuff. So I melded the Feed Forward with a Recurrent Network, just because I could. Still thinking on what to do with it until…a misuse of a compute cluster from years ago came into my head. I might have mentioned this before, to do with weirdo ways of predicting football scores. And a use-case for a neural net. Maybe, just maybe…oh alright, I did. But before I get to that, I should talk on training the net.
Oh You Pretty Things
Don’t you know you’re driving your
Mamas and Papas insane
One of the standard training sets (aside from XOR) is image data. Feed in lots of images of digits into the net, and have it learn to recognise them. Whatever. But what if I fed it in all the football scores ever, could it predict the results for me? Worth a shot. But there are problems with match data. Mostly as there is bias involved (Manchester United are playing Bogtrotters Town? The hallions have no chance. But turns out giant killing happens…). Can I get round that? I’m glad you asked.
Oh You Pretty Things
Don’t you know you’re driving your
Mamas and Papas insane
Stripping the names out isn’t enough. What I did was grab the results of the entire history of football. (Well, I will. Long time ago I dumped the Premier League data into a database, so I had that to hand.) Now, for each week, I build the league table. Easy done, simple rules. However, what I also did was keep the position of the teams in the league, the week of the season and the result. For example, week 3 the fifth placed team played the seventeenth and it was an away win. And I do this, for every week of the season, for every season.
Let me make it plain
You gotta make way for the Robo Superior
And that is my training data. Of course, the eagle-eyed stats chaps will spot I’m giving the net conflicting information: one season the above example could be an away win, a few season down the line a draw, and so on. I only came to this conclusion after I tried to use the net for predictions. However, when I fudged that and averaged the amount of points for that season/positions, it made little to no difference. shrug.
Look at your children
See their faces in golden rays
Don’t kid yourself they belong to you
They’re the start of a coming race
The earth is a bitch
But I have a corpus of data, and I fed it into the neural net. Gah, to save me calling it a ‘neural net’ each time, I’m going to call it what the repo is called: Cortex The Killer.
So I fed Cortex The Killer all this information.
I’ll put an API online in front of this soon, for those who want to play. I even added an interface to continually teach it. Scrapes the sports sites, gets the results, adds it to Cortex’s innards.
We’ve finished our news
Homo Sapiens have outgrown their use
All the strangers came today
And it looks as though they’re here to stay
After the initial training, I asked Cortex to ponder the weekend’s forthcoming games. He got four right out of ten.
Oh You Pretty Things
Don’t you know you’re driving your
Mamas and Papas insane
Four out of ten. That is abysmal. And not one away win predicted at all. (As a last parenthesised aside: he did better than I did. I predicted three right.)
Oh You Pretty Things
Don’t you know you’re driving your
Mamas and Papas insane
But the thing is: I know how it works, I understand all the code, and the theory. And yet, when I throw the training set at it, it arrives at answers, and I have no idea why.
Take us out, Mr Bowie:
Let me say it again
You gotta make way for the Robo Superior
