In my previous post, we developed a better understanding of machine learning by exploring how the human mind works. In today’s post, we’ll further that understanding by briefly discussing the history of A.I. and how machine learning came about.
I Find Humans To Be Illogical and Mildly Irritating
People have obsessed over artificial life since the dawn of time. Golems were ancient ideas of robots. Mary Shelley’s Frankenstein’s monster was a 19th century understanding of artificial intelligence. The word “robot” actually comes from a 1920 play called R. U. R., by Karel Čapek, and literally means “slave labor” in Czech.
Our modern view of A.I. and robotics started in the 1950s. In 1957, Frank Rosenblatt created the perceptron ( a sibling of the neuron ) at the Cornell Aeronautical Laboratory. Amazingly, it was able to distinguish between photos of cats and of dogs. This was the very beginnings of machine learning.
Unfortunately, after an initial surge in popularity, Rosenblatt’s work was dismissed as nothing more than a novelty. Back then, A.I. research was mostly done by mathematicians. Computer science had not yet spun off into its own field and was still a sub-branch of mathematics. A lot of mathematicians would shuffle in and out of A.I. They were convinced that rational thinking was vastly superior to empirical thinking. They felt that Rosenblatt’s work was pointless, and believed that building rational minds based on logic and deductive reasoning ( think Mr. Spock from Star Trek and Sherlock Holmes ), was the best way forward. Although proponents of the perceptron were able to make great strides, the mathematicians eventually won the day. And so, the A.I. community spent the next three decades focused on the rational mind instead.
And you know what? They were absolutely right. Rational thinking is indeed way better than empirical thinking. The rational mind is invulnerable to all the failings that the empirical mind continually suffers from. It doesn’t get hysterical, it doesn’t become racist, and it doesn’t traffic in rumors and innuendos. Only rationality can bring us the paradise that we dream of — a utopia free from prejudice and bigotry.
A Snail's Pace
(image by Capri23auto from Pixabay)
If rational minds are so vastly superior, why does Mother Nature even bother gifting us with empirical minds at all? For one reason, and one reason only. Rational minds are slow as molasses. They take FOREVER to reach even a single conclusion.
Deductive reasoning and rationality require a lot of time and study. Consider Sherlock Holmes. To solve any of his cases, he needs to exert a monumental amount of effort. All the knowledge he has to acquire beforehand. All the logical gymnastics he has to perform in order to get to the truth. For each crime he’s investigating, there’s a whole universe of possibilities that he has to slowly comb through and eliminate as impossible.
What good is it to be able to mathematically prove that a lion is indeed hiding in the grass and ready to pounce on you, if it takes you 20 years to do so? You’d be dead long before you’d ever get to that point. This is where our empirical minds come in. Instead of 20 years to derive a universal truth, empirical thinking allows us to jump to conclusions in mere nanoseconds. In the dangerous world we live in, it actually matters less whether something is true or not. What matters much more is how quickly we spring into action, regardless of whether it’s based on a truth or a lie.
This was a realization that took the A.I. researchers three decades to understand. These mathematicians may have had far superior brains than us mere mortals, but it sure took them forever to figure anything out.
During this period, researchers built what were known as expert systems. The reason why they were called as such was simple. These machines were literally experts! Hundreds of programmers would spend tens of thousands of hours writing billions of lines of code — just to give one machine expertise in a very narrow subject area.
Although the A.I. community had promised the world a Jetsons-like future by the 1970s, these expert systems couldn’t come close to accomplishing any of it. As this 1998 IBM commercial showed, even though Deep Blue was able to beat world chess champion Garry Kasparov at chess, not a single line of its codebase gave it the ability to defend San Antonio Spurs center David Robinson in the post! I’m sorry, but that’s just absolutely pathetic!
Dawn of a New Era
By the 1980s, most universities and research institutions were getting impatient with the lack of tangible progress from their A.I. departments, so they started defunding and shutting them down. It was during these austere times that a new generation of A.I. researchers started to take over. By this point, most universities had already spun off computer science into a separate discipline, and this nascent field tended to attract more open-minded individuals.
Alright, fine — we computer scientists are weirdos.
But it’s our weirdness that motivates us to not dismiss seemingly outlandish ideas as quickly as those stuffed-shirted mathematicians. Many among this new generation of researchers saw the failings of expert systems and were not so stubbornly wedded to the idea that the rational mind was the only way to go. They decided to take a fresh look at Rosenblatt’s work and began expanding upon it. This was the beginnings of a machine learning renaissance.
One of the biggest motivation for moving towards ML was the realization that machines are a million times faster than human beings. Computer scientists became hopeful that if they could program machines to figure out universal truths on their own, these machines could write the billions of lines of code themselves — in a matter of minutes — without needing hundreds of programmers to spend tens of thousands of hours.
Expert systems are the equivalent of God cracking your skull open and rearranging your neurons. Machine learning, on the other hand, is like you telling God, “Don’t worry, I’ve got this. I’ll learn how to walk, talk, and chew gum all by myself — and I’ll rearrange my own neurons.” To which God replies, “Oh my goodness, yes, please do! If I have to program eight billion human brains, I’d have no time to rest on the seventh day!”
Unfortunately, it wasn’t long before these researchers realized that the machines were rarely learning anything close to universal truths. They were mostly learning partial truths. Heck, a good chunk of what they were learning were downright lies.
Despite these flaws, though, these learning systems were able to accomplish so much more than what the expert systems ever could. Partial truths apparently are good enough to get things done. Machines don’t need to know the right answers for all possible situations — just the right answers for the finite number of scenarios they frequently encounter.
As the nineties rolled in, and then the new millennium, researchers kept throwing ever more sophisticated problems at these machines. We’re now at a point where machine learning systems are practically everywhere. You’d be surprised at how much of your life is already managed by them:
Pearls of Wisdom
One final thought. Over the long haul, it’s extremely dangerous to rely on our empirical minds. As time passes, empirical thinking will cause us to develop really bizarre views of the world that nowhere near reflects reality. If you believe that all you need to do when you’re hungry is to howl at the moon to make your hunger pangs go away, then things are going to end very badly for you.
Fortunately, Mother Nature has a solution. By giving all of us rational minds! That’s right! Spock and Holmes are not the only ones who can play the logic game. We all can!
You’re skeptical. “If we all have rational minds, then why do most of us act so irrationally?” Well, in the nonstop world we live in, we’re under tremendous pressure to make decisions yesterday, even when we don’t have sufficient information to make an informed choice. This forces us to rely on empirical thinking.
As we age, though, our rational minds do get stronger. We’ve had time to mull things over, to gain more experience, and to see things from different perspectives. This is why in our youth we tend to run around like chickens without heads — absolutely convinced that we have all the answers. But as we get older, we tend to mellow out.
We don’t normally refer to this as rationality — instead, we call it “getting wiser.” But consider what wisdom actually is. It’s understanding a few universal truths after having lived a lifetime of experiences. Which is exactly what rationality is all about — searching for universal truths.
Police Detective: “Holmes! Great of you to join us! The au pair found the master of the house lying here with the knife in his back. There are no signs of a break-in —”
Sherlock Holmes: “The butler did it.”
Police Detective: “Wow, that’s brilliant! How did you arrive at that conclusion?”
Sherlock Holmes: “Easy. Always blame the butler.”
Police Detective: “Err, you didn’t use any logic or deductive reasoning?”
Sherlock Holmes: “No, no. That’d take too much time. You don’t want this book to be 500 pages long, do you?”
Police Detective: “You know what, Sherlock? I like your style! Whether the butler actually did it or not, we’ll beat a confession out of him!”
Sherlock Holmes: “Excellent! Now, if you’ll excuse me, I’m off to my next gig.”
Police Detective: “And what would that be?”
Sherlock Holmes: “I do stand up at a local nightclub. My routine is all about maids. How they’re weasels and like to steal everything.”
Police Detective: “Sounds like pure gold to me!”
Footnote
For more information about the fight between logical A.I. and statistical A.I., check out the resources highlighted in this website, Minsky vs. Rosenblatt.