The New Turing Test






You only need one question to trip up the traditional Turing test:


“What is 17,352 × 23.78?”


The machine will answer immediately and correctly. The human will reply, “Umm…hang on…” and possibly give a wrong answer.


That is because it is human to err. This is precisely one reason why machines can never truly imitate human minds, or so I thought a few years ago when I posted an article about machine vs. human translation. The idea of machines making mistakes to outperform humans was counter-intuitive. But since then neural networks have finally come of age. Making mistakes is exactly what they do, and they do it brilliantly.


Not long ago, the world champion chess computer was challenged by a new neural-network, deep-learning contender. To put this in perspective, the reigning champion could calculate 70 million chess positions per second and access centuries of human chess experience, on top of decades of computer experience. The contender, created by Google, could calculate only 80,000 positions and had no previous experience. In fact, it learned to play chess from scratch in four hours, learning from its own mistakes as it played against itself. The result? Google’s deep learning creation not only defeated the champion; it thrashed it and didn’t lose a single game.


In order to trounce the champ, the newcomer had to use unexpected strategies and tactics unknown to its rival with all its wealth of human and machine knowledge. In four hours of learning from its own mistakes, it came up with such moves. In other words, it had new ideas that had not been considered before. Now, whether or not having ideas within the confines of a chessboard is comparable to writing War and Peace is still debatable, but it does raise the question: What is an idea, anyway?


We humans like to think that ideas come out of nowhere, which is one thing that makes us special. We may wake up after dreaming at night or have a Eureka moment in the bath and believe that our flash of inspiration is completely new, with no history to it. In fact, as we dream in bed or meditate in the tub, our synapses are still firing away with the information we already have stored away in our grey matter, perhaps more randomly and thus able to consider possibilities that don’t occur to us in our ordered, waking life.


Several years further back, another even more dated article of mine mentioned Gary Kasparov’s challenge to the chess computer Deep Blue. Then, Kasparov was able to win a match using unexpected moves, something I then thought only humans were capable of. That is because I thought only humans were prepared to make mistakes, to test their unknown limits in the hope of making new discoveries. But deep learning has been designed precisely to make mistakes and learn from them at a rate that humans cannot possibly match.


When a machine makes a mistake, it doesn’t think, “Oh no, I’m terrible at this. Maybe I should dedicate my existence to something else. I bet all the other machines are laughing at me now.” No; it simply thinks, “OK, that’s a mistake I won’t make again.” But if it is human to err, then it is even more human to err repeatedly over the same mistake and perhaps never learn from it.


As a human translator and writer, I would like to think that the first entirely machine-written novel is still a long way off, but for the first time I am beginning to have some doubts.


Perhaps the true Turing test of the 21st century is to detect if a machine wrote that text. Or translated it.



Photo: Ex Machina. (Film 4 & DNA Films).

Gary Smith

Professional translator, proofreader and content writer. ES, CA > EN.

Leave a Reply