Artificial Intelligence may not pay the bills for too many people in our field,  and it may not have worked its way into too many products, but I think it still holds a special place of fascination for programmers. For my generation at least, blame it on HAL.

If you actually work for a living, you may not be aware of one the biggest wet blankets at the AI party, John R. Searle. Back in 1980 Searle posed a thought experiment that has become known as the Chinese Room argument, in which many people feel he thoroughly disproved the notion that any computer program could acquire true intelligence.

The Chinese Room argument is taken very seriously in the academic world, and entire forests have been destroyed arguing about it.  To maintain your street cred among the intelligentsia, you need to bone up on Searle. You probably only need to dedicate 30 minutes of your life. There's just one problem: it will drive you nuts.

The Argument

Everyone who writes about this problem gets to create their own little overview, and I guess I'm not going to be an exception.  But I encourage you to read Searle's original argument; it's cogent and concise and won't take too much of your time.

In short, Searle does a very nice job of setting up a straw man and then knocking it down. He first starts with the premise that there exists a computer program that can intelligently converse with humans using written Chinese. Given that this passes the Turing test, we assert that the computer executing the program has achieved understanding and intelligence.

Searle then uses reduction ad absurdum to smite the assertion.  He rightfully supposes that we could implement this computer program by putting John Searle in a locked room, giving him a written instruction book, paper, pencil, filing cabinets, erasers and what not. He would then accept Chinese text through his mail slot, and blindly follow the instruction book in order to create output, which would intelligently respond to the input text.

And at this point could John Searle be said to understand Chinese? Of course not. (For good measure, Searle also points out the program does nothing to explain human intelligence - but I don't think this stirred up the passions of the first part of the argument.)

The Problem

The first problem here is that Searle is a philosopher, and as such he got off on a bad foot. In the Chinese Room, John Searle is clearly functioning as a CPU executing a program, and nobody would seriously posit that the CPU itself is intelligent - any more than we posit that one neuron or DRAM cell is intelligent. 

And the fact that the problem is framed in these erroneous terms is at least annoying to most of us, if not downright infuriating. It tends to bring out angry reactions.

However, it doesn't take much jiggering to reframe the problem. In this computer with a human CPU and paper storage, where exactly is the so-called understanding? You can't exactly point to it. So you end up falling back on the notion that the system as a whole is intelligent, which is somehow unsatisfying. It would feel a lot more comfortable if we could point to a robot and say the same thing.

Our Unfortunate Predisposition

As computer programmers, I think we have a natural tendency to believe in the computational model of the  mind. We work with computers all day, and we are steeped in deterministic thinking. When that is your framework, it's very easy to think of the mind as another computing device. And fair enough, you can explain most of what it does in terms of computation.

Searle seems to be taking a poke at this model. After all, he's asserting that a computer can't have intelligence. Thus the mind must be something more than a simple computation machine. 

What's the Point?

It's easy for anyone to read Searle's paper and immediately toss off a dozen cogent arguments either in favor or opposed to it. But trust me: you probably don't have anything new to say about it - spend a year or two on the exahustive references in the Wikipedia, then reassess.

 Given this, I won't try to formulate any ingenious arguments either. The purpose of my post is just to air the argument, not to refute it.

However, I do think that there are some interesting things popping up right now that may make this paper unimportant in the future.

Turing Was Right

When Alan Turing devised his classic test for machine intelligence, he purposely structure it so as to avoid getting sucked into a Chinese Room argument. He didn't ask for proof that a machine was intelligent, simply that it exhibited intelligence. 

 The difference between Turing and Searle is that with Searle we have to get into definitions of elusive concepts such as mind, consciousness, intelligence, intention, and understanding. Great stuff for a philosopher - for a scientist, not so much.

What's really interesting is that neuroscience is now providing some small measure of visibility into the mind, as opposed to the brain. We can actually draw some conclusions about how cognition occurs.

And the outcome of a lot of this research seems to show that our model of the mind may be fatally flawed. Consciousness is turning out to be a shaky concept, as is the notion that the conscious mind has free will. Intelligence and understanding are turning out to not be nearly as clear-cut as we once thought.

It may turn out that the entire semantic model of cognitive processes that we built up over 2,000 years is about as useful as the luminiferous aether turned out to be. And it may also turn out that Turing's intution is correct: both humans and computers can't be said to possess intelligence or understanding, they can only exhibit it.

 In which case, Searle's work might be described by a line from the Bard:

A tale
Told by an idiot, full of sound and fury,
Signifying nothing.

Which would be fine by me. Twenty-eight years after publication, I still find it pretty annoying.

Comments (6)

Subscribe to this comment's feed

Show/Hide comments

...
written by Jocelyn Paine, June 30, 2008

Annoying, but fun. For one of our AI Society meetings, we put on a stage production of the Chinese Room. I was the man inside the room. Actually, I have to admit, that was pretty boring.

...
written by Mark Nelson, June 30, 2008

When I call it annoying, that's actually a tribute I guess - it's annoying that an argument that I don't like seems so powerful.

>I was the man inside the room. Actually,
>I have to admit, that was pretty boring.

That's because you didn't understand anything! :-)

- Mark

Contradiction to Searle
written by Vedant Kidambi, July 01, 2008

There is one point which I have found against Searle's theory.

Searle seems to have described Intelligence as a quality that an entity (human or machine) "has". I disagree with that.

Intelligence is defined by what one does, by how one reacts to situations, or responds to stimuli. Intelligence is not defined by who you are. Therefore an intelligent entity is an entity that exhibits intelligence, and to say that "the entity has intelligence" would be a meaningless remark.

The way I see it, this invalidates Searle's entire stand.

...
written by Mark Nelson, July 01, 2008

@Vedant:

Yes, I agree completely on this point. The standard Western philosophy (maybe Descartes is responsible for this?) has the notion that the "mind", independent of physical reality, has all these specific attributes, intelligence being one of them.

The problem, which naturally leads to your conclusion, is that there is no place to observe this attribute - it's intangible. That's why I think Turing was so smart to bypass the problem and merely look at what attributes the system can exhibit.

The biggest problem with Searle is that he is making his argument in the domain of Philosophy, and any attempt I would make to duke it out there would be a miserable failure - I am a programmer.

- Mark

Intelligence vs. Consciousness
written by Dale Roberts, October 04, 2008

I find these philosophical thought experiments about intelligence and consciousness "annoying" in the same way you do - they draw me in and occupy my mind.

I first read his argument a long time ago in a book called The Mind's I, edited (and annotated) by Dennet and Hofstadter. They didn't like his argument. And I've been trying to figure out ever since where I stand.

I am currently leaning towards qualified agreement. I've come to understand that part of the problem is the muddling of the two separate concepts of Intelligence and Consciousness. A computer can certainly "demonstrate" intelligence in the Turing sense without being conscious.

I think the problem (and why there is so much argument and confusion over CR) is that Searle is really arguing about computers being conscious ("strong AI"?? in his words), not intelligent.

I think it's time to separate the two concepts. We aren't looking for AI, we are looking for non-biological consciousness (which may or may not be intelligent).

Thank you for a thought-provoking article!

...
written by Mark Nelson, October 04, 2008

@Dale:

Yes Dale, I'm with you on the need to separate consciousness and intelligence. Both concepts are very troublesome, but I don't see how they must be inextricably tied together.

My personal opinion is that consciousness is going to turn out to be somewhat of a fictional concept. (And the parts that aren't fictional will be greatly diminished in importance.) And intelligence? Who knows, but the definition is surely going to involve some arbitrary decisions on our part.

- Mark

Show/Hide comment form