Category: Machine Learning

My limited research to date has led me to the conclusion that learning to talk is hard to do unless there is some level of intelligence behind the learner. That’s where the chatterbots fall short. They exhibit a superficial ability to talk to you, but there’s nothing behind the curtain. They respond credibly only if you stick to the kind of discussion anticipated by their creators. Any deviation produces nonsense that is obvious to the human but not to the machine.

So, teaching a machine to understand normal English is going to require some kind of intelligence. Like many of the ideas I’ve been exploring, this is a well-worn intellectual path. Most of the material on the Internet that I can actually understand is from research done in the 1970s and 1980s. Some researchers concluded that coding up a program with all of the grammer rules of a language will not allow the program to capture the meaning of even a simple sentence. Most of what we say requires some kind of “world knowledge” beforeyou can make sense of it. You figure out what a sentence means by parsing the meaning as much as you parse the grammer, or maybe by doing both together. Some of the buzzwords I’ve come across in this regard are conceptual dependency, procedural semantics, and conceptual analysis. All have to do with knowledge representation and I won’t take the space here to define them as Google does a fine job of covering the topic. SHRDLU (and here) was a program written in 1972 that seemed to capture much of what I’d like to do, albeit in a very small domain. I can’t quite figure out why it was never extended.

I think I’m going to start with a simple program that’s based on these ideas about using world knowledge as as a means of understanding language. A lot of what I’ll try first is described in online course materials from GA Tech and Cardiff Univ among others. The first decisions are going to be how to represent world knowledge, what should the machine “know” to begin with (all of WordNet, for example?), and how will the machine learn new things.

I also look over my shoulder every now and then wondering when Sarah Connor is going to show up to prevent the birth of Skynet. Is this a thing that should be done?

I’ve been thinking lately about how a computer might learn to speak normal English. Of course, there are lots of really bright people working in the area of natural language processing (NLP), and it seems to be a difficult problem to solve. I wonder how children seem to do it so easily, also a topic much studied, and if a computer might be programmed to do it in the same kind of way.

Children learn languages in the context of a body existing in a physical world. They learn to match words, or sounds really, to things they see. Later they learn to connect word-sound-things to actions in order to interact with other speakers. Children seem to naturally want to communicate with others and interact with things. Even so, it takes several years of constant interaction to learn the vocabulary and grammar of a language.

Computer programs don’t really seem to have a direct connection to the physical world. I expect this means the learning process would be different. It’s pretty easy to start the program off with a large vocabulary of words (e.g., WordNet) and some fairly sophisticated grammar rules for parsing sentences. What’s hard is to do is get the program to “understand” what the words and sentences mean. The program might be able to parse a sentence given it, and generate a grammatically correct response, but it doesn’t really understand the language yet. There are lots of these kinds of programs floating about (e.g., Alicebot) and they’re pretty impressive…but they don’t understand.

So, what does it mean for a computer program to understand English? How would a program learn to understand?