Scholarly article on topic 'Theory, Methodology, and Implementation of Robotic Intelligence and Communication'

Theory, Methodology, and Implementation of Robotic Intelligence and Communication Academic research paper on "Computer and information sciences"

Share paper
Academic journal
Procedia Computer Science
OECD Field of science
{"robotic intelligence" / "robotic communication" / "ontological semantic technology"}

Abstract of research paper on Computer and information sciences, author of scientific article — Victor Raskin

Abstract This paper makes several uncommon claims. First, it suggests a theoretical approach, even though the theory comes with a clear methodology and implementation, so that we know what we do, how we do it, and why it is best to do it that way. Second, it pushes forward HARMS communication in natural language and explains why it is better that way. Third, it insists on the semantic approach so that all the HARMS agents “understand” (or even understand) what is going on--along with what is not. Fourth, most specifically and, hence, perhaps least importantly, it proposes one particular method of doing it, mildly hinting that some of its principles are unavoidable.

Academic research paper on topic "Theory, Methodology, and Implementation of Robotic Intelligence and Communication"



Available online at


Procedía Computer Science 56 (2015) 508 - 513

International Workshop on Communication for Humans, Agents, Robots, Machines and Sensors

(HARMS 2015)

Theory, methodology, and implementation of robotic intelligence

and communication

Victor Raskin**

Linguistics, CERIAS, CS, CIT, Purdue University, W. Lafayette, IN 47907, USA


This paper makes several uncommon claims. First, it suggests a theoretical approach, even though the theory comes with a clear methodology and implementation, so that we know what we do, how we do it, and why it is best to do it that way. Second, it pushes forward HARMS communication in natural language and explains why it is better that way. Third, it insists on the semantic approach so that all the HARMS agents "understand" (or even understand) what is going on--along with what is not. Fourth, most specifically and, hence, perhaps least importantly, it proposes one particular method of doing it, mildly hinting that some of its principles are unavoidable.

© 2015 TheAuthors.PublishedbyElsevierB.V.This is an open access article under the CC BY-NC-ND license


Peer-review under responsibility of the Conference Program Chairs

Keywords: robotic intelligence; robotic communication, ontological semantic technology

1. Introduction

Human-computer interaction has been happening since the first command was introduced into the computer. The language of the interaction is code, which sometimes includes words or even sentences of natural language. But each symbol and word come from a finite list and in allowable combinations. Actually, this describes human-human communication as well, does not it? So what is the difference between how we communicate with the machine, on the one hand, and another human, on the other? Or is there a difference? Or, again, should there be a difference?

Corresponding author. Tel.: =1-765-409-0675, fax: +1-765-494-3780 Email address:

1877-0509 © 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license


Peer-review under responsibility of the Conference Program Chairs

doi: 10.1016/j.procs.2015.07.243

There is—or was, mostly, in the 1980s—the whole mass of research and trade literature on the much misrepresented Turing test that would ostensibly show whether my unknown interlocutor is human or a machine, and it was all about intelligence. Since then, our notion of intelligence has changed radically with regard to artificial intelligence while our understanding of our own minds, unadvanced significantly either by the revolutionary progress with mapping the human genome or by mapping out the human brain, has not progressed that much.

In fact, if asked to think of a human mental functionality that a robot or any computer is not capable of, an educated mature thinker will mention language, culture, humor, and on all of those counts, the situation is not clear. The computer may easily know all the 37,000 or so human diseases, of which I, a hereditary hypochondriac, may barely think of a hundred. It keeps the entire inventory of General Motors automobile parts, in the same number brackets, in its memory, and I, an experienced driver of those cars, can probably list about 30.

IBM Watson can defeat human Jeopardy champions, and yet, only the New York Times Tuesday Science section and its multiple trusting readers can believe that the machine is intelligent. So what do I have that IBM Watson does not? I do have an enormous advantage of not running on IBM-produced or procured code, which means that I do not crash that much. Besides, I do carry a large variety of lists, none so huge, but from different spheres of life: people I know and remember, cities I have visited or know about, writers and their books I have read, food I have tasted, and much much more. But my memory of all those lists is flawed and less reliable than that of a computer—I do forget, confuse facts, misremember. Besides, if I am forced to produce a list, it can be immediately entered in the computer.

I can also speak, understand, write and read in English, and as a mater of increasingly rare fact, several other languages. The computer can barely do anything with understanding, even though it can output tons of text, for instance, answer my command to print out any text, including creating new ones, e.g., the list of all human diseases. Yes, but I can write a poem! And so can a computer, when programmed to do so!

This paper will explore robotic intelligence as a particular kind of AI (Section 2), argue for the use of natural language, with understanding capabilities by non-humans in CHARMS (Section 3), and briefly mention Ontological Semantic Technology as a mature implementation of this approach (Section 4). A semantically innocent roboticist, especially one brainwashed by machine-learning-only education should understand that this paper is based on two non-machine-learning principles: it is rule-based rather than statistical and it is meaning-based rather than bag-of-words-based. Nor should it be read as an attempt to project human collaboration into CHARMS—rather, it is a claim that both should be based on a solid computational semantic foundation.

2. Robotic intelligence kind of AI

The differences between human intelligence and artificial/computer/robotic intelligences are seriously masked by our increasing abilities to emulate human behavior in the computer. When working with humans in a CHARMS team, will the robots and agents emulate humans? This is what all these preliminaries were about. But, first, let us make it clear how

• human intelligence,

• AI,

• computer intelligence,

• web intelligence,

• agent intelligence, and our subject,

• robotic intelligence, relate to each other:

Human intelligence includes all mental activities underlying human lives. It prominently includes a full competence, sufficient for each individual lifestyle, in at least one native tongue. The general notion is that we have a pretty solid knowledge base about the world as well as the ability to perceive and respond to current developments in it. We can represent any of these in our native tongue and communicate it to native speakers, including ourselves. We can even communicate things that have never happened or can never happen because they are imaginary. It should be noted that language underdetermines reality, and there are many things that we can perceive only visually, such as, say, the map of Albania or a picture of a human face.

Artificial intelligence (AI) emulates parts and aspects of human intelligence in computer applications, where the machine attempts to fulfill a human intellectual task. The somewhat simplistic view in early AI, with its largely exaggerated expectations and false hopes, was that if such an application is reasonably successful, we would then understand how human intelligence does it because we would, of course, have designed the computer algorithm ourselves. As the field was growing older—I don't want to say, maturing—it became clear that the computer may employ other than human-like methods to achieve some plausible-looking results. Numerous and still growing efforts in machine learning certainly cannot claim the actual AI status because humans do not think statistically. These efforts are also not satisfactory in NLP applications because even their souped-up precision rate of 80% (really, around 60%) is significantly lower than the human user's 95+% expectation of accuracy (make it a maximum 5% error tolerance). In other words, who wants a computer application that is wrong once out of every five occasions—or even twice?! In more objective terms, who can trust a system that tries to manipulate a text without any ability or desire to understand what the text is about?!

Other than serving as research-clique markers, computer intelligence and web intelligence cover much overlapping generic territory and are marginal for us here. It is different than the last two bullets above, which are both firmly in CHARMS land. Both intelligent agents and robots are full-fledged participants of the HARMS hybrid teams, and the whole thrust of the CHARMS system is to maximize the autonomy and, hence, intelligence of the computational components. The fascinating difference that robotic intelligence adds is the cyberphysicality of the robots: they do exist in the physical space, which means having dimensions, being subject to time restrictions and abilities to move, etc. Moreover, the robotic intelligence may include the manipulation of physical sensors, such as distance to another object or ambient temperature.

3. Working together

HARMS is clearly interested in maximizing the efficiency of its efforts, and that means the best efforts by all participants. While the machines and sensors must be mechanically, optically, etc., improved to the best of the state of the arts, the humans, agents, and robots should contribute their intelligence, and for us, in robotic intelligence and communication, it means forever maximizing and optimizing the autonomy, intelligence, and productivity of the robots. This means, among other things, the reduction and, eventually, elimination of regular commands to robots because they should know their tasks, the conditions for their execution, all the pertinent scenarios of their existence, such as recharging and self-checking.

As we have shown in previous publications, there are several parameters defining the space where CHARMS is;

• Organization:

O human and other control,

O division of labor,

O specialization, optimization, and duplication avoidance,

• Communication:

O reporting and understanding,

O interlanguage translation.

3.1. Related research in organization of work

One would think that all of these areas would have been studied extensively and intensively in such diverse fields as control theory, ergonomics, corporate and industrial communication, and NLP, and they have, but never really the way CHARMS can use. Contrary to hybrid human-robot-agent-collaboration, inter-human collaboration has been studied intensely from several disciplinary and interdisciplinary perspectives: those of sociology, management, industrial engineering/ergonomics, human factors, rhetoric/usability, but it is not easily adjustable to the machine-language algorithmic environment because, inevitably, whether explicitly or implicitly, those studies depend on human perception and intelligence. More pertinent to this research, some of aspects of inter-human collaboration were subsequently extended to intelligent agents, and dominant among those are the belief-desire-intention (BDI) studies of intelligent agents1,2; rooted in influential scholarship3 on plans and intentions—see also Wooldridge4. BDI studies focused on hybrid teams' joint intentions5,6,7, shared plans8,9,10, and some other aspects of intelligent agents

architecture and implementation11'12'13. But even those—or certainly efforts of the COIN clique--do not reach the level of formality (meaning no human intelligence involved), sophistication and upward scalability that CHARMS must allow for.

Even more to the point, significant work has been done on the construction of practical, domain-independent teamwork models and architectures14'15'16'17, and some of that scholarship is already absorbed in CHARMS (it is interesting to note, however, that Tambe, the leader of that effort, is no longer continuing with it). Somewhat less pertinently perhaps but not without some relevance to us, there have been some reverberations18 of the 1980s much misguided philosophical discourse about ways to separate human intelligence from machine intelligence (without knowing much about either at the time), in which the Turing Test19 was loosely metaphorized,

20 21 22 23 24

if not actually parodied . It is this effort that we refer to when talking about the ideal of indistiguishability

of communication between a human and a robot. Going back to Section 1, CHARMS is designed to make the task of differentiating between manifestations of human and artificial intelligence even harder than presented there.

3.2. Related research in work communication

The discrepancy between what CHARMS needs with respect to organizing and optimizing actual communication among the HARMS partners and what NLP has made available outside of our group is much starker. While there has apparently been no prior work on porting the Natural Language Processing (NLP) technology, let alone Computational Semantic or any meaning processing technology, into supporting the robot/agent communication without limiting it to specific commands or menus, there have, however, been somewhat pertinent efforts in NLP involving intelligent agents, for instance25,26. Their focus has been on emulating dialog participation by the computer with a single human, and valuable insights have been achieved but not concerning real-life robotic agents nor dealing directly with their native systems of communication—and not with fully semantic methods.

Part of the reason for that paucity of robot-human communication research would be that active collaboration between agent and NLP research groups, outside of CHARMS, has yet to take off, and, hopefully, this effort may lead to more such interdisciplinary efforts. Another reason may be that the problem of the communication system among humans, robots, and agents lacks the main premises and constituents, such as large corpora of related texts, for the successful applications of currently dominant non-representative, non-rule-based, non-semantic methods. The syntax-, statistics-, and machine-learning-based approaches have dominated NLP for several decades and have made very significant inroads into classifying and clustering texts without understanding them and without spending efforts on acquiring such resources as machine-tractable repositories of meanings.

To facilitate robot-human communication, a radically different approach has been attempted, the one similar to the natural development of Pidgin English in the 19th century to mitigate English-Chinese communication in the ports of sea trade, or to the invention of Esperanto, a naive attempt to develop the "easiest" natural language that combines the features of the "most efficient languages" so that it could be adopted as the international language. ROILA27, a spoken language for talking to robots makes both of these claims: it is billed as simple, easy, and exception-free—and it is foreign to both sides, human and robotic, and has to be learned from scratch. Nor does it afford any access to meaning.

Probably the closest NLP has ever come to handling problems that are similar to the ones we deal with in this proposal is in the never-dying dream to program in natural language, a dream that recurs with almost every new approach to NLP—for the latest efforts in this direction, see, for instance28,29. This is not to be confused with the Computing with Words initiative30, which limits its purview to computational interpretations, or "precisations," of just a handful of words, mostly scalarized quantifiers.

4. The Ontological Semantic Technology (OST) component of CHARMS

Throughout this time, our computational semantics, or meaning- and rule-based NLP, has been addressing applications where the very nature of the task calls for comprehensive and direct meaning access, and we proceed on the premise that the hybrid communication does not have to—nor will or should it—generate multimillion-word corpora that lend themselves to the statistical methods. Essentially, it is not a text-clustering or data-mining application, where a considerable level of inaccuracy is tolerated, but rather one, in which immediate and precise

understanding of every command, report, or directive is of essence. Rule-based approaches have their own limitations: they function well where we have knowledge. After they yield meaning-based results, machine learning may still need to kick in for matters of reasoning, and especially abduction.

We have promised to spend a minimal amount of space on the implementation of the OST component of CHARMS because all the resources for it exist and because we showed earlier24 the mechanics of semantic interpretation. It is a mature technology that was developed in the 1990s31, with high-risk NSA grants and developed by Raskin and Taylor with help from Kiki Hempelmann, Max Petrenko and other former and current Ph.D. students at Purdue University32,33,34.

The language-specific ontology in the center of Fig. 1 is, in its latest incarnation, a large linked graph, with concepts as nodes and properties as links. Each concept is directly linked to as many other concepts as it has properties. Unlike many industrial and government ontologies that rarely have many more properties that pure subsumption—like the Linnean zoology: cat is feline—the OST ontology is property-rich. It was designed as language-independent, a true interlingua. Every specific natural language defines each of its words and word-like entities (such as phrasals) through a concept and its properties with their values as other concepts or such literals as numbers.


Morphology, Sunt. Rules


c AVaVAVAO — - - —

ra OZ^ZVv'ZsSZA. —


DC Morphology, ~

Sjail- Rules Lexicon

other languages



Input {Texts, Data, etc.)

POST (Processor for OST)


(Text Meaning Representations)

Fig. 1. OST Architecture

It was when Julia Taylor and I, triggered by our participation in Eric Matson's First Summer School on Humanoid Robotics at Purdue in 2011, realized that our ontology was non-language-specific much more strongly: it underlay formal languages and robotic systems just as it did all natural languages, databases, images and other forms of information. In simple terms, ontology-equipped robot "understands" the meaning of sensor in exactly the same way as its human partner understand the English word—or, for that matter, its counterpart in any other language: namely, they relate it to its ontological concept. And our robots code for that sensor is just another such word in yet another language.

In an interesting recent work35, the mechanical industrial robot was declared to have reached self-awareness because it was programmed to self-check certain elements of itself. What was characteristically missing from this innovative work was to make sure that the robot had any idea that it was checking itself because it has no ontology to make it aware of it. The OST ontology equips the CHARMS robots with a sense, heavily bolstered and additionally anchored with its physicality of its:

• place in the world,

• partners,

• physical parameters,

• position,

• movements,

• repertoire of functions,

and many other elements of knowledge that robot designers and users may not be aware of. References

1. Rao S, Georgeff MP. Modeling Rational Agents within a BDI-Architecture. International Conference on Principles of Knowledge Representation and Reasoning, 1991, p. 473-484,.

2. Rao S, Georgeff. MP. BDI-agents: From Theory to Practice. International Conference onMultiagent Systems (ICMAS'95), San Francisco, 1995.

3. Bratman ME. Intention, Plans, and Practical Reason. CSLI Publications, 1987/99.

4. Wooldridge M. Reasoning About Rational Agents. Cambridge, MA: MIT Press, 2000.

5. Cohen PR, Levesque HJ. Confirmation and Joint Action, IJCAI, 1991a.

6. Cohen PR, and Levesque HJ. Teamwork, Nous 25 (4), 1991b, p. 487-512.

7. Levesque HJ, Cohen PR, Nunes J. On acting together. Proceedings of the National Conference on Artificial Intelligence, 1990.

8. Grosz BJ, Sidner CL. Plans for discourse. In: Cohen PR, Morgan J, Pollack ME, editors. Intentions in Communications. Cambridge, MA: MIT Press, 1990, p. 417-444.

9. Grosz B. Collaborating systems. Artificial Intelligence Magazine 17 (2), 1996, p. 67-85.

10. Grosz B, Kraus S. Collaborative plans for complex group actions. Artificial Intelligence 86, 1996, p. 269-368.

11. Vikhorev KS, Alechina N, Logan B. The ARTS Real-Time Agent Architecture. Second Workshop on Languages, Methodologies and Development Tools for Multi-agent Systems (LADS2009). CEUR Workshop Proceedings, Vol. 494, Turin, Italy, 2009.

12. Sonenberg E, Tidhar G, Werner, E, Kinny D, Ljundberg M, Rao A. Planned team activity. TR 26, Australian AI Institute, 1994.

13. Dunin-Keplicz B, Verbrugge R. Collective commitments, International Conference on Multi-agent Systems, 1996, p. 56-63.

14. Tambe, M. Towards flexible teamwork, Journal of Artificial Intelligence Research 7, 1997, p. 83-124.

15. Pynadath DV, Tambe M, Chauvat N, Cavedon ., Toward team-oriented programming. In: Jennings NR, Lespérance Y, editors. Intelligent Agents VI: Agent Theories, Architectures and Languages, Berlin: Springer-Verlag, 1999, p. 233-247.

16. Yen J, Yin J, Ioerger TR, Miller MS, Xu D, Volz R. CAST: Collaborative Agents for Simulating Teamwork. IJCAI, 2001, p. 1135-1142.

17. Pynadath DV, Tambe M. The communicative multiagent team decision problem. Journal of Artificial Intelligence Research 16, 2002, p. 389423.

18. Churchland p, and Smith Churchland P. Could a machine think? Scientific American 262 (1, January), 1990, p. 32-39.

19. Turing A. Computing machinery and intelligence, Mind LIX (236), 1950, p. 433-460.

20. Searle J. Minds, brains, and programs. Behavioral and Brain Sciences 3, 1980, 417-424.

21. Sea le J. Minds, Brains, and Science. Cambridge: Harvard University Press, 1984.

22. Dennett D. The milk of human intentionality. Behavioral and Brain Sciences 3, 1980, 429-430.

23. Dreyfus SE, Dreyfus HL. A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition, Washington, DC: Storming Media, 1980.

24. Matson ET, Taylor JM, Raskin V, Min B-C, Wilson EC. A natural language Model for Enabling Human, Agent, Robot and Machine Interaction. The 5th IEEE International Conference on Automation, Robotics and Applications, Wellington, New Zealand, 2011.

25. Wilks Y, editor. Artificial Companions in Society: Scientific, Economic,Pps-chological and Philosophical Perspectives. John Benjamins: Amsterdam, 2009.

26. Ruttikay, ZM, Kipp M, Nijholt A, Vilhjlmsson HH, editors. Intelligent Virtual Agents, 9th International Conference, IVA 2009, Lecture Notes in Artificial Intelligence, Vol: 5773. Springer Verlag, London, 2009.

27. ROILA., 2011.

28. Mihalcea R, Liu H, Lieberman H. NLP (Natural Language Processing) for NLP (Natural Language Programming), 7th International Conference on Computational Linguistics and Intelligent Text Processing, LNCS, Mexico City, 2006.

29. Veres SM. Natural Language Programming of Agents and Robotic Devices: Publishing for Agents and Humans in English, London, 2008.

30. Zadeh LA. From computing with numbers to computing with words - From manipulation of measurements to manipulation of perceptions, Int. J. Appl. Math. Comput. Sci., Vol.12, No.3, 2002, p. 307-324.

31. Nirenburg S, Raskin V. Ontological Semantics. Cambridge, MA: MIT Press, 2004.

32. Raskin V, Hempelmann CF, Taylor JM. Guessing vs. knowing: The two approaches to semantics in natural language processing. Annual International Artificial Intelligence Conference Dialogue 2010, Moscow, Russia, 2010, p. 645-652.

33. Taylor JM, Hempelmann CF, Raskin V. On an automatic acquisition toolbox for ontologies and lexicons in ontological semantics.

International Conference on Artificial Intelligence, Las Vegas, NE, 2010, p. 863-869.

34. Taylor JM, Raskin V, Hempelmann CF. From disambiguation failures to common-sense knowledge acquisition: A day in the life of an Ontological Semantic System. Web Iintelligence Conference, Lyon, France, 2011.

35. Kaindl H, Vallée M, Arnautovic E. Self-representation for self-configuration and monitoring in agent-based flexible automation systems. IEEE Transactions on Systems, Man, and Cybernetics 43(1), 2013, p. 164-175.