Scholarly article on topic 'Mapping Human Understanding to Robotic Perception'

Mapping Human Understanding to Robotic Perception Academic research paper on "Computer and information sciences"

CC BY-NC-ND
0
0
Share paper
Academic journal
Procedia Computer Science
OECD Field of science
Keywords
{"robotic ontology" / "ontology matching" / "granulation manipulation"}

Abstract of research paper on Computer and information sciences, author of scientific article — Julia M. Taylor

Abstract Humans are excellent at adapting their knowledge to various situations and adjusting their communication accordingly. Thus, a person who knows a great deal about a subject can still talk about it to a child, albeit in a much more simplified form. What is of interest here is whether a robot can do the reverse: in other words, can it adjust a limited knowledge that it receives from its sensors to a more complicated knowledge of the world that it doesn’t sense, but knows only abstractly? In other words, what kind of mapping is possible to adapt sensory knowledge to a more expressive knowledge of the world (or, in some cases, less expressive). When DARwin sees a red ball, does it really know that it is a ball? Can the fact that the object is moving in certain manner be leveraged for understanding that it is a ball? Similarly, when a robot or agent has access to a very specific domain, what has to happen to relate this knowledge to a more general domain? What kind of information has to be transferred and what can be omitted? The paper will review previous research in ontology mapping and alignment and, based on the existing research, propose some of the solutions.

Academic research paper on topic "Mapping Human Understanding to Robotic Perception"

(8)

CrossMark

Available online at www.sciencedirect.com

ScienceDirect

Procedía Computer Science 56 (2015) 514 - 519

International Workshop on Communication for Humans, Agents, Robots, Machines and Sensors

(HARMS 2015)

Mapping human understanding to robotic perception

Julia M. Taylor*

Purdue University, West Lafayette, IN, USA

Abstract

Humans are excellent at adapting their knowledge to various situations and adjusting their communication accordingly. Thus, a person who knows a great deal about a subject can still talk about it to a child, albeit in a much more simplified form. What is of interest here is whether a robot can do the reverse: in other words, can it adjust a limited knowledge that it receives from its sensors to a more complicated knowledge of the world that it doesn't sense, but knows only abstractly? In other words, what kind of mapping is possible to adapt sensory knowledge to a more expressive knowledge of the world (or, in some cases, less expressive). When DARwin sees a red ball, does it really know that it is a ball? Can the fact that the object is moving in certain manner be leveraged for understanding that it is a ball? Similarly, when a robot or agent has access to a very specific domain, what has to happen to relate this knowledge to a more general domain? What kind of information has to be transferred and what can be omitted? The paper will review previous research in ontology mapping and alignment and, based on the existing research, propose some of the solutions.

© 2015TheAuthors.Publishedby ElsevierB.V.This is an open access article under the CC BY-NC-ND license

(http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the Conference Program Chairs

Keywords: robotic ontology, ontology matching; granulation manipulation

1. Introduction

This paper takes a look at communication between robots and humans at a level that is closer to human-human communication than a less friendly agent/robot and human interaction1'2. It is assumed here that this communication does not have to be flawless - miscommunication is inherently human, but we are interested in getting to a level when a robot can correct its mistakes or at least see a human point of view. Similarly, a robot needs to know what it

* Corresponding author. Tel.: +1-765-494-9525. E-mail address: jtaylor1@purdue.edu

1877-0509 © 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the Conference Program Chairs

doi: 10.1016/j.procs.2015.07.244

doesn't know and what it cannot perceive. With the presence of such understanding - call it awareness if you wish (see other HARMS papers, for example3) - a robot can seek help where needed. In other words, what is of interest, first and foremost, is the acknowledgement of missing data or missing knowledge. It should be stated right away that such communication is not easy to achieve, and many factors have to be considered.

Let us go back to a human to human communication and trace where such awareness would be helpful. It should be noted, that it might be easier to program an agent to achieve detection of the missing data than to teach a human to recognize it. Consider the following scenario: a human receives a message from an airline that a flight is cancelled. A human calls the airline and discovers that the cancellation is due to a mechanical reason and thus, the airline pays for a hotel. The (human) agent confirms that the traveller could select a hotel and will be reimbursed. Notice that no information about the limit of the reimbursement has been exchanged. Upon completion of the trip, the traveller calls the airline again. Another agent takes information about the price and starts the claim. The traveler is asked to send a receipt to receive reimbursement. The traveler sends the receipt. Again, no high amount is discussed. Upon receiving reimbursement, the traveler discovers that the airline only covers partial amount because there is a limit, after all. Explicit knowledge and omitted information for each party is shown in Figure 1. The question is, why was the limit never mentioned? Was it an oversight or is there a more methodological explanation?

Airline's Known information

^ O M ^

Traveler's Known information

Fig. 1: Explicit and omitted knowledge in airline communication example

A possible explanation is that people state only what they don't consider to be obvious4'5 and both airline agents considered the limit being obvious. On the other hand, the traveler does not know that there is a limit and does not recognize the lack of such knowledge in his knowledge base. What is the solution? Can one be created if the travel agent is a machine not a human?

This problem can be generalized to any two entities' communication mechanism. Entity A possesses knowledge that it assumes entity B possesses, or it sees/senses it enough times that it doesn't present this information to entity B. Entity B, however, is not aware of the possibility of such information and thus does not know that it should acquire it. It could be argued that such instances are worse than contradictions as, in the contradiction, there is an explicit and obvious need for conflict resolution and finding an acceptable solution that does not contain surprises. We will deal with these situations through the ontological knowledge and ontological representation. We will assume that it is possible to map knowledge of each entity to an ontology. It is likely that the ontologies have some information in common and some that does not overlap. The question is, how to match or align such ontologies to produce a common knowledge source for the best communication (see Figure 2).

Ontological

Fig. 2: Mapping on sensor information to general ontology

2. Ontologies and Robotics

An ontology is an "explicit specification of conceptualization"6. In other words, an ontology is a specific description of concepts and relationships that are of interest for a particular domain that is useful for the agents that operate on this knowledge. The ontologies can be large or small, domain specific or general, existing at many levels of granulation. What ontology describes is usually dictated by the need of the application. In our case, the ontology has to be as general as it can be in order to accommodate conversations with human beings.

Ontologies has been used in the area of Robotics, and the necessity of their development and use have been

7 8) 9 10

outlined in many papers (for example, see ' Ontologies and semantics play a role in ubiquitous robotics ' , tasks in mobile robotics11, cognitive robotics models12.

There has been a whole array of research on creating and merging ontologies that overlap to some extent with the CHARMS interests pursued here. There are some pretty limited projects focusing on a particular resource, such as Japanese Wikipedia as a knowledge base for a robot16, on a specific application such as ontology for robotic vision17,

18 19 20 21 22

on ontologies for specific domains . Moving from the specific to general domains are other papers '' .. Some pursue a most general and promising direction of building everything on a solid knowledge base; some23, however, are an early and still hopeful attempt to address ontological problems with the indispensable tool of yesterday, machine learning. Paper24, a forthcoming review, is useful.

Unfortunately, the existence of ontologies by themselves is not necessarily enough for a streamlined and unambiguous communication. The reason for this is that even though there may be a standardized ontology, not everyone uses it. Thus, there is a need for ontologies mapping or aligning (what concept in a specific ontology corresponds to a concept in a different ontology) or merging of ontologies. Various methods for ontology merging have been introduced (for example13,14).

Most methods rely on ontologies constructed by the use of words, rather than what these words stand for. While this is helpful for communication in natural language and with humans, some of the complications in merging are not due to the actual task of merging but rather due to the ontology engineering methods. For example, consider some of the earlier examples15 used in comparing various meanings of water-flowing entities between English and French: "In English, size is the feature that distinguishes river from stream; in French, a fleuve is a river that flows into the sea, and a rivière is either a river or a stream that flows into another river. In translating French into English, the word fleuve maps to the French concept type Fleuve, which is a subtype of the English type River. Therefore, river is the closest one-word approximation to fleuve; if more detail is necessary, it could also be translated by the phrase river that runs into the sea. In the reverse direction, river maps to River, which has two subtypes: one is Fleuve, which maps to fleuve; and the other is BigRivière, whose closest approximation in French is the word rivière or the phrase grande rivière."

However, the word matching does not have to be performed at the ontological level. What does have to exist at the ontological level, in this case, is the property of flowing into the sea and a property of size. Each appropriate word can then be mapped to the appropriate concepts with the corresponding properties. The situation, then, can dictate a better word to use in either language. In other words, if we know that a drone accidentally "landed" in the

Potomac and it needs to be translated into French, the only question that should be raised in the translation is whether Potomac flows into a salty body of water. We will assume that such an ontological distinction is useful for our further discussion.

3. General Ontology and Domain Ontology Alignment

It is common to have an ontology for a particular domain, describing the needs and capabilities of the agents involved. Suppose such an ontology exists for a DARwin-OP humanoid robot. This means that a fine grain ontology contains the concepts corresponding to every part that the robot has (from which it is built), how these parts work together, the events that the robot can participate in, the range of sensors and what they can catch, etc. So far, the knowledge is limited to the robots capability: it can walk, it can sit, it can stand, etc. Notice, that, at a very high level, and from the point of view of a novice, these events only require legs as an instrument of action. However, as one becomes more involved, information about high-level instrument legs gets augmented with more knowledge. While it is unlikely that the actual computations will be described in the ontology, the parts required for these computations may very well be there. In other words, the difference between high description movement and low description movement may be very large.

Next, one needs to describe the environment that a robot moves in and could potentially perceive. For example, a red ball that DARwin can follow is not part of its system, yet it may be a good idea to describe it because it is used as one of the out-of-the-box demos. The description should be easy because the property color should have been defined before, based on the sensors that the robot has, and the properties describing the movement have been defined as well. It is possible that the property shape with the value sphere has not been introduced. However, for the most part, the concept ball should not cause any difficulty being inserted at the proper location, based on its properties (see Figure 3).

Fig. 3: (a) high level description (in green) compared to the rest of knowledge; (b) lower level description (in green) compared to the rest of knowledge; (c) introduction of a red ball in addition to the lower level description.

What happens, however, if a robot perceives something that it cannot identify? Suppose, it can see a red color, but unable to identify what it is. If that's the case, any object that could potentially be red is a fair game for recognition. One can play a simple game, by asking a human for help. The game would be similar to the 20 questions game, except that some of the answers are known from the sensor information. For what is not known, DARwin could find a property with the most differentiating features and ask whether the object contains this property and what value this property has. For example, what shape is the red object? This, however, assumes that the properties in the mind of a human being correspond to the properties defined by a computer. It is thus essential that the defined properties are as unambiguous as possible. It is perfectly possible to ask if the object can perform particular events on its own. The events, again, should be selected carefully from the knowledge that is described in Figure 3(b) as the ontological knowledge will be used to correlate the results from the human answer and should be as complete as possible. Notice also that, in this scenario, the robot is aware what properties are filled by the answers and which ones remain unanswered or unfilled. In other words, the robot is aware what it doesn't know.

Let us consider a different situation (see Figure 2). Suppose, the sensor resolution power is lower than that of a human being. For example, it can tell that an object is reddish, but not the exact shade of red. How will the mapping work then? In other words, it is physically impossible for the robot to differentiate something and it has to rely on human knowledge. This means that if a human describes an object that is lower than its resolution, it has to increase the grain size of this object to satisfy its constraints. The reverse process is likely to happen much more often: a

robot may have a resolution power that is of no use to a human being and the description has to come at a higher grain size. For example, a table that is 5.12 meters long by 3.74 meters wide is likely to be classified by a very large table by a human, rather than their being interested in its specific dimensions. The question is then, when should such a summarization be required? The answer is not a straightforward one when it is done for a human, and should depend on the topic of conversation and the domain of interest. However, it is easier when it is done for a robot as the robot doesn't have any stylistic appreciation of natural language: if the sensor resolution power is not enough, but other properties are available, a dive into a finer grain size of human-level ontology may be required.

An overall message of this section is this: it is hard to map ontologies to each other using concepts. It is considerably easier to map or align them using properties that describe the concepts. The properties, however, can become as difficult to use as the concepts themselves if they are non-simple properties. The issue becomes even more complex when composable properties25 such as size are involved. This happens because a) such properties can mean many different things to different ontology engineers; b) because it may not be clear how to map correctly these properties to their components - what, for example, is a relationship, between size and width, or size and length, or size and mass? This issue, though, will be of a concern once the more basic ontology mapping is done.

The similarity between properties is not a comparison of (the number of) labels that match. It must take into account the domain and range of a property, regardless of whether the label matches, as well as the ontological usage. For example, a property of size of a physical object is very different from the property of size of an event or of an abstract object. Thus, even if labels of the ontologies match, one must insure that they correspond (at least approximately) to the similar concepts. The similarity of properties, then, becomes a measure of similarity between domains and ranges much more than the actual labels. One may think that this is the chicken-and-egg problem - if you have to compare concepts using properties, and you have to compare properties using concepts, then where do you start? The answer lies in a realization that a robot (machine, agent, etc.) does not have to know everything and is allowed to ask for help. But, in asking for help, it should identify exactly where it struggles. In other words, understanding what you don't know is half of the battle. Instead of guessing an alignment solution where a starting point is unclear, it may be more productive to ask for help from a human for a property or two, or for a concept or two.

4. Specific Knowledge and Omissions

The next question that should arise is what can and cannot be omitted? If a robot is convening information about something, does it have to specify every single thing, or can some of them be assumed to be known? For example, suppose our DARwin needs to walk from a room into a hallway but the door is closed. What should be said in order for a human to open the door? Will could you open the door be sufficient? Or does it have to explain that it is closed: could you open the door, it is closed now? Such decisions are at the heart of communication without misunderstanding. The issue here is, how does one determine that the information that is not being made explicit is actually understood by the other party?

There are several cases that need to be considered. One is communication between two robots. On the one hand, one can transfer as much information as needed without omitting anything - a robot doesn't get tired from it. On the other hand, it may be an excellent case of testing what is known by another agent based on their ontological representation as mapped to the common ontology. The next case is communication between a human and a robot. Here, the decisions of what to omit is harder: at what level should the robot drop the details? Is it an object that is red and that is rolling on the floor, or can it be generalized to a ball? Is it a hotel room that has a limit on reimbursement or we can assume that miscommunication is allowed?

The answer might lie in negotiation between a human and a robot and building a profile of individual human knowledge and individual human preference. In other words, a robot has to have access to a general ontology and be able to acquire information that may be relevant for a particular situation. A computational agent should do the same thing. Moreover, domain specific properties may be assumed to be unknown by an average human being and thus, the information contained in such properties should be verbalized no matter what profile is obtained. Perhaps, in the future, a robot could prompt a human operator to disclose information that would be useful for a human user.

References

1. Matson, ET, Min B-C. M2M infrastructure to integrate humans, agents and robots into collectives. 2011 IEEE International Instrumentation and Measurement Technology Conference (I2MTC 2011), Hangzhou Binjiang, Hangzhou, China, 2011.

2. Matson ET, Taylor JM, Raskin V, Min B-C, Wilson EC. A natural language model for enabling human, agent, robot and machine interaction. The 5th IEEE International Conference on Automation, Robotics and Applications, Wellington, New Zealand, 2011.

3. Raskin V, Taylor JM, Matson ET. Towards an ontological modeling of something very much like consciousness: The HARMS way. Society for Design and Process Science Conference. Berlin, Germany, 2012.

4. Taylor JM, Raskin V, Hempelmann CF, Attardo S. An unintentional inference and ontological property defaults. IEEE SMC, Istanbul, Turley, 2010.

5. Ringenberg TR. Creating, Testing and Implementing a Method for Retrieving Conversational Inference with Ontological Semantics and Defaults. Master of Science Thesis, Department of Computer and Information Technology, Purdue University, 2015.

6. Gruber T R. Toward principles for the design of ontologies used for knowledge sharing. In: Guarino N, Poli R, editors. Special Issue on The Role of Formal Ontology in the Information Technology, International Journal of Human and Computer Studies 43:5-6, 1995, p. 907-928.

7. Haidegger T, Barreto M, Gonçalves P, Habib MK, , Ragavanf SKV, Li H, Vaccarell A, Perrone R, Prestes E. Applied ontologies and standards for service robots. Robotics and Autonomous Systems 61, 2013, p. 1215-1223.

8. Prestes E, Carbonera JL, Fiorini SR, Jorge VAM, Abel M, Madhavan R, Locoro , Goncalves P, Barreto ME, Habib M, Chibani A, Gérard S, Amirat Y, Schlenoff C. Towards a core ontology for robotics and automation. Robotics and Autonomous Systems 61, 2013, p. 1193-1204.

9. Chibani A,, Amirat Y, Mohammed S, Matson ET, Hagita N, Barreto M. Ubiquitous robotics: Recent challenges and future trends. Robotics and Autonomous Systems 61, 2013, p. 1162-1172.

10. Ayari N, Chibani A, Amirat Y, Matson E. A semantic approach for enhancing assistive services in ubiquitous robotics. Robotics and Autonomous Systems (forthcoming).

11. Kostavelis I, Gasteratos A. Semantic mapping for mobile robotics tasks: A survey. Robotics and Autonomous Systems 66, 2015, p. 86-103.

12. Lemaignan S, Mosenlechner RR, Alami R, Beetz M. ORO, a knowledge management platform for cognitive architectures in robotics. The 2010IEEE/RSJ International Conference on Intelligent Robots and Systems October 18-22, 2010, Taipei, Taiwan, 2010.

13. Kotis K, Vouros GA, Stergiou K. Towards automatic merging of domain ontologies: The HCONE-merge approach. Web Semantics: Science, Services and Agents on the World Wide Web 4, 2006, p. 60-79.

14. Guzman-Arenas A, Cuevas A-D. Knowledge accumulation through automatic merging of ontologies. Expert Systems with Applications 37, 2010, p. 1991-2005.

15. Sowa JF. Building, sharing and merging ontologies. http://www.jfsowa.com/ontology/ontoshar.htm 2009 (last modified).

16. Kobayashi S, Tamagawa S, Morita T, Yamaguchi T. Intelligent humanoid robot with Japanese Wikipedia Ontology and Robot Action Ontology. HRI'11, March 6-9, 2011, Lausanne, Switzerland, 2011.

17. Johnston B, Yang F, Mendoza R, Chen X, Williams M-A. Ontology based object categorization for robots. In: Yamaguchi T, editor. PAKM 2008, LNAI5345. Berlin-Heidelberg:Springer-Verlag, 2008, p. 219-231.

18. Lortal G, Dhouib S, Gérard G. Integrating ontological domain knowledge into a robotic DSL In: Dingel J, Solberg A, editors. MODELS 2010 Workshops, LNCS 6627. Berlin-Heidelberg:Springer-Verlag, 2011, p. 401-414.

19. Alirezaie M, Loutfi A. Towards automatic ontology alignment for enriching sensor data analysis. In: Fred A et al., editors. IC3K 2012, CCIS 415. Berlin-Heidelberg:Springer-Verlag, 2013, p. 179-193.

20. Kwiatkowski T, Choi E, Artzi Y, Zettlemoyer L. Scaling semantic parsers with on-the-fly ontology matching. EMNLP, 2013.

21. Ayari N, Chibani A, Amirat Y. Semantic management of human-robot interaction in ambient intelligence environments using n-ary ontologies. 2013 IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, 2013 .

22. Tenorth M, Beetz M. KnowRob: A knowledge processing infrastructure for cognition-enabled robots. The International Journal of Robotics Research 32(5), 2013, p. 566-590.

23. Doan A, Madhavan J, Domingos P, Halevy A. Ontology matching: A machine learning approach. Proceedings of the 11th international conference on World Wide Web, 2002, p. 662-673.

24. Shvaiko P, Euzenat J. Ontology matching: state of the art and future challenges. IEEE Transaction on Knowledge and Data Engineering (forthcoming).

25. Taylor JM, Raskin V. On the nature of composable properties. Proceedings of AAAI'14 Workshop on Cognitive Computing for Augmented Human Intelligence, 2014.