Scholarly article on topic 'Immersive and Haptic Educational Simulations of Assembly Workplace Conditions'

Immersive and Haptic Educational Simulations of Assembly Workplace Conditions Academic research paper on "Computer and information sciences"

CC BY-NC-ND
0
0
Share paper
Academic journal
Procedia Computer Science
OECD Field of science
Keywords
{"virtual reality" / "haptic devices" / "virtual workplaces" / "immersive simulations ;"}

Abstract of research paper on Computer and information sciences, author of scientific article — Damian Grajewski, Filip Górski, Adam Hamrol, Przemysław Zawadzki

Abstract The paper presents a number of different approaches for creation of realistic immersive educational simulations of conditions of a workplace for assembly operations with aid of haptic and Virtual Reality systems. Such simulations can be used for effective training of future operators of a given workplace and testing its ergonomic quality without need of building a physical prototype. The authors have devised several different approaches to preparation of virtual workplace prototypes for training. The basic approach is utilization of immersive VR systems (Head-Mounted Devices combined with user tracking solutions) to create an interactive simulation of a workplace, on the basis of its CAD model and specification of a realized process. The second approach is using a haptic manipulator with force feedback for interaction with a virtual workplace, for more realistic feeling of a realized activity. The third, novel approach is combination of the two previous approaches. A special system was designed and built for this purpose. It uses a large-space robot to allow haptic feedback for a user equipped with an immersive set of devices. More and more often virtual prototypes of the workplaces replace the physical ones as an innovative training solutions that allow to train the future operators or improve ergonomic quality of the workplace. The reason for that is the reduction of costs needed to develop the real prototype of the workstation as well as ability of complex virtual workplaces to change the variants in order to investigate different situations. Usually such workplaces - created as interactive Virtual Reality applications - consist of a visualization operated with aid of advanced Virtual Reality systems and devices. Implementation of an immersive equipment increases sense of user's presence and use of haptic devices with force feedback makes the experience more similar to reality. The level of realism is a very important factor in an immersive and haptic simulations of workstations for training purposes. As a form of instructional simulation, also called educational simulation, immersive and haptic simulations are a powerful learning tools that require trained users to complete tasks or to solve specific problems within VR environment that replicates the real workplace conditions19. Trainings conducted is this way are particularly beneficial when real activities and task conducted at the workplace is too dangerous, too costly or even almost impossible to do. The two basic approaches presented by the authors are innovative and important on their own, but they usually do not mix, as state-of-the-art haptic devices are stationary. Their application heavily limits spatial freedom of a user, which does not affect the level of realism of presented simulations in a positive way. In order to fully simulate the conditions of an assembly workplace, tactile input must be available, with simultaneous immersion of a user into a Virtual Environment (VE) using a HMD or a CAVE system combined with the user tracking solution. The third approach to simulation of workplace conditions, proposed by the authors, allows to bring a realistic force feedback into an immersive visualization. Thanks to this, operations like assembling a threaded joint between two parts can be trained more realistically than before, without need of building a full physical workplace.

Academic research paper on topic "Immersive and Haptic Educational Simulations of Assembly Workplace Conditions"

CrossMark

Available online at www.sciencedirect.com

ScienceDirect

Procedía Computer Science 75 (2015) 359 - 368

2015 International Conference on Virtual and Augmented Reality in Education

Immersive and Haptic Educational Simulations of Assembly

Workplace Conditions

Damian Grajewskia*, Filip Górskia, Adam Hamrola, Przemyslaw Zawadzkia

a Poznan University of Technology, Chair of Management and Production Engineering, Piotrowo Str. 3, 60-965 Poznan, Poland

Abstract

The paper presents a number of different approaches for creation of realistic immersive educational simulations of conditions of a workplace for assembly operations with aid of haptic and Virtual Reality systems. Such simulations can be used for effective training of future operators of a given workplace and testing its ergonomic quality without need of building a physical prototype. The authors have devised several different approaches to preparation of virtual workplace prototypes for training. The basic approach is utilization of immersive VR systems (Head-Mounted Devices combined with user tracking solutions) to create an interactive simulation of a workplace, on the basis of its CAD model and specification of a realized process. The second approach is using a haptic manipulator with force feedback for interaction with a virtual workplace, for more realistic feeling of a realized activity. The third, novel approach is combination of the two previous approaches. A special system was designed and built for this purpose. It uses a large-space robot to allow haptic feedback for a user equipped with an immersive set of devices. More and more often virtual prototypes of the workplaces replace the physical ones as an innovative training solutions that allow to train the future operators or improve ergonomic quality of the workplace. The reason for that is the reduction of costs needed to develop the real prototype of the workstation as well as ability of complex virtual workplaces to change the variants in order to investigate different situations. Usually such workplaces - created as interactive Virtual Reality applications - consist of a visualization operated with aid of advanced Virtual Reality systems and devices. Implementation of an immersive equipment increases sense of user's presence and use of haptic devices with force feedback makes the experience more similar to reality. The level of realism is a very important factor in an immersive and haptic simulations of workstations for training purposes. As a form of instructional simulation, also called educational simulation, immersive and haptic simulations are a powerful learning tools that require trained users to complete tasks or to solve specific problems within VR environment that replicates the real workplace conditions19. Trainings conducted is this way are particularly beneficial when real activities and task conducted at the workplace is too dangerous, too costly or even almost impossible to do.

* Corresponding author. Tel.: +48 61 665 2718; fax: +48 61 665 2774 E-mail address: damian.grajewski@put.poznan.pl

1877-0509 © 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of organizing committee of the 2015 International Conference on Virtual and Augmented Reality in

Education (VARE 2015)

doi: 10.1016/j.procs.2015.12.258

The two basic approaches presented by the authors are innovative and important on their own, but they usually do not mix, as state-of-the-art haptic devices are stationary. Their application heavily limits spatial freedom of a user, which does not affect the level of realism of presented simulations in a positive way. In order to fully simulate the conditions of an assembly workplace, tactile input must be available, with simultaneous immersion of a user into a Virtual Environment (VE) using a HMD or a CAVE system combined with the user tracking solution. The third approach to simulation of workplace conditions, proposed by the authors, allows to bring a realistic force feedback into an immersive visualization. Thanks to this, operations like assembling a threaded joint between two parts can be trained more realistically than before, without need of building a full physical workplace. © 2015 The Authors. Publishedby ElsevierB.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Peer-reviewunderresponsibilityoforganizing committee of the 2015 International Conference on Virtual and Augmented Reality in Education(VARE2015)

Keywords: virtual reality; haptic devices; virtual workplaces; immersive simulations;

1. Introduction

Over the last several years, a significant evolution of the methods and techniques of Virtual Reality (VR) as well as haptic technologies and devices can be observed. Immersive VR technologies are gaining wider and wider use as engineering design tools to support the product design phase, because of their ability to deliver immersive and user friendly environment that can be used as digital test-bed for prototypes of the product24. They allow placement of a CAD model in a Virtual Environment (VE), in presence of other virtual objects6. VR techniques are also more and more often used as decision-making support systems and advanced training systems as the user/trained person can explore VEs for educational and exercise purposes2.

The main feature of VEs is mapping of mutual interactions of objects and their behavior, in response to events caused by the user or by other objects. VR simulation is different from a typical computer simulation, because the user is always present in the Virtual Environment. Degree of immersion of user in a virtual world depends on the simulation scenario, as well as on the used peripheral devices. The influence of a user on a VE may be changed to a considerable extent, also directly during the simulation. Besides of influencing the sense of sight and hearing, an immersive simulation can be supplemented with tactile feedback that significantly affects the degree of immersion. Tactile feedback is achieved using haptic devices with a force feedback effect (e.g. phantom, glove, manipulator) equipped with a number of sensors, which record parameters like direction and velocity of user's movement7. User is given an opportunity to touch a virtual object, but also can manipulate this object in space and shape its geometry.

VR simulations are also very often expanded with interaction with physical prototypes of investigated objects, integrated with the virtual world, for further improvement of realism of the simulation6.

Large potential of VR systems integrated with haptic technologies and wide range of possibilities of creation of virtual worlds makes these technologies perfect tools for development of interactive training systems. Such systems take a form of haptic, immersive educational simulations that ensure realism and safety of trained/educated users. This is very important especially in field of work safety and ergonomics. Possibility of testing and simulating numerous configurations of the "human-machine" system (manufacturing workplace is a special case of such a system) in a VE reduces costs of such a research work and it is far safer. For example, a lot of dangerous, health-threatening conditions can be tested without required presence of the machine operator in order to prevent and reduce the number of workplace accidents involving operators. Prototypes of the virtual workplaces allow conducting an immersive training of future operators prior to implementation of the real workplace that reduces the time needed for the machine operator to gain skills necessary to work effectively. Immersive and haptic training conducted in a VE permits learning of correct behaviors and reactions and examining workplaces presented in a training scene in a better way than traditional educational methods based on physical prototype of the workstation. Use of VR and haptic technologies may therefore significantly reduce time and cost of implementation of a new manufacturing workplace7,11.

This paper presents different approaches of creation of immersive and haptic educational simulations of conditions of a workplace for assembly operations with the aid of haptic and Virtual Reality systems.

2. VR and haptic systems aiding the process of virtual workplace design

VR and haptic technologies are the most noteworthy techniques aiding the process of construction of workplace prototypes. They allow to design and test virtual prototypes of workplaces without necessity of building their physical models. A detailed analysis and optimization of applied design solutions of the workplaces can be performed, as well as interactive trainings of the future users/operators in order to gain the skills necessary to work effectively7.

2.1. VR and haptic systems for workplace design

Scientifically, Virtual Reality is defined as an application of the computer technology to generate an effect of an interactive, three-dimensional world, in which objects have spatial form. A computer-generated environment with a stereoscopic visualization is a basis for each VR solution. Interactive control over the presented image is very important and gives the user that explores the VE a sense of feeling of presence and of being a part of an immersive virtual scene, not from the position of observer, but as a participant of a virtual simulation. Interaction allows user to control the virtual object and the whole virtual scene in real time23. The immersion is achieved mostly by stereoscopic projection, giving a user an illusion of spatial depth. There are several methods of obtaining the stereoscopic image:

• Active projection with shutter glasses

• Passive projection with polarized glasses

• Personal projection (e.g. Head Mounted Displays presented on Fig.1a, CAVE™ systems)

Fig. 1. (a) nVisor MH-60V HMD, (b) SensAble Phantom Premium 3.0/6 DOF haptic manipulator.

Haptic technology allows users to interact with a Virtual Environment through tactile feedback. A haptic device is equipped with a number of sensors that record a set of parameters (e.g. direction and velocity of the movement). These parameters are processed and in effect, the user is receiving feedback, for example through vibration at selected locations with appropriate amplitude and frequency12,20. A typical haptic system consists of a central computer unit (hardware + software) and an external haptic device. Usually the haptic device resembles a three or six-jointed industrial manipulator. The arm of the manipulator is designed to be operated by user and it is composed of straight segments connected by cylindrical joints (Fig. 1b).

Application of VR and haptic systems allows designing and presenting a virtual workplace prototype in its real operation environment, limiting the need for use of real mock-ups. Therefore, an effective process of conduction a number of analyses related to designed prototypes can be realized. An example of such analysis is examination of dimensions of devices and possibilities of adjustment to height of the human operator or arrangement of control and

signaling devices according to the rules of ergonomic design. To conduct such an analysis, full interaction between a user and a virtual workplace must be programmed, including collision detection, kinematics of the devices and possibilities of activating their various functions in relation with other objects in the virtual scene. If a haptic technology is used, the force response can be programmed, giving a user impression of touching virtual objects. If an immersive technology is used, a user can navigate the Virtual Environment by walking and can interact with various objects by natural movement and gestures. Pure virtual prototypes can be also complemented with physical prototypes, further increasing the realism of the simulation. The more realistic the virtual prototype is, the more tests can be performed solely in the VE, thus reducing a need for the physical prototype7.

Various researchers have investigated different aspects of usage of Virtual Reality and haptic technologies in a complex process of product and process design. A lot of investigations deal with use of VR and haptic technologies for industrial process verification3. As a result, a number of VR and haptic systems concerned with assembly processes simulation have been presented as virtual assembly applications14,15,16,17,24,27. The authors analyzed the scientific results of different investigations, e.g. systems for haptic assembly and realistic prototyping24, haptic based virtual assembly systems for the generation of assembly process plans22 and other environments for virtual assembly design9,10,14.

The authors of this paper took the results of the research work described in the publications mentioned above into consideration and focused their work on effective use of VR and haptic technologies in the design process of virtual workplaces characterized with a high level of ergonomic quality.

2.2. Ergonomic aspect of virtual workplace design

Ergonomic aspects of virtual design of workplaces were also analyzed very carefully and presented in previous work by the authors7. In research area of ergonomics, Virtual Reality technologies are often used to improve not only the quality of the working conditions of the workplace operator, but that of the products, the environment, the systems and the employee training and education8. Chryssolouris et al. used methods based on VR technologies to recognize and analyze factors associated to human performance in the components assembly task. In this research work, ergonomic models were embedded in a immersive environment to conduct an ergonomic analysis of the work conditions3,21. For the purposes of the automotive industry, Grave et al. developed an educational training VR system for future operators on assembly lines of electric cables8,21. Whitman et al. conducted investigations based on performing a simple task like handling and transportation of volumes in the real world and in the virtual world. They compared the results obtained by carrying out that task to understand whether VR is suitable for use in an ergonomic analysis. The results obtained by comparing the movements of the torso showed that Virtual Reality indeed can be compared to a real situation if the evaluations are limited to the range and movements measurement, but without measuring velocity and accelerations21,25. In conclusion, VR environments may be effective solutions for complex analysis of virtual prototypes of workplaces from the viewpoint of meeting the rules of ergonomics. If a virtual workplace is designed in an appropriate manner, the trained operator gains possibility to work in the most effective way possible. Of course risk of injuries and stress is therefore seriously limited13.

2.3. Virtual assembly applications - short review

In order to work on the effective use of VR and haptic systems in the process of creation of immersive educational simulations of assembly workplace conditions, deep analysis of existing virtual assembly applications used for assembly processes simulation was conducted. Grupta et al. developed virtual assembly application named VEDA (Virtual Environment for Design for Assembly) which used physically based modeling for modeling a part behavior24. The VEDA simulation model used visual, auditory and haptic interfaces to enable estimation of ease of part handling and part insertion in assembly processes providing multi-modal simulation3. Another system called IVY allows designers to exploit inherent knowledge of assembly factors within an interactive immersive VE. Designers received an effective tool aiding the evaluation and visualization of candidate assembly plans1,3,17. Jayram et al. developed VADE (Virtual Assembly Design Environment) to demonstrate usage of Virtual Reality tools for the feasibility of creating design information (e.g. tolerance, optimal sequence of components or assembly/disassembly plans)3,14. Ritchie et al. developed the HAMMS system (Haptic Assembly, Manufacturing

and Machining System) to investigate user's response during the process of conducting various engineering tasks in a VE. The HAMMS system identified assembly operations and extracted them to generate the assembly plan. Optimization of assembly operations by elimination of unnecessary ones is supported as well22. Seth, Su and Vance developed the SHARP system (System for Haptic Assembly and Realistic Prototyping) that is a portable VR interface for virtual assembly. By the use of physically based modeling SHARP enable simulation of part-to-part and hand-to-part interactions in Virtual Environments24.

The authors of this paper deeply analyzed the results of the research work briefly described above and focused their work on the effective use of VR and haptic technologies in the development of educational immersive simulations of assembly workplace conditions.

3. Approaches to design virtual workplace for the purpose of educational simulation

In order to prepare immersive and haptic educational simulations of assembly workplace conditions, three approaches to building virtual workplaces were conducted. For the presentation of these three approaches, different VR devices and workplace models were used. Such simulations can be used for effective training of future operators of a given workplace and testing its ergonomic quality without need of building a physical prototype. Separate educational simulations were built to present methodology of creation of virtual workplaces as well as ability of VR and haptic systems for an effective interactive training. The experimental VR setup consisted of two separate stands: the first one with a 3D large screen display with a stationary, commercially available high-end haptic device, the second one with a high-end Head-Mounted Device with an optical tracking system and gesture recognition gloves. The third stand was expanded with a dedicated robot, which moves in response to actions of a tracked user, allowing real-time obstacle generation, to bring haptic feedback into an immersive simulation. Several examples were prepared for all three approaches, using a Virtual Reality authoring tool for preparation of virtual scenes and programming of simulation logic. The exemplary virtual workplaces were tested by unexperienced users for intuitiveness and sense of realism.

4. Immersive and haptic educational simulation

Learning simulations are empirical simulations that produce a virtual event or VE to be explored by a user (learner). The main feature of a simulation is the re-creation of a specific aspect of reality in an interactive way18. Learning is a one of the very important factors of companies' business success that can't be achieved without well skilled staff26. Nowadays, companies are investing large funds for professional skill trainings development, especially in the industrial sector5. The possibility of developing virtual simulation trainings (e.g. heavy equipment operators, underground mining machine operators or users of complex and expensive workplaces) reduces the time and cost of trainings and also is more flexible to carry out the modification or change a variant of a workplace.

Immersive and haptic simulations are one of the types of the workplace simulations. As a form of instructional simulation, also called an educational simulation, immersive and haptic simulations are powerful learning tools that require trained users to complete tasks or to solve specific problems within a VE that replicates real workplace environments19. Simulations "result in the acquisition of complex and higher-order skills, as people learn by experience, taking real-life risks, without bearing the costs of making mistakes"4. Trainings conducted this way are particularly beneficial when real activities and tasks conducted at the workplace are too dangerous, too costly or even almost impossible to do.

With a view to the above-described benefits resulting from the use of immersive simulations, authors focused also on the integration of the hardware. In one of the presented approaches of realistic educational simulations of conditions of a workplace for assembly operations, integration of an active touch device with force feedback effect together with a HMD and tracking system was conducted. The role of an active touch device was served by delta robot - a low-cost device. The authors have developed a special algorithm for positioning of the end effector (empowered frame) of the delta robot according to the tracked object (user's hand). According to authors knowledge this approach is a novel one. Currently first simulations of a selected assembly operations of varying difficulty were already conducted in order detect possible limitations of this technology. It will be presented in next chapter.

5. Examples of immersive and haptic training applications

Three educational immersive simulations of the assembly workplace conditions with the aid of VR and haptic systems were prepared. First example is a set of two virtual workplaces for hole drilling and manual assembly of two halves of a simple clutch. In this case, immersive approach was used, namely HMD device integrated with tracking and gesture recognition systems. Second case study is a virtual workplace for stud welding, where stereoscopic projection system was integrated with a haptic manipulator. The third example is a novel approach. It is combination of the two previous approaches. A special system was designed and built for this purpose. It uses a low cost delta robot to allow haptic feedback for a user equipped with an immersive set of devices. According to the authors' knowledge, it is a good direction to follow, as it combines advantages of the immersive experience with force response. However, at the present stage, it is not elastic enough and very difficult in preparation of a realistic scene. The presented solution is currently being developed and improved and will be expanded in future for both more complex use cases and more realistic force feedback response to user's actions. In this paper authors present the main assumptions about the proposed solution and preliminary results of the first experiment of assembly operation which was screwing bolts into the body of a motor.

The core of methodology of creation of virtual workplaces is practically the same in presented examples.

5.1. Methodology of creation of virtual workplace training application

Each virtual workplace is a system consisting of an appropriate VR hardware (e.g. HMD or CAVE system, tracking and gesture recognition devices for interaction and / or haptic devices) and a VR application that is always a heart of such a system. In the sense of programming, VR application should be defined as one, closed entity, containing virtual models of objects placed in a properly defined environment, ensuring the user with interaction and immersion. Stages of VR application development are presented graphically in Fig. 2.

L CAD model ^^ ■■HH

transfer into VR w jvior

V environment f * Programm ■■■ ing

Fig. 2. Stages of VR application development '

The first stage is the CAD model transfer. It consists in conversion of the parametric solid or surface model created in the selected system to a polygon mesh and then an import of the converted model to a VR environment. This stage includes work related to improvement of 3D object appearance, by application of visualization techniques, such as texture mapping, lighting effects, reflections etc.

The main stage of creation of a VR application is programming of objects behavior. Standard object behaviors include mostly movement as a response to a user-generated event or an event generated by another object. Other object behaviors are deformations (dynamical shape change) or changes in appearance (smooth change of transparency, color, texture etc.). Method of behavior programming highly depends on used software, frequently there is a possibility of utilizing methods of so-called visual programming, consisting in making connections between nodes exchanging data of various types. In any case, object properties must be set first (e.g. a kinematic node needs to be informed of the target position and movement velocity), then connections can be made.

Creating user interface is also an important stage of VR application development - the interface must be intuitive and allow easy launching of all the necessary functions of the virtual model. Almost always some form of the Graphical User Interface (GUI) is in use, in connection of traditional input devices (mouse and keyboard) or with tracking systems, where movement of user's hands, fingers, legs or heads are registered as input. In applications where user is expected to move around a room, 3D interface is often used, where buttons, sliders etc. take a form of animated 3D solid objects with defined collision zones6.

The final stage is the application testing, verification of the studied workplace and possible further iterations (correction of the CAD model, performing import stage once again, object adjustment and testing functionalities again). Finished application may be used in the process of education of future operators of the workplace, which will shorten the time needed to introduce the operator to work on the new workplace with full efficiency7.

5.2. Drilling and manual assembly virtual workplace

Immersive educational simulation of a double workplace for drilling and manual assembly is the first example of interactive training for the future operators. In this case immersive approach was used, namely HMD device was integrated with tracking and gesture recognition systems. Based on CAD models of two workplaces (drilling work stand and manual assembly work stand), interactive model of these joined workplaces was developed in VE (Fig. 3a). On the drilling work stand holes are made in one half of the pin coupling and the assembly work stand is used to join both the halves together with nuts and bolts. Aim of the immersive simulation performed in the VE was to study the indirect interactions between two users immersed in the virtual scene. Future operators of these two workplaces trained different scenarios of cooperation during the production process in order to optimize their effective work.

Fig. 3. (a)Virtual workplace for drilling and assembly, (b) Manipulation on the drilling workplace with application of hand tracking, gesture

recognition gloves and a wrench model.

In order to enhance the immersion of the user, educational VR simulation was improved by adding the hybrid prototyping (mixing virtual models with real objects) method. The hybrid approach consisted in application of physical tool models of a wrench, used in the manufacturing workplace. In connection with the hand and head tracking and a gesture recognition device (Fig. 3b), a possibility of representing almost every activity performed on both workplaces was obtained (among other things: control of the stationary drilling machine, fixing and unfixing the pin coupling half, assembling two pieces of pin coupling together etc.).

The effectiveness of the immersive training performed in VR system was tested using a sample of experienced (users that know the real workplace) and non-experienced users. Both groups of users left their impressions in a survey study. It confirmed high effectiveness of the system in terms of presented conditions very similar to reality. Immersive aspect of the system was also valued very high - usage of the stereoscopic 3D and Head-Mounted Device allowed to present the workplaces in a more realistic way than it was possible earlier. The experienced users were additionally asked to give their opinion about the level of quality of professional skills that can be learned during the immersive training. Based on the results of the survey authors plan to improve the immersive training by adding new scenarios and new tasks that should be conducted in the assembly process.

5.3. Stud welding virtual workplace

Second example of interactive training for the future operators is an immersive and haptic educational simulation of a virtual workplace for stud welding. In this case immersive and haptic approach was used as stereoscopic projection system was integrated with haptic manipulator (Phantom Premium 3.0). Model of the workplace was prepared in a CAD 3D system, according to guidelines prepared for the real manufacturing workplace.

Aim of the immersive and haptic simulation of the stud welding workplace conditions performed in the VR environment was to conduct and train several activities:

• Device control (turn the welding machine on/off, table rotation, opening / closing the pneumatic clamps)

• Placing the semi-finished product (sheet metal) on the workplace

• Stud welding using a special tool (weld gun)

Similarly to the first example, in order to enhance the immersion, a hybrid prototyping method was added. Hybrid approach consisted in application of a physical model of the weld gun. On the basis of the CAD model of the weld gun, a solid model was generated and subjected to adaptation by adjusting the inner shape to geometry of the haptic manipulator end effector. Then, the physical prototypes were prepared using additive technology of Fused Deposition Modeling, obtaining easy to assemble model of the gun. Connection between physical and virtual representation of the tool was prepared. System of collision detection between the gun and its environment was implemented and expanded with the force feedback. Position of the manipulator's end effector was connected to position of the virtual gun. As a result, an immersive and haptic educational application that allows training of various aspects of the stud welding process was created (Fig. 4a and 4b).

Fig. 4. (a) Virtual stud welding workplace, (b) manipulation using a physical model of the weld gun on a haptic manipulator.

The Phantom Premium 3.0 manipulator has workspace of comparable size to workspace on the stud welding workplace, so there were no practical limitations in testing of the welding process itself, however all the supporting activities (turning the devices on/off, placing the sheet metal on the working table) still had to be tested using regular hand tracking. The effectiveness of the immersive and haptic training was also tested. In this case, only a sample of experienced users was used. The reason was that the process of stud welding is a complex process and authors wanted to receive only the impressions of experienced operators that know the technology very well. It was very important as this training application is planned to be an effective tool to improve the basic skills of the operators.

5.4. Integrated immersive and haptic virtual workplace for manual assembly

The third example is a novel approach. Authors developed an integrated immersive and haptic training system of manual assembly virtual workplace. User can explore an interactive educational simulation that uses a low cost delta robot to allow haptic feedback for a user integrated with HMD and tracking system. After the process of study the integration possibilities of various kinds of interaction devices, physical objects and tracking technologies, it was decided to develop a light low-cost delta robot (Fig. 5a) that will improve the human - immersive VR communication and interaction.

In the beginning, the main task of the robot was to move its ending arm to the position in which the operator's hand may touch a visualized object (e.g. wall or table) in order to expand the immersive VR system with tactile feedback. The authors have developed a special algorithm for positioning of the end effector (empowered frame) of the delta robot according to the tracked object (user's hand). The delta robot was initially designed and simulated in a Matlab-Simulink environment. Its kinematic parameters were determined, along with control system development.

Then, a realistically behaving virtual prototype of the robot was created using available VR systems. The next step was to prepare a stand-alone physical device. The method of communication between the robot and VR environment was established via the local network. After the positive tests of the device, a special object representing user's hand was added to VR environment. Position of the virtual hand was connected with data coming from the optical tracking system (the PPT X system by WorldViz company). The user's hand position in space is tracked in real time as long as he holds an optical system marker (or a device with one attached). The virtual hand position data was used to establish the future XYZ position of the virtual robot end-effector. In order to control the robot in such a way, that it acts like a haptic device and represents a shape of a virtual object, a set of constraints is defined in real time by algorithms created by the authors. To represent a simple virtual box (in form of a cube) which can be touched by the user, coordinates in VR of its two opposite corners were entered to a script, which calculates the robot position in virtual space.

Fig. 5. (a) Delta robot, (b) robot's end effector representing the surface of the virtual object, (c) immersive and haptic simulation of the manual

assembly.

This calculation is done during the simulation made in real-time, which allows adaptation of position of the robot to represent objects of various sizes. The robot is not allowed to reach any position outside the defined box, but its position is controlled by this way, that it is always as close to the user's hand as possible. By this way, the user can touch the robot's end effector representing the surface of the virtual object and move his hand on the surface, with the robot following user's path and creating an illusion of touching of a complete object (Fig. 5b). Any other shape can be also defined by this way, as a set of coordinates defining a 3D object, inside which the robot can move. The position of the virtual robot is calculated using the application which works outside the virtual reality application (software), and which uses a network node to send the data to another application, responsible for controlling of the real position of the active haptic device. The whole system operation was tested using the HMD (nVis MH60V device). The test results were promising and it was decided to develop this approach.

In order to enable the simulation of manual assembly for the purposes of immersive and haptic educational simulation, it was decided to modify the end effector by adding a small DC engine with transmission and encoder, in order to simulate the real resistance that can be felt by the user during the process of screwing bolts into the body of a motor (Fig. 5c).

According to the authors knowledge this approach is a novel one, because the realism of the manual process is enhanced with force feedback generated from the DC engine. Currently, first simulations of a selected manual assembly operation were conducted in order to detect possible limitations of this technology. Preliminary group of users (students of the mechanical engineering course:30 people) tested the virtual work stand. Their first impressions gave authors new directions to follow in further development of this solution. In the near future the training system will be expanded for more complex use cases and even more realistic force feedback response to user's actions.

6. Conclusions

The research and literature study showed that virtual and haptic technologies can be used for effective training of future operators of a given workplace and even testing its ergonomic quality, without need of building a physical prototype. Different approaches in creation of immersive and haptic educational simulations of workplace conditions that were presented have of course their pros and cons. The immersive solution on itself is realistic if it comes to exploration and initial learning of the workplace for the unexperienced operator. However, lack of tactile feedback makes it difficult to conduct a full training for future operators or to verify if the workplace is ergonomic. The haptic approach lacks the immersive quality, as the user remains stationary during operation. Limited workspace of even the most versatile commercially available devices also decreases the experience. Still, application of a haptic device allows testing ergonomics and teaching operations on the virtual workplace. The innovative, third approach that is currently under investigation is a good direction to follow, as it combines advantages of the immersive experience with force response.

References

1. Angster S, Jayram S. Open architecture framework for integrated virtual product development systems. Int. J Virtual Reality 1997; 3(1):6-18

2. Burdea G, Coiffet, P. Virtual reality technology. Presence: Teleoperators and virtual environments 2003: 12(6):663-664.

3. Chryssolouris G, Mavrikios D, Fragos D, Karabatsou V. A virtaul reality-based experimentation environment for the verification of human-

related factors in assembly processes. Robotics and Computer Integrated Manufacturing 2000;16:267-276.

4. Chuang CK, Chang M, Wang CY, Chung WC, Chen GD. Application of E-Learning to Pilot Training at TransAsia Airways in Taiwan. International Journal on E-Learning 2008;7(1):23-39.

5. Dealtry R. Global corporate priorities and demand-led learning strategies. Journal of Workplace Learning 2008; 20(4): 286-287.

6. Górski F, Hamrol A, Grajewski D, Zawadzki P. Integracja technik wirtualnej rzeczywistosci i wytwarzania przyrostowego - hybrydowe podejscie do rozwoju wyrobu. Mechanik 2013; 3&4: 173-176 & 266-270.

7. Grajewski D, Górski F, Zawadzki P, Hamrol A. Application of Virtual Reality Techniques in Design of Ergonomic Manufacturing Workplaces. Procedia Computer Science Journal 2013; 25:289-301.

8. Grave L, Escaleira C, Silva AF, Marcos A. . A Realidade Virtual como Ferramenta de Treino para Montagem de Cablagens Eléctricas. Proceddings of 10° Encontro Portugués de Computagáo Grafica 2001;147-63.

9. Grupta R, Zeltzer D. Prototyping and design for assembly analysis using multimodal virtual environments. ASME Computers in Engineering

Conference 1995.

10. Grupta R, Whitney D, Zeltzer D. Prototyping and design for assembly analysis using multimodal virtual environments. Computer Aided Design 1997; 29(8):858-597.

11. Hamrol A, Górski F, Zawadzki P, Grajewski D. Virtual 3D Atlas of a Human Body - Development of an Educational Medical Software Application. Procedia Computer Science Journal 2013; 25:302-314.

12. Hayward V, Choksi J, Lanvin G, Ramstein C. Design and Multi-Objective Optimization of a Linkage for a Haptic Interface, Advances in Robot Kinematics and Computationed Geometry 1994; 359-368.

13. Januszka M. Projektowanie ergonomiczne z zastosowaniem technik poszerzonej rzeczywistosci, XIForum Inzynierskiego ProCAx 2012.

14. Jayram S, Connacher HL, Lyons KW. Virtual assembly using virtula reality techniques. Computer Aided Design 1997; 29(8):575-584.

15. Kim CE, Vance JM,. Using Vps ast the basis for interaction in a virtual assembly environment. ASMEDETC 2003.

16. Kim CE, Vance JM,. Collision detection and part interaction modelling to facilittate immersive virtual assembly methods. ASME Journal of Computing and Information Sciences in Engineering 2004; 4(1):83-90.

17. Kuehne R, Oliver J. A virtual environment for interactive assembly planning and evaluation. ASME DETC 1995.

18. Landriscina F. Simulation and Learning: A Model-Centred Approach. Springer 2013

19. Mallinson B, Miller F. A Theoretical Model for Online Simulation Design and Development. In Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2006;1680-1687.

20. Mleko A, Kotlinski T. Interfejsy haptyczne i force feedback, Informatyka Stosowana EAIiEAGH, 2008.

21. Rebelo F, Duarte E, Noriega P. Virtual Reality in Consumer Product Design: Methods and Applications. Methods and Techniques. Taylor and Francis Group, LLC. 2011.

22. Ritchie JM, Lim T, Medellin H, Sung RS. A haptic based virtual assembly system for the generation of assembly process plans. Memorias del XV Congresso Internacional Anual de la Somim 2009;CD:585-593.

23. Robles De La Torre G. Principles of Haptic Perception in Virtual Environments, Human Haptic Perception: Basics and Applications, 2008.

24. Seth A, Su H-J, Vance J.M. SHARP: A System for Haptic Assembly&Realistic Prototyping. In Proceedings of the DETC'06 2006; 1-8.

25. Whitman LE, Jorgensen M, Hathiyari K, Malzahn D. Virtual reality: Its usefulness for ergonomic analysis. Winter Simulation Conference 2004;1740-45.

26. Vassileva T, Astinov I, Bojkov D. Advanced Interactive Web Technologies in Industry Training. In Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2002;968-975.

27. Yuan X, Sun H. Mechanical assembly with data glove devices. IEEE Canadian Conference of Electrical and Computer Engineering 1997.