Scholarly article on topic 'Increased Awareness for Maritime Human Factors through e-learning in Crew-centered Design'

Increased Awareness for Maritime Human Factors through e-learning in Crew-centered Design Academic research paper on "Materials engineering"

Share paper
Academic journal
Procedia Manufacturing
OECD Field of science
{"Maritime human factors" / "User-centered design" / "Maritime safety" / "International maritime organization" / E-learning}

Abstract of research paper on Materials engineering, author of scientific article — Gesa Praetorius, Aditi Kataria, Erik Styhr Petersen, Jens-Uwe Schröder-Hinrichs, Michael Baldauf, et al.

Abstract In the past two decades, the need to address human factors in shipping through integration of ergonomics in the design of ships and shipboard equipment has increased significantly as a result of the technological development of modern ships. The International Maritime Organization (IMO), the United Nations’ specialized organization for ship safety issues, has adopted a vision to address human factors as a key element for the improvement of maritime safety, and in that context acknowledges the human element as complex and multi-dimensional. IMO's standards focus on the avoidance of human and organization error. But in spite of this, and despite the availability of qualified guidance on maritime human factors, there is little evidence of what could be seen as a comprehensive regulatory framework for crew-centered design, i.e. a design practice where ships and ships’ equipment is explicitly designed with human operator usability as an integral part of the design process. Recently, a European Commission sponsored project CyClaDes has made an attempt to address this paradox from a number of vantage points: An accident analysis, interviews with mariners (n=23), and short visits on board 5 vessels have been conducted to identify knowledge that provides insights into crew involvement in design, which, in turn, have been used to develop five training packages as one outcome of the project.

Academic research paper on topic "Increased Awareness for Maritime Human Factors through e-learning in Crew-centered Design"


Available online at


Procedia Manufacturing 3 (2015) 2824 - 2831

6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015) and the

Affiliated Conferences, AHFE 2015

Increased awareness for maritime human factors through e-learning in crew-centered design

Gesa Praetoriusa'*, Aditi Katariaa, Erik Styhr Petersena,b, Jens-Uwe Schröder-Hinrichsa,

Michael Baldauf0, Nina Kählerc

aWorldMaritime University, Maritime Risk and System Safety (MaRiSa) Research Group, P.O. Box 500, 201 24 Malmö, Sweden bLyngs0Marine, LyngsoAlle 2, 2970Horsholm, Denmark. cDNVGL SE, Brooktorkai 18, 20457Hamburg, Germany


In the past two decades, the need to address human factors in shipping through integration of ergonomics in the design of ships and shipboard equipment has increased significantly as a result of the technological development of modern ships. The International Maritime Organization (IMO), the United Nations' specialized organizationfor ship safety issues, has adopted a vision to address human factors as a key element for the improvement of maritime safety, and in that context acknowledges the human element as complex and multi-dimensional. IMO's standards focus on the avoidance of human and organization error. But in spite of this, and despite the availability of qualified guidance on maritime human factors, there is little evidence of what could be seen as a comprehensive regulatory framework for crew-centered design, i.e. a design practice where ships and ships' equipment is explicitly designed with human operator usability as an integral part of the design process. Recently, a European Commission sponsored project CyClaDes has made an attempt to address this paradox from a number of vantage points: An accident analysis, interviews with mariners (n=23), and short visits on board 5 vessels have been conducted to identify knowledge that provides insights into crew involvement in design, which, in turn, have been used to develop five training packages as one outcome of the project.

© 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license


Peer-review underresponsibilityofAHFEConference

Keywords/Maritime human factors; User-centered design; Maritime safety; International maritime organization; E-learning

Corresponding author. Tel.:+46-40 356 374; fax: +46- 40 128 442.

E-mail address:

2351-9789 © 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license


Peer-review under responsibility of AHFE Conference

doi: 10.1016/j.promfg.2015.07.762

1. Introduction

In the past two decades, the need to address the human element in shipping by integrating ergonomics in the design of ships and shipboard equipment has increased significantly, driven by a corresponding increase in the technological level of modern ships. Especially the two main control centers on ships have been influenced by this trend, partly through the evolvement of regulations; the net result being that such jobs are becoming increasingly of a supervisory nature of advanced and complex electronic systems. The International Maritime Organization (IMO), which is the United Nations' specialized organization responsible for ship safety issues, has the vision to 'significantly enhance maritime safety and the quality of the marine environment by addressing human element issues to improve performance', and in that context acknowledges the human element as a 'complex multidimensional issue'. In the development of new regulations, and appreciating that, for instance, 'automation has qualitative consequences for human work and safety and does not simply replace human work with machine work' (MSC/Circ. 1091)[1], the IMO has emphasized, and continues to emphasize, the necessary focus on the avoidance of human and organization error (e.g. in its Resolution A.947 (23) [2]). In spite of this, and despite the readyavailability of qualified guidance on maritime human factors from well-reputed organizations like the major classification societies, at least at the time of writing, there is little evidence of what could be seen as crew-centered design (CCD), i.e. a design practice where the focus is on human capabilities and limitations, and where ships and ships' equipment in appreciation of these design parameters is explicitly designed with human operator usability as an integral part of the design process and mindset. The absence of such a system can partly be explained in the unstructured way IMO was addressing human factors in earlier years [3].

Today's merchant vessels can be considered as complex socio-technical systems. The notion of a socio-technical system emphasizes the importance of recognizing the interrelationships between humans and technology in a social context in which work occurs [4,5]. Advancements in information technology have evolved the tasks of the mariners within the socio-technical system of a merchant vessel from being deckhands, navigators and engineers to operators interacting with complex technology in control centers [6], i.e. bridge and engine control room, to supervise, monitor and track processes occurring on the vessel. Furthermore, due to the decreasing number of crew onboard, more and more tasks have become automated [7] among a number of benefits however also creating an environment which provides the ground for so-called automation surprises [8], where human operators' expectancies do not match the system output. As Perrow [9] has stressed, the maritime transportation domain can be identified as an error-inducing system, as new technology, although originally introduced to increase the overall safety, has continuously been used to increase the system's efficiency, creating new vulnerabilities as the system pushes towards the boundaries of its performance envelope. One early but illustrative example for such a technology-error induction is the Radar. Once introduced into commercial shipping early after the World War II, with the aim to increase safety margins, it turned out also to enable vessels to sail faster through crowded areas. Radar as a safety measure was thus extended to become a productivity measure by the end-users, enabling more and faster traffic movements, but, sadly, also radar-assisted incidents [10], such as that between the MVStockholm and SS Andrea Doria.

In summary, within the maritime domain mariners interact with complex technology, often used to increase the efficiency of operations, as part of their daily work. While technology advances, mariners are pushed farther away from the original production (navigation, loading goods, operating machinery etc.) processes, becoming part of a socio-technical system incorporating human operator and technology in an organizational setting [11-13]. As a consequence new needs have arisen, such as a need for better and more effective familiarization and training, as well as skill development in terms of electronics and IT [14], the need to integrate human operators and technology better to avoid mismatches in the human-machine interface [15], or the need to address human factors and ergonomics in ship design and operation [16].

The CyClaDes project on CCD and operations of ships has aimed to identify challenges in how human operators, technology and organization are integrated in the design of ships and shipboard operations, as well as the project's goal has been to find ways of how to best pass on and apply human element knowledge within the maritime domain. Developing a CCD perspective, the project has set out to accumulate, test and disseminate knowledge on how the crew can best be integrated into design. One important aspect in this context has been identified as learning which is tuned to the various stakeholders. As such, this article presents findings from the CyClaDes-project, that have

influenced the development of an e-learning platform containing five dedicated learning packages for stakeholder groups (end-users - i.e. the seafarers, designers of ships and ship's equipment, shipping companies, organizations involved in the rule-making for the maritime domain, and the appropriate maritime authorities) involved in the design and operations of vessels. The main motivation for the development of the e-learning courses has been to provide end-users with means to express user needs towards designers and shipping companies, to offer designers new perspectives on why and how end-users should get involved in the design of vessels and equipment, to show shipping companies how the design effects the end-users and how insights from human factors can improve design, and to propose guidance to rule-makers and authorities on how to promote the integration of human element-related knowledge in the design of merchant vesselsand its assessment.

2. Background

The following section introduces the concepts underlying for this research. The first section will introduce the notion of usability and human-centered design, while the second section provides a brief introduction into e-learning.

2.1. Usability

The concept of usability [17,18] attempts to describe the universe of a particular product, and its three product qualities,which jointly define a 'good' - or usable - product over one which is not. Usability is formally defined as the 'extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use' [17], from which a number of salient points should be noted: products are to be designed for specific users; they are to be designed for specific objectives andfor use in specific places and under specific conditions. Further, products should be 'effective', meaning that they should be complete in the sense of including the required features/tools/support; they ought to be 'efficient', meaning that the energy (physical, mental or both) required to undertake a particular task should be as low as practically possible, and they should be 'satisfactory', meaning that products should be subjectively appealing to the users.

As outlined above, the usability of present-day maritime technology leaves something to be desired. For this purpose, the CyClaDes-project set out to define ways and means to support an improvement, building on the notion that 'User Centered Design' (UCD) [19] - or human-centered design (HCD) [18] - in this particular instance, crew-centered design (CCD), is reputedly leading to products which exhibit good usability [20,21].

Within the engineering disciplines, and in particular within software development, an early notion was that a product progressed - uni-directionally - through a number of stages, very roughly starting with the creation of a complete, unambiguous 'Requirements Specification', describing all properties of the artefact to be developed, via high-level and low-level design, to coding and a number of technical test stages, eventually producing a product which could be introduced into the market. Being named the 'Waterfall-model', this process is even today favored in many organizations [22], while others have applied the'V-model'[23] and the 'Spiral-model'[24], increasingly recognizing that the notion of having a complete set of invariable requirements from the outset of a design is flawed. As a product develops, so do the requirements, experientially: everybody learns though the development process, and this learning serves to necessitate adjustments to the original Requirements Specification.

User-centered design (UCD) is built on this notion meaning that end-users are to be included in the entire design process right from the beginning [18] as good usability is not something which can be 'glued' onto a product at a late stage of the design process[25]. Moreover, UCD literature [18,19,21] recognizes that more than design engineering skills are needed when a product is defined or refined: Without precluding any stakeholder, the most important contributions, in terms of skills, are likely to come from the end-users and human factors expertise. The former can inform the design process about the way users are working, thinking and intends to utilize the upcoming design - i.e. end-users are living representations of the context-of-use, and provide that grounding of a design. Complementary, the human factors experts can help design engineering in the all-important 'translation' of user needs to design solutions helping create design solutions which are likely to work for 'a person of average (or even below average) ability and experience'[20].

An additional salient point of UCD is that usability testing of a product is included in the design iterations. Starting as early as possible, design representations and crude computer mock-ups can, and should, be discussed with the end-users. However, one of the most challenging tasks in this respect is sufficient realistic testing including a variety of usual operational conditions and having questions that initiate dialogues rather than yes or no answers. As the design evolves, testing with users should evolve with it, in terms of comprehensiveness and complexity, to shape the design, and to provide insurance that the product meets the needs of users.

In summary, HCD is a question of understanding the user needs in the widest possible context, and to iteratively design and test a product which matches the needs of these users - accepting, and as a consequence, involving, any and all expertise required to make the vision of a useful product come true.

2.2. E-learning

E-learning is defined as "instruction delivered on a digital device such as a computer or mobile device that is intended to support learning "[26]. It has evolved from computer-based training [26,27]. The instruction can either be synchronous, i.e. recorded lectures or virtual classrooms, or asynchronous, i.e. learning material designed for individual self-study. Both forms can either be focused on an individual learner, or they can incorporate features creating an environment to foster collaboration among the participants in a course. E-learning is considered one of the most cost-effective and flexible ways of teaching and training as it can accommodate trainees from a wide spread geographical area and across time-zones. People can take part in courses without the cost of travelling and costs for face-to-face teaching at the same time as many people can be reached at once [27]. Further, due to the development of IT-technology, access to e-learning often only depends on access to an internet connection or a computer. Other benefits associated with e-learning include the ability to custom-tailor the material to the individual learner, to be able to easier engage people in learning activities through using a mixture of multi-media and teaching techniques, and the ability of the individual learner to take in information at his/her own speed.

E-learning has undergone a large development with up to around 40% of training for company employees being delivered through the computer [26]. The increase in online courses, such as massive open online courses (MOOCs), or Moodle-based platforms, also indicates that this form of instruction is getting more and more popular in academia as well, where major universities provide tuition-free courses for thousands of students at once. An example by Vardi [28] is Stanford University where 430,000 students signed up for three courses in computer science.

With the CyClaDes-project addressing a large variety of different stakeholders, it was decided, that e-learning could be one means of reaching out to the community of naval architects, shipping companies, authorities, but foremost also end-users (mariners) as material being developed as asynchronous training package could accommodate seafarers to participate and learn more about how their knowledge could be integrated into the design of vessels, onboard equipment and procedures.

3. Methodology

This section briefly describes the research methodology that has driven the data collection. First, an accident analysis has been carried out to identify critical areas where human-machine interface challenges have occurred. It was followed by a series of semi-structures interviews [29] and complemented by field visits conducted to gain insights into mariners' design involvement of their own work environment.

3.1. Technique for the Retrospective and predictive Analysis of Cognitive Error (TRACEr)

The TRACEr technique was developed by Shorrock and Kirwan [30] asa means to gain a deeper understanding of operator errors and their underlying causes within the air traffic management domain. TRACEr provides a modular structure comprising eight inter-related taxonomies. These taxonomies help the analyst to code events, i.e. based on accidents investigation reports and identify the context, the error that occurred, and the recovery (Table 1).

Table 1. The TRACEr taxonomies.

TRACEr 1st Level TRACEr 2nd Level

l.Task Error

Context of the incident 2. Error Information

3. Casualty Level

4. External Error Mode (EEM)

5. Cognitive Domain

Operator Context 6.Internal Error Mode (IEM)

7. Psychological Error Mechanism (PEM)

8. Performance Shaping Factors (PSF)

Error Recovery 9. Error Recovery

In the context of the CyClaDes-project, 129 accident investigation reports publicly available have been coded and analyzed with help of TRACEr to identify whether the human-machine interface- and if so in what way- has contributed to the adverse event. Another purpose has been to identify the location on the merchant vessel and the personnel involved in the accident. In order to carry out the TRACEr analysis, a taxonomy adapted to the maritime domain had to be developed.

3.2. Interviews

Semi-structured interviews with 23 individuals with seafaring experience were conducted. The majority of the respondents were from the navigation department (n=18), while 4 interviewees have experience from working in the engine department and one is a deck cadet. The interviews followed an interview guide focused on the design of workspaces, equipment and operations on board and took about 45 minutes each.

3.3. Field visits

Field visits onboard 5 vessels (water taxi, harbor tour vessel, ferry, container, ice-breaker) were conducted with the aim to provide an overview on when and how, if at all, the crew is involved in the design of ships and equipment. The visits also served as input to identify examples for good and bad design onboard. Each study visit started with a short introduction about the project, and a short semi-structured interview with questions concerning the crew's involvement in design as well as on how the crew compensates for design not fitting to their work tasks. Then the crew was asked to show two researchers around the vessel and point out examples within their work settings, which represented good/bad design, and share strategies on how design of workspaces and procedures is tailored to the context of work.

4. Results

4.1. Technique for the Retrospective and predictive Analysis of Cognitive Error (TRACEr)

The results of the TRACEr analysis show that 67% of the accidents involved the human-machine interface. The majority of accidents could be identified as personal accidents (44%) followed by collisions (29%) and groundings (15%). Most errors could be associated to operations on the bridge (50%) followed by deck (38%) and engine room (12%). The captain was identified in 19% of the errors in the reported accidents, followed by the chief officer (14%), able bodied seaman (AB) (9%) and pilot (8%). The identified tasks which led to the task errors were navigation (23%), followed by traffic monitoring (18%), cargo work (18%), maintenance work (15%) and mooring operations (4%). The equipment involved in the task errors were identified as radar (n=64), loading devices (n=13), mooring equipment (n= 11), stairs, ladders (n=11), steering panel (n=11), engine room controls (n=10), Very High

Frequency (VHF) radio (10) etc. The accident analysis highlights that the interactions between workspaces, operations, personnel and equipment are crucial for safe operations and should be considered when designing a ships and shipborne equipment or operations.

4.2. Interviews

Results from the interviews show that there are several pieces of equipment, such as mooring winches, purifiers, lifesaving equipment, hatch covers, and ballast water pumps, where CCD could lead to significant safety improvement. Interviewees recalled examples of shipboard equipment and spaces which were unfit for purpose and had poor usability in the operational context. Seafarers devised local solutions to compensate for poor design to carry out the required work, for e.g., fixing a long spanner on to an inaccessible valve and marking it for aiding subsequent maintenance work. Disenchantment with shipping companies and regulatory bodies was also voiced by the mariners who felt that more support could be given to the seafarers to facilitate their work. Injuries apart, poor design has, at times had fatal consequences for the crew working onboard. An interviewee recalled the death of the chief engineer who was decapitated by a parting rope during mooring operations. The chief engineer stood exposed near the additional mooring controls that had been located far up-front to overlook the gun valve during mooring operations. Another interviewee recalled that there had been as many as three fatalities in his company during maintenance operations of the purifier. Three engineers at different times had overlooked to put the locking knot in place prior to starting the purifier. The equipment started without the locking knot in place and nearly two hundred blades came flying at the individuals at ten thousand rotations per minute and led to their deaths.

4.3. Field visits

In the short interviews the mariners in the beginning of the field visits highlighted that there is a difference between special purpose vessels, such as an offshore-support vessel where crew and company actively get involved, and "off-the-shelf'-orders which are considered standard vessels and where involvement in the physical design of workspaces is limited. The crews further emphasized that there are possibilities to contact management and that there is a general attitude to be open to suggestions from the crew in terms of making amendments to current design, e.g. installing a rack within the engine-room to mount barrels containing dangerous liquids to keep these from the operational spaces of the crew, and adoption of operational procedures that can support the crew and often show a positive impact on the overall operation of the vessel. This is exemplified by the quote below

"They have a lot of good ideas and are happy when you ask" (Master)

However, mostly involved in developments related to design and operating procedure are mariners in higher ranks, such as masters, senior officers and senior engineers. In addition, it is mostly the master and the engine room chief that seem to present and communicate the ideas of other crew to the company representatives.

Furthermore, during the study visits it was repeatedly stressed that the increase in legal framework surrounding maritime operations has also increased the overall workload of senior staff, making it hard to get the work done in the allocated time.

"The rules are coming down on us [STCW], but because of the regulations there is more work. We cannot comply" (Crewmember)

5. Discussion

The results presented in the previous section make a strong case for why to integrate mariners in the design of operations and workspaces in the maritime domain. The human-machine interface, for example, has been involved in 67 % of the cases, as well as personal injuries have been identified as the prominent type of accident within the occurrences analyzed. Tasks performed while the accidents occurred primarily showed to be connected to navigation, traffic monitoring and cargo work. All of these are tasks involving advanced technical equipment, such as mooring stations, Radar or Electronic Chart and Information Systems (ECDIS) implying that the match between operators, equipment and procedure was not ideal, and therefore highlighting an area where the end-user involvement in design could potentially lead to significant improvements. Socio-technical systems are always

subject to migration and change, as they are exposed to a political and economic environment [31]. While this is normal and necessary for systems to be able to adapt to a changing environment [32], it is important that stakeholders within the domain understand the consequences and implications of the changes undertaken, which can only be achieved through continuous process incorporating designers, end-users, companies and authorities at all stages of design. Socio-technical systems operate in high hazardous environments and the way that everyday work is understood, is of the essence to make sure that the operation can be maintained in a safe and efficient manner. Endusers are the only ones that can provide essential input that can help and is needed to understand everyday work onboard and the specific needs that can arise from this type of work environment. As the results of the interviews and field visits indicate there are severe risks onboard which might go unnoticed if crew does not get involved in the design of their workspaces in early stages. In addition, if crew was involved by the management in the design of a new-building, it was often limited to senior staff, leaving the rest of the crew in a vulnerable position, as reporting about design amendments and changes in procedures has to pass the barrier of a senior staff member and cannot be directly issued. It is therefore recommended, that one method within CCD should comprise visiting the crew onboard. Only with an inside view of the work environment and the amendments made to it by the crew, i.e. task and system tailoring [33], flaws indicating poor usability in current systems can be identified to be able to address these in future designs.

E-learning courses offers a large flexibility to disseminate knowledge to potential users as learners can access and work with them in their own pace[26], offering an attractive solution for the maritime domain, where training needs to either be conducted on board, or where mariners need to take time off to be able to attend a training course. Therefore, five dedicated training packages have been developed within the context of the CyClaDes- project to promote, disseminate and emphasize the need to integrate human element knowledge within the maritime domain. The training has been developed as e-learning packages in a moodle ( platform. Each of the learning packages addresses a specific need identified in terms of knowledge relevant for one or more user group, and material created has aimed at supporting asynchronous [26] learning. However, the learning packages have just been launched and it remains to be seen whether they achieve the desired results of supporting each party involved in design (designers, end-users, shipping companies, rule-makers) and raising awareness for the need to integrate human element knowledge within the design of operation, equipment and workspaces onboard. Feedback from the testing activity will be included in the next iteration of the material included in the training to increase the potential effect of such training within the maritime domain.

6. Concluding remarks

Crew-centered design (CCD), as outlined above, is an approach to integrate the crew into the design of vessels, equipment and organization of work within the maritime domain. It entails the integration of end-user feedback in all stages of the design process and highlights usability as the main goal. Findings from an accident analysis, interviews and field visits have shown that mariners offer a tremendous source of information about challenges in the interaction between human operators and the designed work environment. Although the results indicate that there is a general acknowledgement of mariners' opinion, there is often only senior officer involvement, creating a barrier for lower ranked seamen to provide feedback. The training packages developed by the CyClaDes-project are a first step to disseminate knowledge about CCD and its benefits to different stakeholder groups with the hope to create an environment, in which end-users become involved in all stages of the design to increase the chances of achieving good match between mariner, equipment and workspaces, and therefore creating safe operations within the maritime domain.


The research presented has been financed by the CyClaDes-project, grant agreement number 313972, part of the research financed through the 7th Framework Program initiated by the European Commission. Opinions expressed in this work reflect the authors' views only and the CyClaDes-project shall not be considered liable for them or for any use that may be made of the information contained herein. Further, the authors would like to acknowledge the

support of LT Jessica Flennoy, LCDR Jarrod DeWitz, and the US Coast Guard, Sector Puget Sound, Inspections Division for facilitating field visits.


[1] IMO. Issues to be considered when introducing new technology on board ship. IMO document MSC/Circ. 1091 dated 06 June 2003.London:

IMO; 2003.

[2] IMO. Human Element Vision, Principles and Goals for the Organization. IMO document Resolution A.947(23) adopted 27 November 2003.

London: IMO; 2004.

[3] Schröder-Hinrichs J-U, Hollnagel E, Baldauf M, Hofmann S, Kataria A. Maritime human factors and IMO policy. Maritime Policy &

Management. 2013;40:243-60.

[4] Hollnagel E, Woods DD. Joint Cognitive Systems. Foundations of Cognitive Systems Engineering. Boca Raton, FL: CRC Press; 2005.

[5] Vicente KJ. The Human Factor. New York: Taylor & Francis Group; 2006.

[6] Lützhöft M, Lundh M. Maritime Application of Control Systems. In: Ivergard T, Hunt B, editors. Handbook of Control Room Design and

Ergonomics - A Perspective for the Future. Boca Raton, FL: CRC Press; 2009.

[7] Hetherington C, Flin R, Mearns K. Safety in shipping: The human element. Journal of Safety Research. 2006;37:401-11.

[8] Bainbridge L. Ironies of Automation. Automatica. 1983;19:775-9.

[9] Perrow C. Normal Accidents. Living with High-Risk Technologies. New York: Basic Books, Inc., Publishers; 1984.

[10] Schröder-Hinrichs J-U, Hollnagel E, Baldauf M. From Titanic to Costa Concordia— A century of lessons not learned. WMU Journal of Maritime Affairs. 2012;11:151-67.

[11] Baxter G, Sommerville I. Socio-technical systems: From design methods to systems engineering. Interacting with Computers. 2011;23:4-17.

[12] Militello LG, Dominguez CO, Lintern G, Klein G. The role of cognitive systems engineering in the systems engineering design process. Systems Engineering. 2010;13:261-73.

[13] Praetorius G, Lützhöft M. Safety Is Everywhere-The Constituents of Maritime Safety. Human Factors and Ergonomics Society 55th Annual Meeting. Las Vegas: Human Factors and Ergonomics Society; 2011. p. 1798 - 802.

[14] Ljung M, Lützhöft M. Functions, performances and perceptions of work on ships. WMU Journal of Maritime Affairs. 2014;13:231-50.

[15] Lützhöft M, Dekker SWA. On Your Watch: Automation on the Bridge. Journal of Navigation. 2002;55:83-96.

[16] Schröder-Hinrichs JU. Human and Organizational Factors in the Maritime World-Are We Keeping up to Speed. WMU Journal of Maritime Affairs. 2010;9:1-3.

[17] International Organization for Standardization. ISO 9241-11 Ergonomic requirements for office work

with visual display terminals (VDTs) Guidance on usability: International Organization for Standardization; 1998.

[18] International Organization for Standardization. Ergonomics of human-system interaction --Part 210: Human-centred design for interactive systems. International Organization for Standardization,; 2008.

[19] Nielsen J. Usability Engineering: Elsevier Science; 1994.

[20] Krug S. Don't Make Me Think: A Common Sense Approach to Web Usability: Pearson Education; 2009.

[21] Maguire M. Methods to support human-centred design. Int J Human-Computer Studies. 2001:587-634.

[22] Laplante PA, Neill CJ. The Demise of the Waterfall Model Is Imminent. Queue. 2004;1:10-5.

[23] IABG Information Technology. V-Model. Lifecycle Process Model. 1992.

[24] Boehm BW. A Spiral Model of Software Development and Enhancement. Computer. 1988;21:61-72.

[25] Gulliksen J, Boivie I, Göransson B. Usability professionals—current practices and future development. Interacting with Computers. 2006;18:568-600.

[26] Clark RC, Mayer RE. e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning: Wiley; 2011.

[27] Clark RC, Kwinn A. The New Virtual Classroom: Evidence-based Guidelines for Synchronous e-Learning: Wiley; 2007.

[28] Vardi MY. Will MOOCs destroy academia? Commun ACM. 2012;55:5.

[29] Patton MQ. Qualitative research & evaluation methods: Sage Publication; 2002.

[30] Shorrock ST, Kirwan B. Development and application of a human error identification tool for air traffic control. Applied Ergonomics. 2002;33:319-36.

[31] Rasmussen J. Risk management in a dynamic society: a modelling problem. Safety Science. 1997;27:183-213.

[32] Hollnagel E. The ETTO Principle: Efficiency-Thoroughness Trade-Off. Why Things That Go Right Sometimes Go Wrong. Burlington: Ashgate Publishing Company; 2009.

[33] Norman DA. Things that make us smart. New York: Addison-Wesley; 1993.