Scholarly article on topic 'Methodology for the Evaluation of Forecast Reliability of Production Planning Systems'

Methodology for the Evaluation of Forecast Reliability of Production Planning Systems Academic research paper on "Computer and information sciences"

CC BY-NC-ND
0
0
Share paper
Academic journal
Procedia CIRP
OECD Field of science
Keywords
{"production planning" / "production control" / changeability / re-configuration}

Abstract of research paper on Computer and information sciences, author of scientific article — Günther Schuh, Till Potente, Annika Hauptvogel

Abstract The high dynamics of markets are only one reason for the increasing complexity of production planning and control. To handle this complexity manufacturing companies have implemented IT systems to support decision-making in detailed scheduling processes. However, applied IT systems often do not provide a reliable forecast of delivery dates, because the planning models are implemented uniquely and have never been adapted due to changes in the production system. This paper presents an approach to verify the forecast reliability of detailed planning systems by identifying deviations between the predicted production schedule, determined by the IT system, and the observed production processes in reality. The paper introduces the reasons for deviations and explains how they can be determined. The approach represents how categorical and continuous verification methods can be applied to identify the described deviations. Depending on the determined deviations the forecast quality index of detailed planning systems is developed. Besides the assessment of the forecast quality the reasons for deviations are of interest to production planners. Identified reasons are the starting point for adaptions in planning models to enable a reliable forecast of re-configurable production planning systems in the future.

Academic research paper on topic "Methodology for the Evaluation of Forecast Reliability of Production Planning Systems"

Available online at www.sciencedirect.com

ScienceDirect

Procedía CIRP 17 (2014) 469 - 474

Variety Management in Manufacturing. Proceedings of the 47th CIRP Conference on Manufacturing

Systems

Methodology for the evaluation of forecast reliability of production

planning systems

Günther Schuh, Till Potente, Annika Hauptvogel *

Laboratory for Machine Tools and Production Engineering (WZL) at Aachen University, Steinbachstraße 19, Aachen 52074, Germany * Corresponding author. Tel.: +49-241-80-28390; fax: +49-241-80-22293. E-mail address: a.hauptvogel@wzl.rwth-aachen.de

Abstract

The high dynamics of markets are only one reason for the increasing complexity of production planning and control. To handle this complexity manufacturing companies have implemented IT systems to support decision-making in detailed scheduling processes. However, applied IT systems often do not provide a reliable forecast of delivery dates, because the planning models are implemented uniquely and have never been adapted due to changes in the production system. This paper presents an approach to verify the forecast reliability of detailed planning systems by identifying deviations between the predicted production schedule, determined by the IT system, and the observed production processes in reality. The paper introduces the reasons for deviations and explains how they can be determined. The approach represents how categorical and continuous verification methods can be applied to identify the described deviations. Depending on the determined deviations the forecast quality index of detailed planning systems is developed. Besides the assessment of the forecast quality the reasons for deviations are of interest to production planners. Identified reasons are the starting point for adaptions in planning models to enable a reliable forecast of re-configurable production planning systems in the future.

© 2014Elsevier B.V. Thisisanopenaccessarticle under the CC BY-NC-ND license (http://creativecommons.Org/licenses/by-nc-nd/3.0/).

Selectionandpeer-review under responsibilityofthe International Scientific Committee of "The 47th CIRP Conference on Manufacturing Systems" in the person of the Conference Chair Professor Hoda ElMaraghy"

Keywords: production planning; production control; changeability; re-configuration

1. Introduction

Due to the increasing need for high quality customized products at competitive prices dynamics in manufacturing processes have risen further [1]. To cope with this challenge a capable production planning and control (PPC) is essential [2]. Therefore, IT systems are indispensable. IT systems supporting the user in production planning such as Advanced Planning and Scheduling Systems (APS Systems) are called production planning systems. Their functional model and data model should be changeable according to the high dynamics in manufacturing processes [3]. However, there is a gap between the implemented models and the reality on the shop floor as analyses in manufacturing companies confirm. The similarity between the planning determined by the planning system and the real situation on the shop floor decreases to 25 % after just 3 days, as analysis in companies

with individual and small series production show [4]. Because a high adherence to delivery dates is the main logistic target [5], every manufacturing company is interested in the reliability of the forecasted production plan to use it as a basis for secured delivery date confirmations. Following lacks of current planning systems can be summarized:

• Insufficient image of the actual situation in terms of feedback

• Rigid structures and a lack of adaptability of planning systems

• No continuous adjustment of master and transaction data as well as models to the real environment

• No regard of deviations between the model implemented in the planning system and the real situation of the manufacturing processes

These problems result in unreliable predictions about the future situation in the manufacturing. Hence, an accurate

2212-8271 © 2014 Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/).

Selection and peer-review under responsibility of the International Scientific Committee of "The 47th CIRP Conference on Manufacturing Systems"

in the person of the Conference Chair Professor Hoda ElMaraghy"

doi:10.1016/j.procir.2014.01.069

image of the manufacturing will be even more important in the future.

Within the publicly funded research project "ProSense" a high resolution production management based on cybernetic assistance systems and smart sensors is developed [6]. Using intelligent sensors more data is generated to get a more detailed image of the situation of the manufacturing processes. Based on second order cybernetic control problems in the planning and manufacturing processes are identified to define measures in order to prevent these problems in the future. This paper focuses on the determination and adaption of deviations in the plans determined by detailed scheduling system to ensure a reliable manufacturing planning.

2. Requirements

Today companies rely on detailed planning systems to manage the complexity of the production and to support the PPC. Requirements for such IT systems include a reliable statement on the completion date of all orders and thus the reliable attainment of the customer agreed date. The presented lacks of current planning systems, which are applied in manufacturing, results in several requirements for planning systems:

• Continuous adaption of the data used for planning

• Continuous adjustment of the models used for planning

• Always representing the real situation on the shop floor

• Determination of deviations between the planning and reality

• Determination of the reasons for the deviation from the plan generated by the applied planning system

• Determination of the forecast reliability

In the following current approaches are described in the state of the art considering the mentioned requirements. In the second part of the paper an approach to identify deviations between planned and real manufacturing processes is described. Afterwards a forecast quality index of detailed planning systems is introduced.

3. State of the Art

3.1. Self-Optimization

The demand for continuous adaption is also regarded within the approach of self-optimization. The principle of self-optimization is approved to face the complex environment of manufacturing processes [7]. Self-optimization consists of three actions: Analyzing the current situation, determining the system's objectives and adapting the system's behavior [8]. The current concepts of self-optimization in manufacturing processes can be reduced to three initial approaches: the fractal company [9], the holonic manufacturing system [10] and the bionic manufacturing System [11, 12].

The fractal factory is defined by several fractals which represent independently acting manufacturing units. Fractals can be described as self-similar, self-organizing and self-optimizing. Each fractal follows its own goal, which lies

within a goal system. The fractals are linked through a dynamic and self-regulating network. [9]

A Holonic Manufacturing Systems (HMS) is made up of holons. Holons are autonomous and co-operative building blocks that carry and process information or physical objects arranged within a system of cooperation to achieve a common goal, the so called "holarchy".[13]

Bionic approaches replicate the structure of a living organism. The manufacturing system is arranged hierarchically into several components. Furthermore, the concepts of differentiation and proliferation of cells, the genetic function, evolution and self-organization as well as an enzymatic function and the autonomous distributed system are reflected within a so called life software.[10]

These approaches are transferable to the organizational structure of production systems. Within these approaches decisions are taken based on current data from production. Since the highly decentralized approaches prevent a prediction of their future behavior, they react quickly to current events rather than to plan prospectively.

3.2. Automatic Model Generation

An approach to face the demand for continuous adjustment of the models used for planning and representing the real situation on the shop floor is the application of automatically generated models of the manufacturing system. This so called automatic model generation (AMG) was developed to accelerate the generation process and enhance the accuracy of the simulation models [14]. Four approaches are regarded in the following.

Selke describes a solution to generate models automatically from the analysis of applied strategies and operations regarding scheduling, sequencing and determination of lot sizes in production and logistics [15]. The approach mainly focuses on the development of a strategy analysis [16]. Plan data generated by detailed scheduling systems is not regarded and the approach is not feasible for daily use in production planning and control.

Horn develops a scheduling and sequencing system, which is based on a simulation-based optimization combined with an AMG [17]. The simulation models are automatically generated from mini-templates and parameterized with data from a simulation-repository database. This database is filled by the production-related databases (ERP, MES). The simulation models are automatically generated based on the current data provided by the applied IT systems. An adaptation of the models depending on identified deviations in process times does not take place.

Pfeiffer develops a simulation system using an AMG and focuses on the sensitivity analysis and off-line validation of schedules as well as on a plant-level disturbance handling [18]. The relevant data is extracted from the MES and enriched by information from the ERP system. Beside the heavily customized model generation to fit the enterprises IT structure, Pfeiffer implements several dispatching rules and loading logics [19, 20]. The hereby generated model is used to evaluate new scheduling rules under consideration of the WIP, queue sizes and output. This approach considers plan

data and evaluates it by using a simulation model generated with feedback data. However, the comparison of the simulated and the real events is only carried out regarding the particular throughput. The transfer of the results of the comparison to future planning is not explained.

Kapp develops an object-oriented AMG, which provides a real-time factory model. It is based on a reference model and an adaptable component-library [21, 22]. The model generator is suitable for short, middle- and long-term problems. The user can influence the model generation in three different layers, whereas the first layer is the one used to create simulation scenarios. The second and third layers are needed to adapt the model to the enterprise's structure. However, the approach still lacks a validation of the input data and has to be adapted to scheduling problems with a time horizon less than a week. This approach takes the dynamic behavior of the manufacturing into consideration by constantly updating the real scenario. Though, any adjustment of the reference model must be implemented by a user with programming skills.

Within the presented approaches for AMG the generated models are compared to real manufacturing processes in order to detect deviations. However, this is only done based on specific indicators and mainly through a visual alignment. Resulting consequences of the deviations for future generations of models are not considered by any of the approaches.

3.3. Cyber-Physical Systems

To handle the challenges described in the beginning a promising approach are so called cyber-physical systems (CPS). CPS are physical and engineered systems integrating computational and physical capabilities [23]. These embedded systems are able to interact with each other and also with humans [24]. CPS "control the physical processes, usually with feedback loops where physical processes affect computations and vice versa" [25]. Using appropriate sensor technology, CPS are able to directly receive physical data as well as use all the available data by connecting through digital networks. Such embedded systems are characterized by a higher reliability and predictability standard than generalpurpose computing [25]. CPS offer new opportunities for data acquisition and processing, which enables to get an accurate image of the real manufacturing processes. As the historical data is stored, the history of the images of the manufacturing processes is also available at any time. Through the networking of IT systems manufacturing information are available more quickly.

4. Deviations between planned and real manufacturing

processes

The described approaches do not meet all requirements stated in the beginning, the comparison of the planning and the reality is not conducted. Therefore, the determination of reasons for the deviation from the plan generated by the applied IT system, the determination of deviations between the planning and reality and the determination of the planning quality are regarded more detailed in the following.

4.1. Reasons for deviations

In this paper, a deviation is defined as the difference between a forecasted parameter and its true value. The planning is carried out by detailed scheduling systems and the true value can be extracted from the feedback data of the manufacturing. A comparison of both determines the deviation (see Fig. 1).

Production order Process i Workstation Plan start of process Plan endof process Order quantity

4711 20 ! 139 21.11.2013 10:24 21.11.2013 12:24 20

4712 30 ¡ 145 21.11.2013 10:27 21.11.2013 12:33 10

4715 20 ! 139 21.11.2013 12:42 21.11.2013 13:50 30

Manufacturing

Production order Process i Workstation Actual start of process Actual end of process Order quantity

4711 20 ¡ 139 21.11.2013 10:24 21.11.20C 13:13 20

4712 30 ¡ 155 21.11.2013 10:27 21.11.2013 14:37 10

4715 20 ! 139 22.11.20 3 09:38 22.11.2013 12:46 30

Fig. 1 Comparison of planning and reality.

Manufacturing processes can be influenced by both exogenous and endogenous effects as missing material or disturbed tools and machines. These effects might potentially cause deviations between planning and reality. These deviations are named in the following (see Fig. 3):

• Workstation of a process:

A process step is executed at another workstation than given in the work plan.

• Date of a process (start or end dates):

A process step starts earlier or later than the date given in the planning.

• Duration of a process:

A process step takes longer or shorter than the process time given in the planning.

• Sequence of the processed orders:

The orders of a machine are produced in a difference sequence than given in the planning.

• Output of a process:

Reduced or increased output of a process compared to the planning.

Deviation Workstation Start Duration Sequence Output

Forecast Process No2 Í 00 o0

1 | |

Reality Process No2 1 1 □O □

Fig. 2 Deviations in manufacturing processes.

These deviations occur due to disturbances, which are listed in Table 1 [26]. A different workstation for example can be the result of a machine substitution.

Table 1. Reasons for deviations.

No disturbance Exemplary reasons for disturbance

1 Machine breakdown Technical defect

2 Employee absence Personal illness

3 Process change or re-routing Product deficits

4 Machine substitution Capacity utilization

5 Rush orders Remaining time until scheduled date

6 Over- or underestimation of Estimated value

process time

7 Change of sequence Set-up time optimization

8 Shortage of materials Transportation problems

9 Due date changes Cause lies with the customer

(delay or advance)

10 Quantity change Cause lies with the customer

11 Rework Quality problems

12 Reject Quality problems

As this paper focuses on PPC, quality problems and deviations caused by changing customer requirements are not further considered, because they cannot be restricted by the PPC.

4.2. Requirements for the analysis

The most important step to enable the determination of deviations is the storage of plan data. Usually, the current planning is overwritten with the updated plan after each planning process of the detailed scheduling system. Thus, the original plan is not subject to disposal. Within the research project ProSense the storage of daily schedules is enabled. Hence, the data history of all planning processes is available. The frequency of storage is defined by the frequency of the planning system, which is normally daily.

To control dynamic systems, such as manufacturing systems, a control loop is needed. To ensure the production of the desired goods at the right date, certain control mechanisms are required, which is shown in Fig. 3. Different IT systems (ERP system and MES) are part of the control loop and use data, which is generated in the production, to handle the complex manufacturing processes. In order to compare the planning and the reality, these feedback data must be stored. A comparison of daily schedules and the corresponding feedback data only makes sense over a certain period. The consideration of only one day is too short whereas a month is too long, because of the high dynamics in manufacturing processes.

The plan includes the planned start, the planned duration (difference between planned end and planned start) and the planned amount for each operation of all scheduled jobs. The feedback data also contain this information, which enables a comparison based on the order number and the operation. This is indicated by the dotted line in Fig. 1, which marks the necessary basic data for the comparison.

Fig. 3 Control loop for the evaluation of forecast reliability of production planning systems, following [27].

4.3. Determination of deviations

In the following, it is introduced, how the four relevant deviations workstation, date and duration of a process as well as the sequence of the processed orders can be determined using an application example. In Table 2 and Table 3 a production schedule and its according feedback data is presented.

Table 2. Production schedule.

No Production order Process Workstation Plan start of process Plan end of process Order quantity

1 4711 20 139 21.11.2013 10:24 21.11.2013 12:24 20

2 4712 30 145 21.11.2013 10:27 21.11.2013 12:33 10

3 4715 20 139 21.11.2013 12:42 21.11.2013 13:50 30

4 4716 40 139 21.11.2013 14:32 21.11.2013 15:16 10

Table 3. Feedback data.

No Production order Process Workstation Start of process End of process Order quantity

1 4711 20 139 21.11.2013 10:24 21.11.2013 13:13 20

2 4712 30 155 21.11.2013 10:27 21.11.2013 14:37 10

3 4715 20 139 22.11.2013 09:38 22.11.2013 10:46 30

4 4716 40 139 21.11.2013 13:30 21.11.2013 14:14 10

Workstation of a process

The comparison of the information in Table 2 and Table 3 makes clear that process 30 of order 4712 was carried out on a different workstation. As described before possible reasons for that are machine substitutions or the planned machine 145 was not available e.g. due to machine breakdown. If it turns out that for specific products the machine 155 is often used instead of machine 145, the process planning should be adapted. If analyses show that machine 145 often fails, appropriate measures should be taken. The higher duration results from the higher processing time of machine 155.

Date of a process

Deviations in the scheduled start dates are determined by the difference of the real start and the scheduled start of a

process. A comparison of Table 2 and Table 3 clarifies that process 40 of order 4176 was started delayed (difference of 1256 minutes) and process 20 of order 4715 was started too early (difference of -62 minutes). A positive result expresses a delay and a negative result an early arrival. An explanation for the earlier processing of order 4715 can be found by taking the list of prioritized orders into account. If order 4715 is a rush order, its prioritization could be explained.

Duration of a process

The duration of a process is measured by the difference of the end and start of a particular process. As described the process 30 of order 4712 has taken longer than forecasted because of the processing on a different workstation. Process 20 of order 4711 has also taken longer although it was carried out on the planned workstation. A possible reason for this is an underestimation of process time. If analyses show that for this product the process step on machine 139 often takes longer, the corresponding process time for this step should be adapted in the process planning.

Sequence of the processed orders

With the data of Table 2 and Table 3 the sequence of workstation 139 can be observed. The sequence forecasted by the schedule is order 4711, order 4715 and finally order 4716. However, Table 3 shows that the executed sequence was order 4711, order 4716 and finally order 4715. Hence, the sequence of the last two orders has changed. The change of the sequence can be explained by the allocation of priorities. Another reason could be a missing transport of the material of order 4715, so that the processing could not start and the worker decided to choose order 4716 instead. New technologies such as the RFID technology support the tracking of products and work pieces. Thus, transportation problems can be identified and improved in the future.

5. Evaluation of the planning

Having explained how deviations can be determined and which specific adaptions can prevent deviations in future planning, the next section introduces how the planning of IT systems can be evaluated. Forecasts are given in many fields such as weather reports and banking. The evaluation of these forecasts in terms of their truth content is a fundamental concern for all users. It is important to know the typical deviations of the forecasts in order to eliminate systematic errors in future planning. For this purpose an absolute consideration does not prove to be useful. A more accurate alternative is offered by categorical and continuous parameters. Categorical variables describe the occurrence or non-occurrence of an event, such as order is in the queue of the machine yes or no. Continuous variables have a specific value, such as deviation of the duration of 4 minutes. All continuous variables can also be categorized by defining certain thresholds. [28]

In the following, a categorical and a continuous variable are presented exemplarily. The Probability of detection POD as categorical variable measures the forecast quality of

production schedules (see equation 1). Therefore, every order position is regarded in the plan and the feedback data for an interval of one week. A is the number of orders which were observed and predicted at the same workstation at the same date. C is the number of orders which were observed but not predicted.

P0D=aTC ^

Distance functions like the Euclidean distance d (see equation 2) are continues variables, which quantify the dissimilarity of pairs of objects, e.g. the deviation of the scheduled start x and the real start y of every process of each workstation. Table 4 presents exemplarily the results for 3 workstations of a company with individual and small series production.

d(x,y) =

Yp - *)2

Table 4. Euclidean distance d of different workstations

Workstation Euclidean distance d

139 5,39

155 2,00

145 107,26

With these parameters critical machine can be identified. In the given example e.g. workstation 145 is critical because of its high distance d between the planning and the reality. The idea of cyber-physical systems is to consistently collected and link different data (e.g. ODA and MDA) to aggregated information with a higher information content (e.g. quality of planning) to support the production planners decision-making process. Due to the high amount of data, the linking cannot be done by the planner but must be executed automatically by corresponding IT systems. The presented approach makes clear, how the problematic machine is identified by the Euclidean distance from the mass of all machines. With the consideration of other data sources, e.g. list of rush orders and fault list, the reasons for the deviations of this machine can be detailed. With this information the production planner is able to take appropriate measures to enable a reliable production schedule in the future.

6. Conclusion and Outlook

This paper introduces a methodology for the evaluation of forecast reliability of production planning systems. At first the term deviation is defined and reasons for deviations between production schedules and the reality happening on the shop floor are given. Furthermore, a method to determine four relevant deviations is presented. In the last part of the paper methods to evaluate the reliability of production planning systems are introduced.

Although, the presented approach shows how the reliability of production schedules can be improved, it also contains several risks and limitations: the feedback date serving as a basis for the analysis often contains inconsistencies. Besides, further research is necessary in several directions: The

described methodology for the evaluation of forecast reliability of production planning systems has been applied to one company. A further verification of the methodology should be made by an application in other companies. Additionally, it must be analyzed which categorical and continuous parameters are most promising in terms of forecast evaluation. In addition, the measures, which can be derived from the identified deviations, should be developed further. Nevertheless, the methodology seems to be a contribution to further improve the production planning.

Acknowledgements

The presented approach for evaluating the forecast reliability of production planning systems was developed within the research project "ProSense", which is funded by the German Federal Ministry of Education and Research (BMBF) within the Framework Concept "Research for Tomorrow's Production" (funding number 02PJ2490) and managed by the Project Management Agency Karlsruhe (PTKA).

References

[1] ElMaraghy H, AlGeddawy T, Azab A, ElMaraghy W. Change in Manufacturing - Research and Industrial Challenges. In: ElMaraghy HA, editor. Enabling manufacturing competitiveness and economic sustainability: Proceedings of the 4th International Conference on Changeable, Agile, Reconfigurable and Virtual production (CARV2011), Montreal, Canada, 3-5 October 2011. Berlin: Springer; 2011, p. 2-9.

[2] Wiendahl H, ElMaraghy HA, Nyhuis P, Zäh MF, Wiendahl H, Duffle N et al. Changeable Manufacturing - Classification, Design and Operation. CIRP Annals - Manufacturing Technology 2007;56(2):783-809.

[3] ElMaraghy HA. Changeable and reconfigurable manufacturing systems. London: Springer; 2009.

[4] Schuh G, Potente T, Thomas C, Hauptvogel A. Cyber-Physical Production Management. In: Prabhu V, Taisch M, Kiritsis D, editors. Advances in Production Management Systems. Sustainable Production and Service Supply Chains: Proceedings. Berlin: Springer; 2013, p. 477484.

[5] Schuh G, Stich V (eds.). Produktion am Standort Deutschland (Production in Germany); 2013.

[6] Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University. ProSense; Available from: www.prosense.info.

[7] Zäh MF, Reinhart G, Ostgathe M, Geiger F, Lau C. A holistic approach for the cognitive control of production systems. Adv. Eng. Inform 2010;24(3):300-7.

[8] Gausemeier J, Frank U, Donoth J, Kahl S. Specification technique for the description of self-optimizing mechatronic systems. Res Eng Design 2009;20(4):201-23.

[9] Warnecke H, Hüser M. The fractal company: A revolution in corporate culture. Berlin: Springer; 1993.

[10] Valckenaers P, Bonneville F, van Brussel H, Bongaerts L, Wyns J. Results of the Holonic Control System Benchmark at KULeuven. In: Proceedings of the Fourth International Conference on Computer Integrated Manufacturing and Automation Technology; 1994, p. 128-133.

[11] Okino N. Bionic Manufacturing Systems: Modelon-Based Approach. In: Proceedings of the CAM-I 18th Annual International Conference; 1989, p. 485-492.

[12] Ueda K. A concept for bionic manufacturing systems based on dna-type information. In: Proceedings of the IFIP TC5 / WG5.3 Eight International PROLAMAT Conference on Human Aspects in Computer Integrated Manufacturing; 1992, p. 853-863.

[13] Van Brussel H, Wyns J, Valckenaers P, Bongaerts L, Peeters P. Reference architecture for holonic manufacturing systems: PROSA. Computers in industry 1998;37(3):255-74.

[14] Fowler JW, Rose O. Grand Challenges in Modeling and Simulation of Complex Manufacturing Systems. SIMULATION: Transactions of The Society for Modeling and Simulation International 2004;80(9):469-76.

[15] Selke C. Entwicklung von Methoden zur automatischen Simulationsmodellgenerierung (Development of methods for automatic model generation). PhD-Thesis. Germany, Munich: Utz; 2005.

[16] Reinhart G, Gyger T. Identification of implicit strategies in production control. In: IEEM 2008: The IEEE International Conference on Industrial Engineering and Engineering Management 8 to 11 December, 2008, Singapore. Piscataway, N.J.: IEEE Xplore; 2008, p. 302-306.

[17] Horn S, Weigert G, Werner S. Data coupling strategies in production environments. In: Proceedings of the 28th International Spring Seminar on Electronics Technology: Meeting the challenges of electronics technology progress. Piscataway, NJ: IEEE; 2005, p. 278-282.

[18] Pfeiffer A. Novel Methods for Decision Support in Production Planning and Control. Dissertation. Budapest, Ungarn; 2007.

[19] Pfeiffer A, Kádár B, Szathmári M, Popovics G, Vén Z, Monostori L. Self-building simulation tool for daily decision support in production control. In: Proceedings of the International Workshop on Modeling & Applied Simulation MAS 2009; 2009, p. 246-254.

[20] Kádár B, Pfeiffer A, Monostori L, Popovics G, Vén Z. Co-analysing situations and production control rules in a large-scale manufacturing environment. In: Proc. of the 44th CIRP conference on manufacturing systems; 2011, p. 3-5.

[21] Kapp R, Le Blond J, Löffler B, Westkämper E. Integrated Factory and Logistics Planning. In: Zaeh M, Reinhart G, editors. 1st International Conference on Changeable, Agile, Reconfigurable and Virtual Production (CARV 2005). München: Herbert Utz Verlag; 2005, p. 193-199.

[22] Kapp R. Ein betriebsbegleitendes Fabriksimulationssystem zur durchgängigen Unterstützung der kontinuierlichen Fabrikadaption (Online factory simulation to support continuous adaption of factories). PhD-Thesis. Germany, Heimsheim: Jost-Jetter; 2011.

[23] Baheti R, Gill H. Cyber-physical Systems. In: Samad T, Annaswamy A, editors. The Impact of Control Technology; 2011, p. 161-166.

[24] Rajkumar R, Lee I, Sha L, Stankovic J. Cyber-physical systems: the next computing revolution. In: Proceedings of the 47th Design Automation Conference. Anaheim, California: ACM; 2010, p. 731-736.

[25] Lee EA. Cyber Physical Systems: Design Challenges. In: 11th IEEE International Symposium on: Object Oriented Real-Time Distributed Computing (ISORC); 2008, p. 363-369.

[26] Li R, Shyu Y, Adiga S. A heuristic rescheduling algorithm for computer-based production scheduling systems. International Journal of Production Research 1993;31(8): 1815-26.

[27] Kletti J, Schumacher J. Die perfekte Produktion (Perfect manufacturing). Berlin: Springer; 2011.

[28] Jolliffe IT, Stephenson DB. Forecast verification: A practioner's guide in atmospheric science. 2nd ed. Oxford: Wiley-Blackwell; 2012.