Available online at www.sciencedirect.com

ScienceDirect

Procedía CIRP 25 (2014) 185 - 191

8th International Conference on Digital Enterprise Technology - DET 2014 - "Disruptive Innovation in Manufacturing Engineering towards the 4th Industrial Revolution

Capacity planning and resource allocation in assembly systems consisting of dedicated and reconfigurable lines

Dávid Gyulaia,b*, Botond Kádára, László Monostori

aFraunhofer Project Center on Production Management and Informatics, Institute for Computer Science and Control, Hungarian Academy of Sciences, H-1111 Budapest, Kende u. 13-17, Hungary b Department of Manufacturing Scence and Technology, Budapest University of Technology and Economics, H-1111 Budapest, Egry József u. 1. Hungary

* Corresponding author. Tel.: +36-1-279-6176; fax: +36-1-466-7503, E-mail address: david.gyulai@sztaki.mta.hu

Abstract

Companies with diverse product portfolio often face capacity planning problems due to the diversity of the products and the fluctuation of the order stream. High volume products can be produced cost-efficiently in dedicated assembly lines, but the assembly of low-volume products in such lines involves high idle times and operation costs. Reconfigurable assembly lines offer reasonable solution for the problem; however, it is still complicated to identify the set of products which are worth to assemble in such a line instead of dedicated ones. In the paper a novel method is introduced that supports the long-term decision to relocate the assembly of a product with decreasing demand from a dedicated to a reconfigurable line, based on the calculated investment and operational costs. In order to handle the complex aspects of the planning problem a new approach is proposed that combines discrete-event simulation and machine learning techniques. The feasibility of the approach is demonstrated through the results of an industrial case study.

© 2014 The Authors.PublishedbyElsevierB.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/).

Peer-review under responsibility of The International Scientific Committee of the 8th International Conference on Digital Enterprise

Technology - DET 2014 - "Disruptive Innovation in Manufacturing Engineering towards the 4th Industrial Revolution"

Keywords: Capacity planning, Reconfigurable assembly systems, Machine learning

1. Introduction

Capacity management and long-term resource allocation are complex strategic planning tasks in a sense that they have to deal not only with the available production capacities and financial constraints but also need to consider the possible future changes in the order stream and product life-cycles. In general, production planning is a hierarchical process that has three main levels based on the length of planning horizon and level of aggregation. Capacity planning and management is on the strategic level that is based on long-term business goals and market forecasts. The main function of the strategic planning is to decide about the products to be produced and the necessary capacities that are required to fulfill the demands.

In order to meet the customer demands, not only the production process, but also the structure of the production system has to be robust against both internal and external

conditions. Dynamic changes of the market environments and high variety in the product portfolio require responsive production systems that are able to react to external changes by the ability of adaptation and robustness [1].

The efficient management of variety in production is one of the greatest challenges in today's industry [2]. Depending on the order volumes of products and the diversity of the product portfolio, different solutions exist to ensure cost-efficient production. The capacity planning method proposed in the paper considers two main types of manufacturing lines, namely dedicated and reconfigurable ones. Dedicated lines are designed around a certain product/product family, and they enable efficient production of large volumes with low product variety. In contrast, reconfigurable lines are suitable to produce a set of different products with high variety in the volumes as well as in the product structure. Naturally, the main drawbacks of the reconfigurable lines are the lower throughput and

2212-8271 © 2014 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/).

Peer-review under responsibility of The International Scientific Committee of the 8th International Conference on Digital Enterprise Technology - DET 2014 - "Disruptive Innovation in Manufacturing Engineering towards the 4th Industrial Revolution" doi: 10.1016/j.procir.2014.10.028

complex planning and control methods necessary to use them efficiently.

Although, both capacity planning and reconfigurable assembly systems (RAS) have broad literature, only few papers consider methods that combine dedicated and reconfigurable lines and dynamic assignment of products to the systems over the planning horizon. In [3] Ceryan and Koren introduce a method that formalizes the capacity planning as an optimization problem based on the flexible premium and determines the optimal resource portfolio for a fixed planning horizon. In [4], [5], [6] and [7] the capacity and system configuration planning process is formulated as a Markovian Decision Problem (MDP) in order to handle the dynamic changes of the order stream and system structure. These methods consider the capacity planning and management as a sequence of decisions on a longer horizon, thus their objective is to find an optimal capacity management policy to minimize costs on the long run. Hon and Xu propose a simulation-based method to optimize the system structure of a reconfigurable system based on the different stages of the products lifecycle [8]. Arafa and ElMaraghy introduce a method that investigates the manufacturing strategy of a firm that faces dynamic market environment, by calculating the volume flexibility of production [9]. Bohnen et al. propose a method for leveling low volume and high mix production based on the identified patterns in the assembly processes [10]. Though, leveling of production is an efficient method to decouple production orders and customer demands, it is more suitable for multi-product assembly lines to minimize the losses caused by the changeovers. In [11], Bruccoleri et al. introduce an agent-based production planning model that considers a multi-level, multi-period decision making process with reconfigurable enterprises and production systems.

In the paper, a novel method is introduced that supports the long-term decision to relocate the assembly of a product with decreasing demand from a dedicated to a reconfigurable line, based on the calculated investment and operational costs. The proposed solution considers two enablers in variety management strategies, planning and manufacturing [2].

The proposed capacity planning method combines discrete-event simulation and machine learning techniques in order to provide a reliable solution in short time. Machine learning-based prediction of dynamic system behavior is applied not only for decreasing the time-consumption planning process caused by the simulation analysis, but also represents an important step towards more complex methods, as for example reinforcement learning (RL). In RL-based formulation of the resource management problem simulation cannot be applied directly, but only as a prediction model of the cost functions.

2. Problem formulation

In the following sections, the considered planning problem is formulated. First the assembly system is introduced that is consisted of reconfigurable and dedicated lines. Then the objective of the capacity management method is stated, namely to reduce the production costs on the long run by assigning the products to the proper type of assembly line. The formulated capacity planning and resource allocation problem is visualized in Fig. 1.

Fig. 1. Illustration of the considered planning problem

2.1. Structure of the considered assembly system

In manufacturing systems that handle diverse product portfolio with both low- and high-volume products, fluctuating production volumes and different stages of products' lifecycle require the regular revision of the production structure applied. In order to minimize the operation and investment costs, effective capacity planning and resource allocation methods are required.

In a preceding publication the authors of the paper proposed a method for replacing dedicated assembly lines with modular reconfigurable ones on the base of standardized assembly processes [12]. The modular reconfigurable lines are set-up by mobile standard workstations based on the sequence of the assembly processes, and they are installed on the shop-floor one after each other by human operators. The dynamic changeability of such a system provides efficient production for low-volume products with high variety. Despite the dynamic behavior of RAS, the necessary resource pool configuration can be estimated based on the order stream and the throughput of the lines can be analyzed by applying discrete event simulation [12].

The goal of the presented capacity planning and resource allocation method is to assign the low- and high-volume products to reconfigurable and dedicated assembly lines respectively, in order to minimize the cost of production in a certain period. In order to determine the optimal set of products that should be assigned to reconfigurable lines, a cost model is defined as it follows.

2.2. Cost model

Although manually operated modular reconfigurable systems offer cost-efficient solution for the production of low-volume products, the investment and operational costs of the system increase significantly with the production volume and also influenced highly by the assigned product mix and production plan. For high volume production, dedicated production systems are applied that have high throughput, high level of automation and thus high investment costs. In the planning problem, decisions are made in discrete time steps regarding the assignment of the products and necessary investments. The proposed approach applies statistical learning techniques to estimate the costs by predicting the number of machines and the makespan based on the given order volumes (section 3.2). In order to manage the resources consisting of

dedicated and reconfigurable machines, a cost function is applied that are generally based on the production volumes, the price of the resources and the operational costs. The variables of the functions are concerned on a fix time interval and the price of the machines (based on depreciation) and cost of the operators are proportional to the time. The line assignment problem can be seen as subdividing the set of products into products assembled on the dedicated lines and on the reconfigurable lines by determining the value of Xp (section 3.3).

a R for the reconfigurable, and D for the dedicated lines

production cost in system a set of machine types

set of products

set of shifts

purchase price of machine j

o " operation cost of machine j per shift

h cost of an operator per shift

rjp required number from machine j by product p

tm(xp ) makespan in system a

da(x ) required number of machines j in system a

cost of the space per machines

x x=l if peR, x=0 otherwise

The applied cost function:

l \ da(ea+sa')+c

h + oada 1 +

c" =ta

m J' J'

V j=i J

cb = (tm~tn )( h + ]To )

a 2I a |

L. - ' J

Objective function: min (cR(xp)+cD(xp~j)

The following assumptions are made. Order volumes and forecasts are available for the given time horizon. All products can be assembled in either a reconfigurable or a dedicated line. It is assumed that the capacity of a single line is sufficient to assemble the product in the desired volume, and therefore, the option of dividing the order volume between different production modes can be ignored. Although certain products exist in different variants, these variants are always produced together, and therefore, they can be considered as single products on this level of the production planning hierarchy. The price of the machines and the costs of human operators are constant over time. The throughput of dedicated lines are rather higher compared to the reconfigurable ones, therefore backlogs can occur only in the reconfigurable lines.

2.3. Assignment of products to dedicated or reconfigurable lines

The capacity management method is based on a sequence of decisions; each of them determines the set of products that should be relocated from a dedicated to a reconfigurable line, based on the production costs. The initial step of the planning sequence considers a company with a diverse product portfolio that includes several products with different order volumes and life-cycle stages. It is assumed that all products are assembled on dedicated lines at the initial state. The first decision determines a set of products that should be assigned to reconfigurable lines, and the required set of modular resources. In the following steps, the decisions involve the expansions of this resource set, and the assignment of additional products to either of the lines. The planning method has to prevent the time to time reassignment of the products, since a dedicated production line for a certain product is only installed one time, typically in the ramp-up phase of the life-cycle. Once a product is relocated from the dedicated line to the reconfigurable one, the dedicated line is dismantled and could not be operated again.

3. Solution method

The proposed planning approach combines discrete-event

simulation (DES) with machine learning techniques, in order to

determine the parameters of the cost function for a given

period. Simulation is applied to determine the makespan and

resource pool configuration of several random-scenarios.

Based on the data provided by the analysis, a prediction method is defined, that can be integrated in a mathematical model and

able to predict the production costs for given order volumes and resource pools. To assign the products to dedicated and reconfigurable lines, a mixed-integer problem is formulated

that determine the values of Xp for each product p. 3.1. Performance analysis with discrete-event simulation

Discrete-event simulation is a generic Digital Enterprise Technology (DET) for evaluating the performance of manufacturing systems and analyzing the underlying processes. The greatest benefit of using DES is to make detailed experiments without having the real production systems. It becomes even more beneficial in case of dealing with complex system structures and product portfolios on a long planning horizon.

In this case, the simulation analysis is applied to determine the production costs of the dedicated and reconfigurable lines by investigating different production scenarios. The scenarios are artificially generated in order to simulate the response of the systems for different order volumes and resource pool configurations. The output values of the experiments are the variables of the presented cost functions, such as the necessary resource pool configurations and the makespan. The solution considers a discrete planning horizon with discrete time steps and the possibility of capacity expansion. In the planning process all products of the portfolio are encountered and the assignment of the products to

reconfigurable or dedicated lines is based on the costs of production considering the order volumes as well as their resource requirements. The optimal resource allocation and capacity plan is based on the work content w of the assembly processes, that is given by the ordered volumes qtp, process time pp and setup times tsp for each product p. The work content of the assembly processes for a certain planning horizon and productp is calculated as it follows:

-qtp pp

By using the output of the simulation study, an equilibrium can be determined at which the production cost of the allocated work content is the same for the dedicated and reconfigurable lines (Fig. 2).

25000 20000 15000 10000 5000 0

-Dedicated -Reconfigurable

Fig. 2. Trends of the production costs for the dedicated and reconfigurable lines

3.2. Prediction with random forests

Although DES is an efficient tool for system performance evaluation, it has significant costs originating from the time consumption of the model building and data collection. Therefore a machine learning technique is proposed that is much faster, and predicts the variables of the cost-function accurately.

The considered planning problem has to deal with all the products in the portfolio as regressor variables, in order to estimate the parameters of the cost function accurately. Common applied machine learning techniques such as support vector machines or neural networks are effective techniques for multinomial regression, however their prediction error is too large in case of many, or loosely correlated variables, such as the products in the portfolio.

Regression with tree-based models is a fundamental approach of the data analysis and machine learning. The general idea behind these methods is the partition of the feature space into a set of disjoint rectangular regions, and fit a simple model in each one [13]. However, regression with tree models is quite an accurate method; overfitting is a common problem in case of several regressor variables.

Random forest (RF) is an ensemble method for classification and regression that is based on building a large collection of de-correlated trees and averaging them by bagging (or bootstrap aggregation) [14]. Each tree is built with a maximum depth over a different bootstrap sample of the training data that ensure the accuracy of the method. The RF models are fast to train, robust against overfitting and generally

the method outperforms many other classifiers including discriminant analysis, support vector machines and neural networks [15][16]. The most important limitation of this method is that regression cannot be applied beyond the ranges of the training dataset.

Applying RF-s, the variables of the cost functions such as the required number of resources and the makespan can be predicted based on the data provided by the DES. The input data of the RF prediction is given by the results of the

experiments that is split up into a training and a test dataset. The regressor variables are the amount of the resources (e.g. the number of the machines) and the order volume of the products for the considered period. The outputs of the method are the predicted values of tmR and JR that are necessary to calculate the production costs for both types of lines. The RF models -built over the training set can predict accurately all the parameters of the cost model by having the ordered quantities and the available resources. Although the method is used to predict the available performance of the existing system, it also provides information about necessary changes regarding the quantity of different resources.

Pseudocode: Random Forest algorithm for regression [13]

1. For b=l to B

(a) Draw a bootstrap sample Z* of size N from the training data.

(b) Grow a random-forest tree Tb to the bootstrapped data, by recursively repeating the following steps for each terminal node of the tree, until the minimum node size nmin is reached.

i. Select m variables at random from the p variables (typically, m=p/3).

ii. Pick the best variable/split-point among the m.

iii. Split the node into two child nodes.

2. Output the ensemble of treps {Tb^!

To make a prediction x: fB (x) T (x)

f = _ / b

3.3. Planning sequence

The planning sequence starts from an initial state at where all the products are assembled on dedicated lines. The planning decisions are made over a horizon in discrete time steps ti...t„. On the one hand, decisions involve capacity allocation, namely to assemble a product in a dedicated or a reconfigurable line based on the predicted operational and investment costs. On the other hand, necessary capacity expansions are also calculated in case it is required by the ordered volumes. As the considered capacity planning and allocation problem requires long term decisions, both the orders-on-hand and forecast data are used for calculating the required capacities. A capacity plan at a certain time period regards a volume of a product by calculating the discounted weighted average of the order volumes and the forecast volumes for the next periods.

wp = tsp

Initialization

Cost equlibrium (W)

Training dataset: tm, J

Prediction with Random Forest

<- q <

max X xp

Cost calculation

( Stop )

Planning on rolling horizon

Fig. 3. Schematics of the proposed planning sequence

In the initial state at to, the calculations are started with the estimation of the required resources and assignment of the products to a dedicated or a reconfigurable line, based on the simulation results. Having the production volumes for each products of the portfolio, the work contents can be calculated. Based on the equilibrium point W that is defined by the simulation, an upper bound of the total work content can be determined that is worth to be assigned to a reconfigurable line (Fig. 3). In order to maximize the savings, the selection of the products to be relocated is formulated as a knapsack problem whose objective is to maximize the number of the products assigned to reconfigurable lines:

max /x,

p=1 subject to:

^Xp Wp < W p=1

In case a product should be relocated from one type of line to another based on the previous selection, it is given a "flag" (if Xp=1) in order to avoid the frequent reassignment, since in an ideal case a product is only relocated two times during its lifecycle (ramp-up and ramp-down). As the portfolio is separated by the previous steps, the cost functions of both the dedicated and reconfigurable lines are calculated with random forest prediction, the regressor variables are qtp, JR and JD.

As the capacity management considers the planning horizon in discrete steps, the above calculations are made in every tl.tn with some modifications from the initial state. At to, we suppose that the reconfigurable lines do not exist, in contrast to the next steps when both types have some assigned products and resource pools. Thus, in the next time steps the total work content of the "flagged" lines are calculated first, and the knapsack problem is solved by the difference to the equilibrium that prevents the time to time reassignment of a product. Then, the capacity requirement of the orders is recalculated with the already available resources, and the required expansions in the

resource pool configurations can be performed by investing in new resources. The last step is the calculation of the production costs with the considered orders. The steps of the whole planning process are shown in Fig. 3.

4. Experimental results

The proposed capacity planning and resource allocation method was tested on an industry-related dataset, considering both historical and forecast volumes and real production lines. The horizon of the test case was 10 years that was divided into three-month steps when capacity management decisions are made. The product portfolio consisted of 67 products with various volumes and assembly processes.

The simulation models of the dedicated and reconfigurable lines were implemented in Siemens Plant Simulation by applying simple models. As the input of the simulation, different production scenarios were generated with different order streams and resource pool configurations. Both in the dedicated and reconfigurable cases, different scenarios were analyzed, and the production costs were calculated based on the results of the simulation. According to the simulation results, the production cost of the dedicated lines is in linear correlation with the allocated work content in contrast to the reconfigurable case, where dynamic changes of the lines result in a noisy cost function (Fig. 4). The noisy nature of the reconfigurable costs is resulted by the large amount of backlogs caused by insufficient number of resources. According to the results, the equilibrium is W=1100 hours, that is the capacity constraint of the knapsack problem (Eq. 5).

The production planner is implemented in R that is a commonly applied software environment for data analysis [17]. First the simulation results of the reconfigurable lines were split into training and a test dataset in both cases. Then random forest models were fit on the cost function variables applying a

training dataset, and the models were validated by predictions on the test dataset. Despite the dynamic behavior of the reconfigurable lines, the random forest method provided an accurate prediction with a ~3% error over the test dataset. The time consumption of RF-based prediction of the cost function is ~2% of the DES's running time.

The simulation dealt with a dataset that contains the production data with orders-on-hand (~1 year) and forecast volumes. Order volumes of the further periods (from the 14th period) are forecasted with prediction functions based on the

9000 8000 7000 ™ 6000 J 5000 £ 4000 S 3000 2000 1000 0

—I Jrlrt

Iii e y

^ > . j|> yvJW T' ft^

i^HI 1 1 III ■ III! IIJVilUl 1 -

910 1110 1310 1510 1710 Work content [Hours]

-Dedicated — Reconfigurable

Fig. 4. Cumulated cost savings over the planning horizon

available production data, in order to simulate a discrete planning horizon.

12000 '

10000 :

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 Time [Months]

Workcontent -Dedicated & Reconfigurable -Reference

Fig. 5. Production costs over the planning horizon

According to the results the cost savings are significant, applying modular reconfigurable lines results in ~30% savings in the overall costs of the production (Fig. 5). Applying only dedicated lines (reference values), the trend of the production cost is the same as the trend of the order stream; while the trend of the production costs when applying a reconfigurable resources can be different due to the more complex underlying dynamic processes. In case a high volume is required from a certain type of product that is already an end-of-lifecycle product and assigned to the reconfigurable lines, the throughput may not be enough to produce the ordered quantities without delay. As the backlogs are penalized with a quadratic formula in the cost function (Eq. 3), high production costs can be resulted in that cases (e.g. 14th or 24th periods in the test case).

Although such peaks can occur during a long horizon caused by unpredictable factors, operating modular reconfigurable lines with proper resource pool and resource allocation method results significant savings on the long run. Despite the fluctuating order stream and divers product portfolio, the value of cumulated savings increase monotonously (Fig. 6).

140000 120000 100000 80000 S 60000 U 40000 20000 0

16000 14000 12000 10000 8000 6000 4000 2000 0

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 Time [Months]

Workcontent -Cumulated savings

Fig. 6. Cumulated cost savings over the planning horizon

5. Summary

Both dedicated and reconfigurable assembly lines can show up benefits and drawbacks, consequently, their co-existence in the industrial practice is natural and frequent. In the paper, a novel approach for capacity management for assembly system with dedicated and reconfigurable assembly lines was presented that facilitates the economical production of a diverse, varying

product portfolio consisting of high- and low- volume products. The presented approach is generally based on discrete-event simulation and random forest based regression, and able to handle the changes and disturbances of the production caused by the fluctuating order stream and diverse product portfolio with low- and high-volume products. The considered approach supports effectively the strategic planning processes on a long-term horizon, based on the forecasted market conditions. First, the structure of the modular reconfigurable assembly system was introduced, and production cost functions were formulated with the factors that are relevant in the strategic planning. The proposed method assigns the products/product families to dedicated or reconfigurable assembly lines, based on the forecast volumes and predicted production costs. The solution is generally based on the combination of discrete-event simulation and random-forest regression. The simulation analysis is required by the dynamic underlying processes of the reconfigurable lines, and provides training data for the next phase. Random forest regression is a novel machine learning technique, which is used to predict production costs accurately based on the order stream and different resource pool configurations.

The proposed planning method was evaluated by using real industrial data. The test results show that significant savings can be realized by applying modular reconfigurable lines with proper planning and control method.

The other more important point is that the regression with a tree-based method provides the possibility to implement more complex planning models that require mathematical functions for optimization, and simulation results cannot be used directly. The proposed capacity management method is a step towards more complex robust planning models such as MDP and RL that are aimed at minimizing the cost functions on the long run beside the stochastic nature of some parameters. In those cases the time consumption of the simulation would result in complex problems that cannot be solved efficiently in finite time.

Acknowledgements

Research has been partially supported by National Development Agency, Hungary Grant No. ED_13-2-2013-0002 and by the European Union 7th Framework Programme Project No: NMP 2013-609087, Shock-robust Design of Plants and their Supply Chain Networks (RobustPlaNet).

References

[1] Vancza J, Monostori L, Lutters D, Kumara S, Tseng M, Valckenaers P, Van Brussel H. Cooperative and responsive manufacturing enterprises. CIRP Annals - Manufacturing Technology 2011; 60(2) 797-820.

[2] ElMaraghy H, Schuh G, ElMaraghy W, Piller F, Schonsleben P,

Tseng M, Bernard A. Product variety management. CIRP Annals -Manufacturing Technology 2013; 62(2) 629-652.

[3] Ceryan O, Koren Y. Manufacturing capacity planning strategies. CIRP Annals - Manufacturing Technology 2009; 58(1) 403-406.

[4] Lanza G, Peters S, Herrmann HG. Dynamic optimization of manufacturing systems in automotive industries. CIRP Journal of Manufacturing Science and Technology 2012; 5(4) 235-240.

Dávíd Gyulaí et al. /Procedía CIRP 25 (20l4) l85 - l9l

[5] Lanza G, Peters S. Integrated capacity planning over highly volatile

horizons. CIRP Annals - Manufacturing Technology 2012; 61(1) 395-398.

[6] Asl FM, Ulsoy AG. Stochastic optimal capacity management in

reconfigurable manufacturing systems. CIRP Annals -Manufacturing Technology 2003; 52(1) 371-374.

[7] Colledani M, Tolio T. A decomposition method to support the

configuration/reconfiguration of production systems. CIRP Annals - Manufacturing Technology 2005; 54(1) 441-444.

[8] Hon K, Xu S. Impact of product life cycle on manufacturing

systems reconfiguration. CIRP Annals - Manufacturing Technology 2007 56(1) 455-458.

[9] Arafa A, ElMaraghy W. Manufacturing strategy and enterprise

dynamic capability. CIRP Annals - Manufacturing Technology 2011; 60(1) 507-510.

[10] Bohnen F, Buhl M, Deuse J. Systematic procedure for leveling of low volume and high mix production. CIRP Annals -Manufacturing Technology 2013; 6(1) 53-58.

[11] Bruccoleri M, Lo Nigro G, Perrone G, Renna P, Noto La Diega S, Production planning in reconfigurable enterprises and

reconfigurable production systems. CIRP Annals - Manufacturing Technology 2005; 54(1) 433-436.

[12] Gyulai D, Vén Z, Pfeiffer A, Váncza J, Monostori L. Matching demand and system structure in reconfigurable assembly systems. Procedia CIRP 2012; 3 579-584.

[13] Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning, Springer Series in Statistics, Springer New York Inc., New York, NY, USA, 2001.

[14] Breiman L, Random forests. Machine Learning 2011; 45(1) 5-32.

[15] Liaw A, Wiener M. Classification and regression by randomforest. R News 2002; 2(3) 18-22.

[16] Chan JCW, Paelinckx D. Evaluation of random forest and adaboost tree-based ensemble classification and spectral band selection for ecotope mapping using airborne hyperspectral imagery Remote Sensing of Environment 2008; 112(6) 2999-3011.

[17] R Core Team. A Language and Environment for Statistical Computing. R Foundation for Statistical Computing Vienna, Austria 2013; http://www.R-project.org/