Scholarly article on topic 'Supplier Selection based on AHP QFD Methodology'

Supplier Selection based on AHP QFD Methodology Academic research paper on "Materials engineering"

CC BY-NC-ND
0
0
Share paper
Academic journal
Procedia Engineering
OECD Field of science
Keywords
{"Analytic hierarchy process" / "Quality Function Deployment" / "Supplier selection"}

Abstract of research paper on Materials engineering, author of scientific article — G. Rajesh, P. Malliga

Abstract Strategic partnership with better suppliers needs to be formed to improve quality, flexibility as well as to reduce lead time. The “voice” of company stakeholders has to be considered so that suppliers selected may provide what the company wants. In this paper an integrated approach, combining Analytic Hierarchy Process (AHP) and Quality Function Deployment (QFD) is developed to select suppliers strategically. When the House of Quality (HOQ) is used in supplier selection, the company begins with the features that the purchased product must have in order to meet certain requirements that the company has established and then tries to identify which of the supplier's attributes have the greatest impact on the achievement of its established objectives. QFD provides the importance weightings of evaluating criterion, which are derived by the importance ratings of stakeholder requirements together with the relationship weightings between stakeholder requirements and evaluating criterion. Based on the ranked criteria, alternative suppliers are evaluated and compared with each other using AHP again to make an optimal selection. A case study of a Precision Machined High Pressure Die Casting components company in selecting its supplier by using the proposed AHP QFD technique is presented.

Academic research paper on topic "Supplier Selection based on AHP QFD Methodology"

Available online at www.sciencedirect.com

ScienceDirect Procedia

Engineering

ELSEVIER Procedia Engineering 64 (2013) 1283 -1292 ;

www.elsevier.com/locate/procedia

International Conference On DESIGN AND MANUFACTURING, IConDM 2013

Supplier Selection Based on AHP QFD Methodology

G. Rajesha, P. Mal^a1^*

aResearch Scholar, Department of Industrial Engineering, Anna University,Chennai - 600025, India. bProfessor, Department of Industrial Engineering, Anna University, Chennai - 600025, India.

Abstract

Strategic partnership with better suppliers needs to be formed to improve quality, flexibility as well as to reduce lead time. The ''voice" of company stakeholders has to be considered so that suppliers selected may provide what the company wants. In this paper an integrated approach, combining Analytic Hierarchy Process (AHP) and Quality Function Deployment (QFD) is developed to select suppliers strategically. When the House of Quality (HOQ) is used in supplier selection, the company begins with the features that the purchased product must have in order to meet certain requirements that the company has established and then tries to identify which of the supplier's attributes have the greatest impact on the achievement of its established objectives. QFD provides the importance weightings of evaluating criterion, which are derived by the importance ratings of stakeholder requirements together with the relationship weightings between stakeholder requirements and evaluating criterion. Based on the ranked criteria, alternative suppliers are evaluated and compared with each other using AHP again to make an optimal selection. A case study of a Precision Machined High Pressure Die Casting components company in selecting its supplier by using the proposed AHP QFD technique is presented.

© 2013 The Authors. Published by Elsevier Ltd.

Selection and peer-review under responsibility of the organizing and review committee of IConDM 2013 Keywords: Analytic hierarchy process; Quality Function Deployment; Supplier selection.

* Corresponding author. Tel.: +91-44-22357685. E-mail address: mallipoo@annauniv.edu

1877-7058 © 2013 The Authors. Published by Elsevier Ltd.

Selection and peer-review under responsibility of the organizing and review committee of IConDM 2013 doi: 10. 1016/j .proeng.2013.09.209

1. Introduction

In increasingly competitive markets, customer satisfaction is a vital corporate objective. Key elements to increasing customer satisfaction include producing consistently high quality products and providing high quality customer service. In addition, the intensive global competition among manufacturers to coordinate and respond quickly the industry value chain from suppliers to customers has made customer supplier relationship management important in the new business era. In such circumstances the decision making in each business plays a key role in the cost reduction, and supplier selection is one of the important functions in the supplier relationship management. Very few manufactures now own all the activities along the chain but integrate the supply network from various supplier networks and the ability to make fast and accurate decision often constitute a competitive advantage compared with the competitors or other networks [1].Today's highly competitive environment is forcing the manufacturing organizations to establish a long-term effective collaboration with the efficient organizations. As a result an effective supplier selection process is very important to the success of any manufacturing organization [2].

With the increasing significance of the purchasing function, purchasing decisions become more important. As organizations become more dependent on suppliers the direct and indirect consequences of poor decision making become more severe. In addition, several developments further complicate purchasing decision making. Globalization of trade and the Internet enlarge a purchaser's choice set. Changing customer preferences require a broader and faster supplier selection. Public Procurement regulations demand more transparency in decisionmaking. New organizational forms lead to the involvement of more decision-makers. These developments strongly urge for a more systematic and transparent approach to purchasing decision making, especially regarding the area of supplier selection. Contemporary operations research offers a range of methods and techniques that may support the purchasing decision maker in dealing with the increased complexity and importance of his/her decisions. Examples of such techniques are multi-criteria decision aid, problem structuring approaches, mathematical programming and data mining techniques [3].

This paper addresses the relationship among the criteria for supplier selection decision making. Supplier selection is viewed as a combination of both customer requirements and engineering requirements. Customers are the companies that purchase the technical expertise of the suppliers. Therefore, such a company supplier relation can be viewed as a 'house of quality' model. The outcome of the integrated methodology presented in this paper. QFD and AHP techniques have been briefed and subsequently the proposed integrated hierarchical methodology for supplier selection is delineated.

2. Literature Review

Weber et al. (1991) [4] reviewed 74 supplier selection articles from 1966 to 1991 and showed that more than 63% of them were in multi-criteria environment. A recent trend in manufacturing strategy is the implementation of the Just-In-Time (JIT) philosophy influences the selection criteria. De Boer et al. (2001) [3] in his monumental review initially discusses the importance and complexity involved in the purchasing decisions and gives a frame work for purchasing decisions. Decision methods for problem definition and formulation of criteria, decision methods for pre-qualification of suitable suppliers are presented.

Lee et al. (2001) [5] suggested a methodology leading to effective supplier management processes utilizing information obtained from the supplier selection processes. For this methodology, they propose the supplier selection and management system (SSMS) that includes purchasing strategy system, supplier selection system, and supplier management system and explained how the SSMS is applied to a real supply chain. William Ho et al. (2010) [6] in his paper reviewed 78 journal articles between 2000 and 2008. It was found that numerous individual and integrated approaches were proposed to solve the supplier selection problem. The most prevalent individual approach is DEA, whereas the most popular integrated approach is AHP-GP. Second, it was observed that price or cost is not the most widely adopted criterion. Instead, the most popular criterion used for evaluating the performance of suppliers is quality, followed by delivery, price or cost, and so on.

Bevilacqua et al. (2006) [7] starts the study by identifying the features that the purchased product should have (internal variables ''WHAT'') in order to satisfy the company's needs, then seeks to establish the relevant supplier

assessment criteria (external variables ''HOW'') in order to come up with a final ranking based on the fuzzy suitability index (FSI).

Ha and Krishnan (2008) [8] presented a hybrid method, which incorporates multiple techniques (AHP, DEA, and NN) into an evaluation process, in order to select competitive suppliers in a supply chain. A combined supplier score and a supplier map were devised. Faez et al. (2009) [9] developed an integrated model based on the Case Based Reasoning (CBR) method in a fuzzy environment and mathematical programming for a single item vendor selection process.

Bhattacharya et al. (2010) [10] delineated a concurrent engineering approach integrating analytic hierarchy process (AHP) with quality function deployment (QFD) in combination with cost factor measure (CFM) to rank and subsequently select candidate suppliers under multiple, conflicting-in-nature criteria environment within a value-chain framework. Vinodh et al. (2010) [11] developed a model based on supplier selection weighted index (SSWI) generated out of fuzzy ANP, the best supplier is selected and this is followed by the sensitivity analysis for validating the results of fuzzy ANP.

Punniyamoorthy et al. (2011) [12] presented a model in which structural equation modelling (SEM) approach is used to arrive at the relative weightage of the criteria, Fuzzy AHP is used to arrive at the relative weightage of the suppliers with respect to each criteria. Mithat Zeydan et al. (2011) [13] used three multi-criteria decision making method to find efficient and inefficient suppliers sensitively. In the first step, fuzzy AHP is used for the determination of criteria weights and fuzzy TOPSIS to transform the qualitative variables into only one quantitative variable. In the second step, DEA is used for the ranking of efficient and inefficient suppliers.

William Ho et al. (2010) [6] in his suggestions for future work stated that although the popular approaches can deal with multiple and conflicting criteria, they have not taken into consideration the impact of business objectives and requirements of company stakeholders on the evaluating criteria. In reality, the weightings of supplier evaluating criteria depend a lot on business priorities and strategies. In cases where the weightings are assigned arbitrarily and subjectively without considering the ''voice" of company stakeholders, the suppliers selected may not provide what the company exactly wants. To enable the ''voice" of company stakeholders is considered, an integrated analytical approach, combining AHP and QFD, should be developed to select suppliers strategically. Specifically, multiple evaluating criteria are derived from the requirements of company stakeholders using a series of house of quality. The importance of evaluating criteria is prioritized with respect to the degree of achieving the stakeholder requirements using AHP. Based on the ranked criteria, alternative suppliers are evaluated and compared with each other using AHP again to make an optimal selection. Nevertheless, the proposed AHP-QFD approach has not been applied to the supplier selection problem yet. So, in this paper Hierarchical QFD Methodology, which integrates the AHP and QFD, is developed.

3. Quality Function Deployment

Quality Function Deployment (QFD) is a well-structured, cross-functional planning technique that is used to hear the customers' voice throughout the product planning, development, engineering and manufacturing stages of any product. It is stated that if appropriate methods are not used to establish importance weights, the weights cannot be assumed to be on a valid ratio scale in the house of quality (HOQ) matrix. Extensive analysis of literature reveals that the HOQ matrix is an almost universal tool that can be used for prioritising most of the tasks of any industry. The QFD technique is not designed as a general planning process. The technique is aimed at planning one specific product as opposed to quantifying the worth of alternatives. As the results of QFD are not intended for use as anything more than a general guideline for choosing priority items, it is required to make the QFD matrix more acceptable than an ordinary "guideline". Thus, integration with the operational research tool, AHP makes the QFD technique more acceptable to decision makers rather than using the same as a general guideline [10].

Bevilacqua et al. (2006) [10] in his paper describes the HOQ and its process following the approaches suggested by Brown (1991) and Griffin and Hauser (1992).

Relationship Matrix

Weight of HOWs (F)

Fig.l.House of quality.

Step 1: Identify the WHATs. The wanted benefits in a product or service in the customer's own words are customer needs and are usually called customer attributes (CA) or ''WHATs'', area (A) in Fig. 1. In assigning priorities to WHATs, it is necessary to balance efforts in order to accomplish those needs that add value to the customer. The priorities are usually indicated in the area designated as (B) in Fig. 1.

Step 2: Determination of HOWs. Engineering characteristics are specified as the ''HOWs'' of the HOQ and also called measurable requirements. HOWs are identified by a multidisciplinary team and positioned on the area marked as (C) on the matrix diagram, Fig. 1.

Step 3: Preparation of the relationship matrix (D). A team judges which WHATs impact which HOWs and to what degree.

Step 4: Elaboration of the correlation matrix. The physical relationships among the technical requirements are specified on an array known as ''the roof matrix'' and identified as (E) in Fig. 1.

Step 5: Action plan. The weights of the HOWs, identified as area (F), are placed at the base of the quality matrix. These weights are one of the main outputs of the HOQ, and are determined by

Weight (HOW)i = V(HOW)ii X imp(WHATi) + .....+ V(HOW)m X imp(WHATn)

Where V(HOW)m is the correlation value of HOWi with WHATn or priority of WHATn.

and imp (WHATn) represents the importance

4. Analytical Hierarchy Process

The Analytic Hierarchy Process (AHP) is a multi-criteria decision-making approach and was introduced by Saaty. The AHP has attracted the interest of many researchers mainly due to the nice mathematical properties of the method and the fact that the required input data are rather easy to obtain.

Fig. 2 The hierarchical structure of the decision making problem.

The AHP encompasses six basic steps as summarized as follows:

Step 1. AHP uses several small sub problems to present a complex decision problem. Thus, the first act is to decompose the decision problem into a hierarchy with a goal at the top, criteria and sub-criteria at levels and sub-levels of and decision alternatives at the bottom of the hierarchy (Fig. 2).

Table 1. The numerical assessments and their linguistic meanings.

Numerical assessment_Linguistic meaning

1 Equal important

3 Moderately more important

5 Strongly more important

7 Very strongly important

9 Extremely more important

2,4,6,8 Intermediate values of importance

Step 2. The decision matrix, which is based on Saaty's nine-point scale, is constructed. The decision maker uses the fundamental 1-9 scale defined by Saaty to assess the priority score. In this context, the assessment of 1 indicates equal importance, 3 moderately more, 5 strongly more, 7 very strongly and 9 indicates extremely more importance. The values of 2, 4, 6, and 8 are allotted to indicate compromise values of importance.

The decision matrix involves the assessments of each alternative in respect to the decision criteria. If the decision making problem consists of n criteria and m alternatives; the decision matrix takes the form:

r du di2 .

d21 d22 .

dm 1 dm

The elements {d^} signify the rating of the ith alternative in respect to the jth criteria.

Step 3. The third step involves the comparison in pairs of the elements of the constructed hierarchy. The aim is to set their relative priorities with respect to each of the elements at the next higher level. The pairwise comparison matrix, which is based on the Saaty's 1-9 scale, has the form:

'"$11 ai2 . . . ain

a11 a12 . . . a1n

a11 a12 . . . a1n

fw1lw1 w2/w1

w1lw2 .

w2lw2 .

w1lwn w2lwn

wnlw1 wnlw2 . . . wnlwn

If n(n - 1)/2 comparisons are consistent with n is the number of criteria, then the elements {a^} will satisfy the following conditions: ajj = w1/wj = 1/ajl and au = 1 with i, j, k = 1, 2,. . .,n.

In the comparison matrix, aij can be interpreted as the degree of preference of ith criteria over jth criteria. It appears that the weight determination of criteria is more reliable when using pairwise comparisons than obtaining them directly, because it is easier to make a comparison between two attributes than make an overall weight assignment.

Step 4. AHP also calculates an inconsistency index (or consistency ratio) to reflect the consistency of decision maker's judgments during the evaluation phase. The inconsistency index in both the decision matrix and in pairwise comparison matrices could be calculated with the equation:

CI = (?w-n) l ( n -1)

The closer the inconsistency index is to zero, the greater the consistency. The consistency of the assessments is ensured if the equality ajj - aik = a^ holds for all criteria. The relevant index should be lower than 0.10 to accept the AHP results as consistent. If this is not the case, the decision-maker should go back to Steps 2 and 3 and redo the assessments and comparisons.

Step 5. Before all the calculations of vector of priorities, the comparison matrix has to be normalized. Therefore, each column has to be divided by the sum of entries of the corresponding column. In that way, a normalized matrix is obtained in which the sum of the elements of each column vector is 1.

Step 6. For the following part, the eigenvalues of this matrix are needed to be calculated which would give the relative weights of criteria. This procedure is common in mathematics. The relative weights obtained in the third step should verify

A .W = Xmax .W (5)

where A represents the pairwise comparison matrix, W the eigenvector and Xmax the highest eigenvalue. If there are elements at the higher levels of the hierarchy, the obtained weight vector is multiplied with the weight coefficients of the elements at the higher levels, until the top of the hierarchy is reached. The alternative with the highest weight coefficient value should be taken as the best alternative [14] [15].

5. AHP QFD Methodology

Identify WHATs

Fig. 3 AHP QFD Methodology.

In this paper we propose the AHP-QFD methodology for supplier selection. The conceptual and procedural approach of the HOQ remains, though the roles have been inverted. In traditional QFD applications, the company has to identify its customers' expectations and their relative importance (external variables) in order to identify which design characteristics (internal variables) should be allocated the most resources; when the HOQ is used in supplier selection, on the other hand, the company starts with the features that the outsourced product/service must have in order to meet certain requirements that the company has established and consequently knows very well (so the customer's expectations become internal variables, since the company itself is the customer) and then tries to identify which of the suppliers' attributes (external variables) have the greatest impact on the achievement of its established objectives.

6. Case Study

AHP - QFD method is applied to a supplier selection process for a medium scale industry that manufactures precision machined aluminium alloy die cast components.

6.1. Identify the WHATs.

The fundamental characteristics required of products purchased from outside suppliers by the company considered in this study are Quality, Cost and Delivery

6.2. Determination of HOWs.

After review of the supply management literature, experts were presented with various criteria. Analysis identified seven criteria crucial to supplier assessment in our specific case. The following criteria (''HOWs'') were considered: (1) Experience , (2) Technical Capability, (3) Quality system certification, (4) Geographical position, (5) Raw material procurement, (6) Financial stability, (7) Attitude . Customers are showing a growing interest in the supplier's capacity for improvement in its organization and production, with the promise of more reliable and less costly products. The company's purchasing managers have indicated technological capability and financial considerations as priority issues. Apart from this raw material procurement of the supplier and transportation distance are found to be the additional factors indentified to play a significant role in this scenario.

6.3. Determination of Weights of WHATs using AHP

Using the scale shown in Table 1 the team was asked to compare the criteria. Results are written in matrix form.

Table 2. Comparison Matrix

Cost Delivery

Cost Delivery Quality

Table 3. Reciprocal Matrix

Quality

Criteria Cost Delivery Quality

Cost 1.00 0.50 0.14

Delivery 2.00 1.00 0.17

Quality 7.00 6.00 1.00

Sum 10.00 7.50 1.31

Table 4. Normalized Matrix

Criteria Cost Delivery Quality Sum Priority vector

Cost 0.10 0.07 0.11 0.28 0.09

Delivery 0.20 0.13 0.13 0.46 0.15

Quality 0.70 0.80 0.76 2.26 0.75

Sum 1.00 1.00 1.00 3.00 1.00

Lambda max = 3.06, Consistency index (CI) = 2.94%, Consistency ratio (CR) = 5.07% Reciprocal and Normalized matrix are calculated. The consistency ratio is within the acceptable limits.

6.4. Preparation of the relationship matrix

The impact of each ''HOW'' on each ''WHAT'' were recorded as linguistic variable High, Medium and Low. The numerical values of 9, 3 and 1 were assigned to high, medium and low respectively.

Table 5. Relationship matrix

What's Importance Experience Technical Capability Quality system certification Geographical Position Financial stability Raw material procurement Attitude

Quality 0.75 M H H

Cost 0.09 L L M H

Delivery 0.15 L M M L M

Table 6. HOQ (Weights of HOWs)

What's Importance Experience Technical Capabili Quality system certification Geographical Position Financial stability Raw material procurement Attitude Total

Quality 0.75 2.26 6.79 6.79 15.85

Cost 0.09 0.09 0.09 0.28 0.83 1.29

Delivery 0.15 0.15 0.46 0.46 0.15 0.46 1.69

Total 2.51 7.25 6.79 0.55 0.28 0.98 0.46 18.82

Relative Weight 13% 39% 36% 3% 1% 5% 2%

6.5. Calculation of Weights of HOWs

From the constructed House of Quality the relative weights of the How's (Criteria) are calculated. Scores reveal that technical capability and quality system certification are the major criteria fallowed by experience.

6.6. Computation of individual scores for each supplier using AHP

For every single criteria the six suppliers are compared using AHP, scores of each supplier for the criteria are computed individually. Table reveals the individual scores of each supplier.

Table 7. Suppliers scores for each Criteria.

S1 S2 S3 S4 S5 S6

Experience 0.32 0. 12 0.06 0.05 0 33 0.11

Technical Capability 0.43 0. 11 0.26 0.05 0 11 0.05

Quality system certification 0.33 0. 19 0.19 0.08 0 18 0.04

Geographical Position 0.08 0. 14 0.30 0.14 0 30 0.04

Financial stability 0.36 0. 11 0.11 0.06 0 06 0.29

Raw material procurement 0.36 0. 11 0.12 0.04 0 25 0.11

Attitude 0.32 0. 32 0.13 0.13 0 05 0.05

6.7. Ranking of Suppliers

Final ranking of suppliers is obtained by multiplying the individual scores of the suppliers (Table 7) with the relative weight of criteria (Table 6).

0.32 0.43 0.33 0.08 0.36 0.36 0.32 r 0.13 r "\ 0.36

0.12 0.11 0.19 0.14 0.11 0.11 0.32 0.39 0.14

0.06 0.26 0.19 0.30 0.11 0.12 0.13 X 0.36 = 0.19

0.05 0.05 0.08 0.14 0.06 0.04 0.13 0.03 0.06

0.33 0.11 0.18 0.30 0.06 0.25 0.05 0.01 0.18

0.11 0.05 0.04 0.04 0.29 0.11 0.05 J 0.05 0.02 0.06

Results reveal the ranking of suppliers in the order of S1 > S3 > S5 > S2 > S4 > S6. 7. Conclusion

In a manufacturing system which constantly strives to better quality and reduce costs, vendors play a crucial role. Although the popular approaches can deal with multiple and conflicting criteria, they have not taken into consideration the impact of business objectives and requirements of company stakeholders on the evaluating criteria. Integrated approach, combining AHP and QFD is developed to select suppliers strategically. This method uses the QFD methodology to consider the "voice" of company stakeholders. AHP offers a methodology to rank alternative courses of action based on the decision maker's judgments concerning the importance of the criteria and the extent to which they are met by each alternative. It provides a unique means of quantifying judgmental

consistency. For this reason, AHP is ideally suited for the supplier selection problem. To test efficacy of this method, it is applied to a supplier selection process for a Precision Machined High Pressure Die Casting components company. The study starts by identifying the features that the purchased product should have (internal variables ''WHAT''), then it seeks to establish the relevant supplier assessment criteria (external variables ''HOW''). Weights of WHATs are calculated using AHP. QFD provides the importance weightings of evaluating criterion. Alternative suppliers are evaluated and compared with each other using AHP. Final ranking of suppliers is obtained by multiplying the individual scores of the suppliers for criteria with the relative weight of criteria. The results reveal the practical feasibility and practical adaptability of this approach in the contemporary industrial scenario.

References

[1] K.L. Choy, W.B. Lee and Victor Lo, 2003. Design of a case based intelligent supplier relationship management system—the integration of supplier rating system and product coding system, Expert Systems with Applications Vol.25, pp 87-100.

[2] Felix T. S. Chan, N. Kumar, M. K. Tiwari, H. C. W. Lau and K. L. Choy, 2008 . Global supplier selection: a fuzzy-AHP approach, International Journal of Production Research, Vol. 46, No. 14, 15 July, 3825-3857.

[3] Luitzen de Boer, Eva Labro and Pierangela Morlacchi, 1996. A review of methods supporting supplier selection', European journal of purchasing & supply management, Vol.23, No.92, pp 75-238.

[4] Charles A. Weber, John R. Current and W. C. Benton., 1991 .Vendor selection criteria and methods., European Journal of Operational Research Vol.50, No.1, pp 2-18.

[5] Eon-Kyung Lee, Sungdo Ha, and Sheung-Kown Kim, 2001. Supplier Selection and Management System Considering Relationships in Supply Chain Management. IEEE Transactions on Engineering Management, vol. 48, No. 3.

[6] William Ho, Xiaowei Xu and Prasanta K. Dey., 2010.Multi-criteria decision making approaches for supplier evaluation and selection: A literature review. European Journal of Operational Research 202, pp 16-24.

[7] M. Bevilacqua, F.E. Ciarapica, G. Giacchetta, 2006. A fuzzy-QFD approach to supplier selection, Journal of Purchasing & Supply Management Vol.12 14-27.

[8] S.H. Ha, R. Krishnan., 2008. A hybrid approach to supplier selection for the maintenance of a competitive supply chain. Expert Systems with Applications 34 (2), 1303-1311.

[9] F. Faez, S.H. Ghodsypour and C.O' Brien., 2009. Vendor selection and order allocation using an integrated fuzzy case-based reasoning and mathematical programming model, Int. J. Production Economics 121 pp 395-408.

[10] Arijit Bhattacharya, John Geraghty ,Paul Young., 2010. Supplier selection paradigm: An integrated hierarchical QFD methodology under multiple-criteria environment, Applied Soft Computing, Volume 10, Issue 4, September, Pages 1013-1027.

[11] Vinodh S., Ramiya R.A., Gautham S.G., 2011. Application of fuzzy analytic network process for supplier selection in a manufacturing organisation. Expert Systems with Applications 38 (1), 272-280.

[12] Punniyamoorthy M., Mathiyalagan P., Parthiban P., 2011. A strategic model using structural equation modeling and fuzzy logic in supplier selection. Expert Systems with Applications 38 (1), 458-474.

[13] Zeydan M., Colpan C., Cobanoglu C., 2011. A combined methodology for supplier selection and performance evaluation. Expert Systems with Applications 38 (3), 2741-2751.

[14] Ali Yousefi , Abdollah Hadi-Vencheh, 2010. An integrated group decision making model and its evaluation by DEA for automobile industry, Expert Systems with Applications, Volume 37, Issue 12, December, 8543-8556.

[15] Isiklar G., Buyukozkan G., 2007. Using a multi-criteria decision making approach to evaluate mobile phone alternatives. Computer Standards & Interfaces, 29, 265-274.