BIOSCIENCE FRONTIERS

Accepted Manuscript

State-of-the-art review of some artificial intelligence applications in pile foundations Mohamed A. Shahin

PII: S1674-9871(14)00132-7

DOI: 10.1016/j.gsf.2014.10.002

Reference: GSF 327

To appear in: Geoscience Frontiers

Received Date: 11 August 2014 Revised Date: 17 October 2014 Accepted Date: 26 October 2014

Please cite this article as: Shahin, M.A., State-of-the-art review of some artificial intelligence applications in pile foundations, Geoscience Frontiers (2014), doi: 10.1016/j.gsf.2014.10.002.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Input data

x (input)

Output

_ Hidden Layer

Input Layer

State-of-the-art review of some artificial intelligence applications in pile foundations

Mohamed A. Shahin*

Department of Civil Engineering, Curtin University, Perth WA 6845, Australia

ABSTRACT

Geotechnical engineering deals with materials (e.g. soil and rock) that, by their very nature, exhibit varied and uncertain behavior due to the imprecise physical processes associated with the formation of these materials. Modeling the behavior of such materials in geotechnical engineering applications is complex and sometimes beyond the ability of most traditional forms of physically-based engineering methods. Artificial intelligence (AI) is becoming more popular and particularly amenable to modeling the complex behavior of most geotechnical engineering applications because it has demonstrated superior predictive ability compared to traditional methods. This paper provides state-of-the-art review of some selected AI techniques and their applications in pile foundations, and presents the salient features associated with the modeling development of these AI techniques. The paper also discusses the strength and limitations of the selected AI techniques compared to other available modeling approaches.

Keywords: Artificial intelligence, Pile foundations, Artificial neural networks, Genetic programming, Evolutionary polynomial regression

Corresponding author. Tel:+61 8 9266 1822. E-mail address: m.shahin@curtin.edu.au

1. Introduction

Over the last decade, artificial intelligence (AI) has been applied successfully to virtually every problem in geotechnical engineering. Examples of the available AI techniques are artificial neural networks (ANNs), genetic programming (GP), evolutionary polynomial regression (EPR), support vector machines (SVM), M5 model trees, and k-nearest neighbors (Elshorbagy et al., 2010). Of these, ANNs are by far the most commonly used AI technique in geotechnical engineering. More recently, GP and EPR have been frequently used in geotechnical engineering and have proved to be successful. The main focus of the current paper is on the use of ANNs, GP, and EPR in pile foundations.

The behavior of pile foundations in soils is complex, uncertain, and not yet entirely understood. This fact has encouraged many researchers to apply the AI techniques for prediction and modelling of the behavior of pile foundations, including the ultimate bearing capacity, settlement estimation, and load-settlement response. The objective of this paper is to provide an overview of the salient features relevant to the process and operation of ANNs, GP, and EPR, and to present a review of their applications to date in pile foundations. The paper also discusses most of the current challenges as well as future directions in relation to the use of AI techniques in geotechnical engineering prediction and modelling.

2. Overview of artificial intelligence

Artificial intelligence (AI) is a computational method that attempts to mimic, in a very simplistic way, the human cognition capability so as to solve engineering problems that have defied solution using conventional computational techniques (Flood, 2008). The

essence of AI techniques in solving any engineering problem is to learn by examples of data inputs and outputs presented to them so that the subtle functional relationships among the data are captured, even if the underlying relationships are unknown or the physical meaning is difficult to explain. Thus, AI models are data-driven models that rely on the data alone to determine the structure and parameters that govern a phenomenon (or system), with less assumptions about the physical behavior of the system. This is in contrast to most physically-based models that use the first principles (e.g., physical laws) to derive the underlying relationships of the system, which usually justifiably simplified with many assumptions and require prior knowledge about the nature of the relationships among the data. This is one of the main benefits of AI techniques when compared to most physically-based empirical and statistical methods.

The AI modeling philosophy in attempting to capture the relationship between a historical set of model inputs and the corresponding outputs is similar to a number of conventional statistical models. For example, imagine a set of x-values and corresponding y-values in two-dimensional space, where y = fx). The objective is to find the unknown function f that relates the input variable x to the output variable y. In a linear regression

statistical model, the function f can be obtained by changing the slope tan^ and intercept

/ of the straight line in Fig. 1a, so that the error between the actual outputs and the outputs of the straight line is minimized. The same principle is used in AI models. Artificial intelligence can form the simple linear regression model by having one input and one output (Fig. 1b). Artificial intelligence uses available data to map between the system inputs and the corresponding outputs using machine learning by repeatedly presenting examples of the model inputs and outputs (training) in order to find the

function y = fx) that minimizes the error between the historical (actual) outputs and the outputs predicted by the AI model.

If the relationship between x and y is non-linear, statistical regression analysis can be applied successfully only if prior knowledge of the nature of the non-linearity exists. On the contrary, this prior knowledge of the nature of the non-linearity is not required for AI models. In the real world, it is likely that complex and highly non-linear problems are encountered, and in such situations, traditional regression analyses are inadequate (Gardner and Dorling, 1998). In this section, a brief overview of three selected AI techniques (i.e., ANNs, GP, and EPR) is presented below.

2.1 Artificial neural networks

Artificial neural networks (ANNs) are a form of AI that attempt to mimic the function of the human brain and nervous system. Although the concept of ANNs was first introduced in 1943 (McCulloch and Pitts, 1943), research into applications of ANNs has blossomed since the introduction of the back-propagation training algorithm for feedforward multi-layer perceptrons (MLPs) in 1986 (Rumelhart et al., 1986). Many authors have described the structure and operation of ANNs (e.g., Zurada 1992; Fausett 1994). Typically, the architecture of ANNs consists of a series of processing elements (PEs), or nodes, that are usually arranged in layers: an input layer, an output layer, and one or more hidden layers, as shown in Fig. 2.

The input from each PE in the previous layer xi is multiplied by an adjustable connection weight w;i. At each PE, the weighted input signals are summed and a threshold value q is added. This combined input Ij is then passed through a non-linear transfer functionf(.) to produce the output of the PE y. The output of one PE provides the

input to the PEs in the next layer. This process is summarized in Equations (1) and (2), and illustrated in Fig. 2.

The propagation of information in an ANN starts at the input layer, where the input data are presented. The network adjusts its weights on the presentation of a training data set and uses a learning rule to find a set of weights that produces the input/output mapping that has the smallest possible error. This process is called learning or training. Once the training of the model has successfully accomplished, the performance of the trained model needs to be validated using an independent validation set. The main steps involved in the development of an ANN, as suggested by Maier and Dandy (2000), are illustrated in Fig. 3 and discussed in some depth in Shahin (2013).

2.2 Genetic programming

Genetic programming (GP) is an extension of genetic algorithms (GA), which are evolutionary computing search (optimization) methods that are based on the principles of genetics and natural selection. In GA, some of the natural evolutionary mechanisms, such as reproduction, cross-over, and mutation, are usually implemented to solve function identification problems. GA was first introduced by Holland (1975) and developed by Goldberg (1989), whereas GP was invented by Cramer (1985) and further developed by Koza (1992). The difference between GA and GP is that GA is generally used to evolve

I j = £ w H x, + 0 j summation

J Jj ' J

yj = f (Ij) transfer

the best values for a given set of model parameters (i.e., parameters optimization), whereas GP generates a structured representation for a set of input variables and corresponding outputs (i.e., modeling or programming).

Genetic programming manipulates and optimizes a population of computer models (or programs) proposed to solve a particular problem, so that the model that best fits the problem is obtained. A detailed description of GP can be found in many publications (e.g., Koza, 1992), and a brief overview is given herein. The modelling steps by GP start with the creation of an initial population of computer models (also called chromosomes) that are composed of two sets (i.e., a set of functions and a set of terminals) that are defined by the user to suit a certain problem. The functions and terminals are selected randomly and arranged in a tree-like structure to form a computer model that contains a root node, branches of functional nodes, and terminals, as shown by the typical example of GP tree representation in Fig. 4. The functions can contain basic mathematical operators (e.g., +, -, x, /), Boolean logic functions (e.g., AND, OR, NOT), trigonometric functions (e.g., sin, cos), or any other user-defined functions. The terminals, on the other hand, may consist of numerical constants, logical constants, or variables.

Once a population of computer models has been created, each model is executed using available data for the problem at hand, and the model fitness is evaluated depending on how well it is able to solve the problem. For many problems, the model fitness is measured by the error between the output provided by the model and the desired actual output. One could measure the fitness fi of an individual chromosome i using the following expression:

where M is the range of selection, C(i.j) is the value returned by the individual chromosome i for fitness case j (out of Ct fitness cases), and Tj is the target value for the fitness case J. There are, of course, other fitness functions available that can be appropriate for different problems. If the desired results (according to the measured errors) are satisfactory, the GP process is stopped, otherwise, a generation of new population of computer models is then created to replace the existing population, and the process is repeated for a certain number of generation or until the desired fitness score is obtained. The new population is created by applying the following three main operations: reproduction, cross-over, and mutation. These three operations are applied on certain proportions of the computer models in the existing population, and the models are selected according to their fitness. Reproduction is copying a computer model from an existing population into the new population without alteration. Cross-over is genetically recombining (swapping) randomly chosen parts of two computer models. Mutation is replacing a randomly selected functional or terminal node with another node from the same function or terminal set, provided that a functional node replaces a functional node and a terminal node replaces a terminal node. The evolutionary process of evaluating the fitness of an existing population and producing new population is continued until a termination criterion is met, which can be either a particular acceptable error or a certain maximum number of generations. The best computer model that appears in any generation designates the result of the GP process.

2.3 Evolutionary polynomial regression

Evolutionary polynomial regression (EPR) is a hybrid regression technique based on evolutionary computing that was developed by Giustolisi and Savic (2006). It constructs

symbolic models by integrating the soundest features of numerical regression, with genetic programming and symbolic regression (Koza, 1992). The following two steps roughly describe the underlying features of the EPR technique, aimed to search for polynomial structures representing a system. In the first step, the selection of exponents for polynomial expressions is carried out, employing an evolutionary searching strategy by means of GA (Goldberg, 1989). In the second step, numerical regression using the least square method is conducted, aiming to compute the coefficients of the previously selected polynomial terms. The general form of expression in EPR can be presented as follows (Giustolisi and Savic, 2006):

= ¿F(X, f (X),a])+ Qo

where y is the estimated vector of output of the process, m is the number of terms of the target expression, F is a function constructed by the process, X is the matrix of input variables, f is a function defined by the user, and aj is a constant. A typical example of EPR pseudo-polynomial expression that belongs to the class of Equation (4) is as follows (Giustolisi and Savic, 2006):

7 = ao + £aj . (X1)^(J1 ...(Xk)ES(jk).f [(X1 )ES(jk+1).„(Xk)(j,2k)

where Y is the vector of target values, m is the length of the expression, aj is the value of the constants, Xi is the vector(s) of the k candidate inputs, ES is the matrix of exponents, and f is a function selected by the user.

Evolutionary polynomial regression is suitable for modeling physical phenomena, based on two features (Savic et al. 2006): (i) the introduction of prior knowledge about the physical system/process, to be modeled at three different times, namely before, during, and after EPR modelling calibration; and (ii) the production of symbolic formulas, enabling data mining to discover patterns that describe the desired parameters. In the first EPR feature (i) above, before the construction of the EPR model, the modeler selects the relevant inputs and arranges them in a suitable format according to their physical meaning. During the EPR model construction, model structures are determined by following user-defined settings such as general polynomial structure, user-defined function types (e.g., natural logarithms, exponentials, tangential hyperbolics), and searching strategy parameters. The EPR starts from true polynomials and also allows for the development of non-polynomial expressions containing user-defined functions (e.g., natural logarithms). After EPR model calibration, an optimum model can be selected from among the series of models returned. The optimum model is selected based on the modeller's judgement, in addition to statistical performance indicators such as the coefficient of determination. A typical flow diagram of the EPR procedure is shown in Fig. 5, and a detailed description of the technique can be found in Giustolisi and Savic (2006).

3. Artificial intelligence applications in pile foundations

This section provides an overview of the applications of three selected AI techniques, including ANNs, GP, and EPR, that have appeared to date in relation to examining the relative success or otherwise of AI in pile foundations. It should be noted that it is not intended in the current paper to cover every single application or scientific paper of the three selected AI techniques in pile foundations that can be found in the literature but rather the intention is to provide a general overview of some of the more relevant applications in engineering problem of pile foundations. Some works are selected to be described in some detail, while others are acknowledged for reference purposes. On the other hand, the applications of the three selected AI techniques in geotechnical engineering are beyond the scope of the current paper and can be found elsewhere. Interested readers are referred to Shahin et al. (2001), where the pre-2001 ANN applications in geotechnical engineering are reviewed in some detail, and Shahin et al. (2009) and Shahin (2013), where the post-2001 papers of ANN applications in geotechnical engineering are briefly examined. Interested readers are also referred to Shahin (2013), where applications of GP and EPR in geotechnical engineering are presented.

Based on the author's experience, there are several factors in the use of AI techniques that need to be systematically investigated when developing AI models for geotechnical engineering problems, including pile foundations, so that model performance can be improved. These factors include the determination of adequate model inputs, data division, data preparation, model validation, model robustness, model transparency, knowledge extraction, and model uncertainty. Some of these factors have received recent attention, whereas others require further research. Discussion of these factors are beyond

the scope of this paper but can be found in Shahin (2013). Some of these factors are briefly discussed in the applications presented below.

3.1 Bearing capacity prediction

The design of foundations is generally controlled by two major criteria, i.e., bearing capacity and settlement. For pile foundations, prediction of the load carrying capacity is often being the governing factor; hence, has been examined by several AI researchers especially using ANNs. For example, Goh (1994, 1995b) presented a neural network model to predict the friction capacity of piles in clays and the model was trained with field data of actual case records. The considered model inputs were the pile length, pile diameter, mean effective stress, and undrained shear strength. The skin friction resistance was the only model output. The results obtained from the neural network model were compared with those calculated using the method proposed by Semple and Rigden (1986) as well as the / method developed by Burland (1973), as shown in Table 1. The performance measures used were the coefficient of correlation, r, and error rate between the predicted versus measured bearing capacities. It is evident from Table 1 that the ANN model outperforms the conventional methods. Goh (1995a, 1996), soon after, developed another neural network model to estimate the ultimate load capacity of driven piles in cohesionless soils. In this study, the data used were derived from the results of load testing carried out on piles made of timber, precast concrete, and steel, driven into sandy soils. The inputs to the ANN model that found to be more significant were the hammer weight, drop and type, and pile length, weight, cross sectional area, set and modulus of elasticity. The model output was the pile load capacity. When the model was examined using a testing set, it was observed that the model successfully predicted the pile load

capacity. By examining the connection weights, it was observed that the more important input factors are the pile set as well as the hammer weight and type. The study compared the results of the ANN model with the following common formulae: Engineering News formula (Wellington, 1892), Hiley method (Hiley, 1922), and Janbu method (Janbu, 1953). Table 2 summarises the results, which indicate that ANN predictions of the load carrying capacity of driven piles are significantly better than those obtained from the traditional methods. More recently, Goh et al. (2005) used a Bayesian neural network algorithm to model the relationship between the soil undrained shear strength, effective overburden pressure, and undrained side resistance alpha factor for drilled shafts (bored piles). The advantage of using the Bayesian ANN approach is that instead of just giving a single prediction as in conventional back-propagation ANN, it produces a probability distribution over the predicted value. The benefit of this distribution is that it provides information on the characteristic error of the prediction that arises from the uncertainty associated with interpolating noisy data. It also allows assessment of the confidence associated with any prediction. The model was trained using a database that contained 127 field load tests on drilled shafts in a variety of cohesive soil profiles. Comparison was made between the ANN predictions and those obtained from the method proposed by Chen and Kulhawy (1994). The comparison indicated that the ANN model was reasonably accurate in its predictions and achieved an improvement over those calculated using the method of Chen and Kulhawy (1994), especially in the training set.

Among the available methods for predicting the axial capacity of pile foundations that have been shown to give better predictions in many situations, are the cone penetration test (CPT)-based models. This can be attributed to the fact that CPT-based methods have been developed in accordance with the CPT results, which have been

found to yield reliable soil properties; hence, more accurate axial pile capacity predictions. In an attempt to develop more well-established CPT-based pile capacity prediction models that provide more accurate axial capacity predictions, Shahin (2010) developed ANN models for driven piles and drilled shafts using a series of in-situ load tests, as well as CTP results. The data were collected form the literature and comprised 80 driven pile and 94 drilled-shaft load tests. The predictive ability of the ANN models was examined by comparing their predictions with those obtained from the most commonly used CPT-based pile capacity prediction methods. For driven piles, the ANN model was compared with the European method (de Ruiter and Beringen, 1979), Laboratoire Central des Ponts et Chaussees (LCPC) method (Bustamante and Gianeselli, 1982), and the method by Eslami and Fellenius (1997). For drilled shafts, the ANN model was compared with the Schmertmann method (Schmertmann, 1978), LCPC method (Bustamante and Gianeselli, 1982), and Alsamman method (Alsamman, 1995). The comparison was carried out analytically using the rank index, RI, proposed by Abu-Farsakh and Titi (2004), which comprises of four combined statistical performance criteria. Sensitivity analyses were also carried out on the ANN models to explore their generalization ability (robustness). The results indicated that the ANN models were capable of accurately predicting the ultimate capacity of pile foundations with high level of performance. The RI results yielded the following overall rank: ANN model (Shahin, 2010), Eslami and Fellenius (1997), LCPC method (Bustamante and Gianeselli, 1982), and European method (de Ruiter and Beringen, 1979). On the other hand, for drilled shafts, the results of RI showed an equal overall rank for the ANN model (Shahin, 2010) and the method proposed by Alsamman (1995), followed by the Schmertmann method (Schmertmann, 1978) and the LCPC method (Bustamante and Gianeselli, 1982). The sensitivity analyses

indicated that predictions from the ANN models compare well with what one would expect based on available geotechnical knowledge and underlying physical meaning, as well as experimental results.

In an attempt to facilitate the use of the obtained ANN models and to make them more accessible, Shahin (2010) translated the connection weights and biases of the developed neural network models into tractable and relatively simple formula suitable for hand calculations. The derived formula can be used to calculate the ultimate bearing capacity of driven piles, Qu (kN), as follows (Shahin, 2010):

(lANN = 290 +

x^-u (driven. piles)

(-1.699-4.193tanh H1 +2.242tanh H 2)

H1 and H2 are two parameters obtained for steel piles, as follows:

H1 =-5.1 +10-3 (3.59 Deq + 45.51Lp + 112.23^ - 21.39qc_shaft + 6.86 fs) (7)

H2 = 1.164 -10-3(2.47Deq + 33.96Lp - 8.37qc_tip + 1.58qc_shaft - 0.24fS) (8)

where Deq (mm) is the equivalent pile diameter, Lp (m) is the pile embedment length, qc-tip (MPa) is the weighted average cone point resistance over pile tip failure zone;

qc-shafi (MPa) is the weighted average cone point resistance along pile embedment length,

and fs (kPa) is the weighted average sleeve friction along pile embedment length.

Alternatively, for concrete piles:

H1 =-5.158 +10"3(3.59D + 45.5L + 112.23qc.tip - 21.39^ + 6.86f,) (9)

H2 = 0.816-10-3(2.47Deq + 33.96Lp -8.37qc_tip + 1.58qc-shaft -0.24f,)

On the other hand, the ultimate drilled shafts capacity, Qu (kN), can be calculated as follows:

Qu(drilled.shafts) = 355.8 +

9296.3

(-1.673-3.364tanh H1 +4.223tanh H2 +3.336tanh H3)

H1 =-6.509 + 10~3(1.069Dstem + 2.351A_ -41.152L,-2.174qc-base +11.271*^) (12) H 2 = 0.528 +10 -3(0.553D,tem + DbaSe + 38.75L,+1.59^ + 5.344qc-^) (13)

H3 = 3.777 +10-3(0.772D,tem -0.537Dbfl,e + 83.37L, +23.31^ + 56.23q^fft) (14)

where Dstem (mm) is the shaft stem diameter, Dbase (mm) is the shaft base diameter, L (m) is the shaft embedment length, qc_base (MPa) is the weighted average cone point resistance over shaft base failure zone, and qc-shifi (MPa) is the weighted average cone tip resistance

along shaft embedment length.

Shahin and Jaksa (2005, 2006) assessed the applicability of ANNs for predicting the pull-out capacity of marquee ground anchors (these are, in effect, micro-piles) using

multi-layer perceptrons (MLPs) and B-spline neurofuzzy networks. Neurofuzzy networks are a type of ANN modeling technique that combines the explicit linguistic knowledge representation of fuzzy systems with the learning power of neural networks (Brown and Harris 1995). Neurofuzzy networks can be trained by processing data samples to perform input/output mappings, similar to the way traditional neural networks do, with the additional benefit of being able to provide a set of production If-then linguistic fuzzy rules that describe the model input/output relationships in a transparent way, such as:

IF (x1 is high AND x2 is low) THEN (y is high), c = 0.9 (15)

where x1 and x2 are input variables, y is the corresponding output variable, and (c = 0.9) is the rule confidence which indicates the degree to which the above rule has contributed to the output. Both the MLP and B-spline neurofuzzy models were trained using five inputs including the anchor diameter, anchor embedment length, average cone tip resistance from the cone penetration test along the anchor embedment length, average cone sleeve friction along the embedment length, and installation technique. The single model output was the ultimate anchor pull-out capacity. The results obtained were also compared with those obtained from three of the most commonly used traditional methods, namely, the LCPC method proposed by Bustamante and Gianeselli (1982), and the methods proposed by Das (1995) and Bowles (1997). The results indicated that the MLP and B-spline models were able to predict well the pull-out capacity of marquee ground anchors and significantly outperform the traditional methods. Over the full range of pull-out capacity prediction, the coefficients of correlation, r, using the MLP and B-spline models were

0.83 and 0.84, respectively. In contrast, these measures ranged from 0.46 to 0.69 when the other methods were used.

To predict the pile capacity from dynamic testing data, Chan et al. (1995) developed a neural network model as an alternative to the commonly used pile driving formula approach. The neural network model was trained with the same input parameters listed in the simplified Hiley formula (Broms and Lim, 1988), including the elastic compression of pile and soil, pile set, and driving energy delivered to the pile. The model output considered was, again, the pile capacity. The root mean squared percentage error of the neural network model was found to be 13.5 and 12.0% for the training and testing sets, respectively, compared with 15.7% in both the training and testing sets for the simplified Hiley formula.

Lee and Lee (1996) utilized neural networks to predict the ultimate bearing capacity of piles using data obtained from a calibration chamber model pile load tests as well as results of in-situ pile load tests. For the simulation using the model pile load test data, the neural network model inputs were the penetration depth ratio (i.e., penetration depth of pile/pile diameter), mean normal stress of the calibration chamber, and number of blows. The ultimate bearing capacity was the model output. The prediction of the neural network model showed a maximum error not greater than 20% and an average summed square error of less than 15%. For simulation using the in-situ pile load test data, five input variables were used representing the penetration depth ratio, average standard penetration number along the pile shaft, average standard penetration number near the pile tip, pile set, and hammer energy. The data were arbitrarily partitioned into two parts, odd and even numbered sets and two neural network models were developed. The results of these models were compared with Meyerhofs equation (Meyerhof, 1976), based on the

average standard penetration value. The results of the estimated versus measured pile bearing capacities obtained from the neural network models and Meyerhofs equation showed that the predicted values from the neural networks matched the measured values much better than those obtained from Meyerhofs equation.

Abu-Kiefa (1998) introduced three ANN models (referred to in his paper as GRNNM1, GRNNM2, and GRNNM3) to predict the capacity of driven piles in cohesionless soils. The first model was developed to estimate the total pile capacity, whereas the second and third models were employed to estimate the pile tip and shaft capacities. In the first model, five variables were selected to be the model inputs including the angle of shear resistance for the soil surrounding the pile shaft, angle of shear resistance of soil at the tip of the pile, effective overburden pressure at the tip of the pile, pile length, and equivalent pile cross-sectional area. The model, again, had one output representing the total pile capacity. In the model used to evaluate the pile tip capacity, the above variables were also used. The number of input variables used to predict the pile shaft capacity was four, representing the average standard penetration number around the pile shaft, angle of shear resistance around the pile shaft, pile length, and pile diameter. The results of the neural network models obtained in this study were compared with four other empirical techniques including those proposed by Meyerhof (1976), Coyle and Castello (1981), American Petroleum Institute (1984), and Randolph (1985). The results of the total pile capacity prediction demonstrated high coefficients of determination (R = 0.95) for all data records obtained from the neural network models, while those for the other methods ranged between 0.52 and 0.63.

Teh et al. (1997) proposed a neural network model for estimating the static pile capacity determined from dynamic stress-wave data for precast reinforced concrete piles

with a square section. The neural network model was trained to associate the input stress-wave data with capacities derived from the CAPWAP technique (Rausche et al. 1972). The study was concerned with predicting the 'CAPWAP predicted capacity' rather than the true bearing capacity of piles. The neural network model learned the training data set almost perfectly for predicting the static total pile capacity with a root mean square error of less than 0.0003. The trained neural network model was assessed for its ability to generalize by means of a testing data set. Good prediction was obtained for seven out of ten piles. Another application of ANNs includes the prediction of axial and lateral load capacity of steel H-piles, steel piles and pre-stressed and reinforced concrete piles by Nawari et al. (1999). In this application, ANNs were found to be an accurate technique for the design of pile foundations. Prediction of the undrained lateral load pile capacity of piles in clay was modelled using ANNs by Das and Basudhar (2006), and a model equation based on the produced neural network parameters was developed.

Other ANN applications in pile foundations include predicting the total pile capacity by generalized regression neural networks developed using stress-wave data (Pal and Deswal 2010), modelling pile shaft capacity from CPT and CPTU data by polynomial neural networks (Ardalan et al., 2009), predicting the total resistance of driven piles as well as the resistance at the tip and along the shaft using dynamic load tests (Park and Cho, 2010), predicting pile setup for three pile types (pipe, concrete, and H-pile) using dynamic load tests (Tarawneh, 2013; Tarawneh and Imam, 2014), and analysing mechanism of time effect and soil consolidation on vertical ultimate bearing capacity of preformed concrete piles (Tian et al., 2010).

The application of GP technique in estimating the capacity of pile foundations is relatively recent. Alkroosh and Nikraz (2011a, 2012) developed GP correlation models

for predicting the relationship between pile axial capacity and CPT data. The GP models were developed for bored piles as well as driven piles (a model for each of concrete and steel piles). The performance of the GP models was evaluated by comparing their results with experimental data as well as the results of a number of currently used CPT-based methods. The results indicated the potential ability of GP models in predicting the bearing capacity of pile foundations and outperformance of the developed models over existing methods. More recently, Alkroosh and Nikraz (2014) developed a GP model that correlates the pile capacity with the dynamic input and SPT data. The performance of the GP model was assessed by comparing its predictions with those calculated using two commonly used traditional methods and an ANN model. It was found that the GP model performed well with coefficients of determination of 0.94 and 0.96 in the training and testing sets, respectively. The results of comparison with other available methods showed that the GP model predicted the pile capacity more accurately than the existing traditional methods and ANN model. Another successful application of Genetic programming in pile capacity prediction was carried out by Gandomi and Alavi (2012) for the assessment of the undrained lateral load capacity of driven piles and undrained side resistance alpha factor of drilled shafts.

The application of EPR in predicting the capacity of pile foundations is very recent. Using the same database of Shahin (2010) and similar model inputs and outputs, Shahin (2014c) developed successful EPR models for predicting the axial capacity, Qu (kN), of driven piles and drilled shafts. The formulations of the developed EPR models are as follows (Shahin, 2014c):

For driven (steel) piles:

QZlurlven) =-2.77 Dq^ + 0.096DL +1.714x 10-4D2qc_ttp4Z

^c-shaftfs—shaft

c-shaftfs-

- 6.279 x10-9 D2qc_tipfs_tip + 243.39 (16)

Alternatively, for driven (concrete) piles:

QZncrete,nven) =-2.777 DLq-'l + 0.096DL +1.714 x10-4 D

A/ qc-shaftfs-shaft

- 6.279x 10-9D2L^qc-tipfs-tip + 486.78 (17)

For drilled shafts:

QSRuf = 0.6878L2ff + 1.581X 10-4 B2 ff +1.294 x 10-4 LqlpJD

+ 7.8 x10-5 Dqc_shJs-shafif~P (18)

where D (mm) is the pile perimeter/n (for driven piles) or pile stem diameter (for drilled shafts), L (m) is the pile embedment length, B (mm) is the drilled shaft base diameter, qc (MPa) is the weighted average cone point resistance over pile tip failure zone,

fs-ip (kPa) is the weighted average cone sleeve friction over pile tip failure zone, qc_shaft (MPa) is the weighted average cone point resistance over pile embedment length,

and fs_shahft (kPa) is the weighted average cone sleeve friction over pile embedment

length. The above EPR models represented by equations we compared with the traditional methods and were found to outperform most available methods.

Finally, using the same database of Shahin and Jaksa (2005, 2006) and similar model inputs and outputs, Shahin (2014c) developed successful EPR models for predicting the ultimate pull-out capacity of marquee anchors, Qu (kN), that yielded the following two formula, for static and dynamics installation, respectively:

QELc, =-0.376^ _ 6.727 X 10_9 LqC f2 + 5.357 X 10_5 Lpqf + 0.75 (19)

QEL^ =_0.37^V2qC _ 6.727 X 10_9 LqlfS + 5.357 X 10_5 L^DJS + 0.75 (20)

where D (mm) is the equivalent anchor parameter (= anchor perimeter/n), L (m) is the anchor embedment length, qc (MPa) is the arithmetic average cone tip resistance along the embedment length, and fs (kPa) is the arithmetic average sleeve friction along the

embedment length. The performance of the EPR models in the training and validation sets is given in Table 3, and the comparison of model performance in the validation set with the other available methods in given in Table 4. The methods used for comparison include the ANN model developed by Shahin and Jaksa (2005), LCPC method (Bustamante and Gianeselli, 1982), Das method (Das, 1995) and Bowles method (Bowles, 1997). The performance of the EPR models and comparison with other methods were evaluated using five different analytical standard measures including the coefficient of correlation, r, the coefficient of determination, R , root mean squared error, RMSE, mean

absolute error, MAE, and ratio of average measured to predicted outputs, i. It can be seen in Table 3 that the EPR models perform well in the training and validation sets, and that the EPR models outperform the other available methods including the ANN model.

3.2 Settlement estimation

Settlement is one of the two criteria that govern the design of pile foundations as settlement needs to be checked to ensure that it does not to exceed certain limits. However, settlement of pile foundations is less significant compared to bearing capacity and thus received less attention from the AI researchers. The number of AI publications for settlement prediction is significantly less than those of bearing capacity and solely related to the use of artificial neural networks (no applications are currently available for the use of either GP or EPR in settlement prediction of pile foundations). For example, Goh (1994) developed a neural network for the prediction of settlement of a vertically loaded pile foundation in a homogeneous soil stratum. The input variables for the neural network consisted of the ratio of the elastic modulus of the pile to the shear modulus of the soil, pile length, pile load, shear modulus of the soil, Poisson's ratio of the soil, and radius of the pile. The output variable was the pile settlement. The desired output that was used for the neural network model training was obtained by means of finite element and integral equation analyses developed by Randolph and Wroth (1978). A comparison of the theoretical and predicted settlements for the training and testing sets is given in Figure 6. The results show that the neural network was able to model successfully the settlement of pile foundations.

Nawari et al. (1999) developed neural network models to predict the deflection of drilled shafts based on the standard penetration test (SPT) data and the shaft geometry.

The developed models involved back-propagation as well as generalized regression neural networks. Prediction results from the developed neural network models were compared with the classical technique, namely the p-y method, after Reese et al. (2006). The deviation of prediction of deflection with depth at a specific load level from the measured deflections, in case of the back-propagation neural network model, was found to be between 9-15%. On the other hand, the generalized neural network model gave prediction of good approximation and the deflection with depth was found to correlate very well with the predicted values with variation within 10%. The results also indicated that the neural network models correlate closer to the measured values than the p-y solution.

More recently, Nejad et al. (2009) developed neural network a model for predicting pile settlement also based on SPT data. Approximately 1000 data sets, obtained from the published literature, were used for model development. Model predictions were also compared with those obtained from a number of traditional methods; namely those of Vesic (1977), Poulos and Davis (1980), Das (1995), and the non-linear t-z method of Reese et al. (2006). The results indicated that the neural network model has the ability to predict the settlement of pile with an acceptable degree of accuracy of correlation coefficient r = 0.972 for settlement up to 185 mm. sensitivity analyses carried out on the developed model indicated that the applied load, embedded length of pile, and soil properties, in this case the SPT-N values, have the most significant effect on the predicted settlement. It was also demonstrated that the neural network model outperforms the traditional methods and provides more accurate pile settlement predictions.

3.3 Load-settlement response modeling

As mentioned earlier, the design of pile foundations requires good estimation of the pile load-carrying capacity and settlement. Design for bearing capacity and design for settlement have been traditionally carried out separately. However, soil resistance and settlement are influenced by each other, and the design of pile foundations should thus consider the bearing capacity and settlement inseparably. This requires the full load-settlement response of piles to be well predicted. However, it is well known that the actual load-settlement response of pile foundations can be obtained only by load tests carried out in situ, which are expensive and time-consuming. Consequently, some AI researchers have made attempts to develop AI prediction models that can resemble the full load-settlement response of piles. However, all attempts have used ANNs and no attempts are currently available that use either GP or EPR.

Shahin (2014a,b) used recurrent neural networks (RNN) to develop prediction models for the full load-settlement response of drilled shafts and steel driven piles, subjected to axial loading. The developed RNN models were calibrated and validated using several in-situ full-scale pile load tests, as well as cone penetration test (CPT) data. The tests were conducted on sites of different soil types and geotechnical conditions, ranging from cohesive clays to cohesionless sands including layered soils. Six factors affecting the capacity of piles were considered as potential model input variables. These factors include the pile diameter, pile embedment length, weighted average cone point resistance over pile tip failure zone, weighted average friction ratio over pile tip failure zone, weighted average cone point resistance over pile embedment length, and weighted average friction ratio over pile embedment length. Three other input variables are also considered to represent the current state of stress/strain including the normalized axial

settlement (= pile settlement/pile diameter), increment of axial settlement, and pile load. The single model output variable is the pile load at the next state of loading. The models yielded high level of correlation between the measured and predicted data, and the graphical performance of the models in the training and validations sets are shown in Figs. 7 and 8. It can be seen that excellent agreement between the actual pile load tests and the RNN models' predictions are obtained for both the drilled shafts and driven piles. The nonlinear relationships of the load-settlement response are well predicted, and the results demonstrate that the RNN models have a strong capability to simulate the behaviour of pile foundations quite well.

Ismail and Jeng (2011) developed a high-order neural (HON) network to simulate the pile load-settlement curves using properties of the pile and SPT data along the depth of pile embedment as inputs. HON networks use polynomial functions to map inputs into output and can be trained through error back-propagation (BP) algorithm. As discussed by Ismail and Jeng (2011), the main advantage of HON networks over traditional BPN networks is that BPN networks use the sigmoid transfer function which is bi-asymptotic and becomes insensitive to the variation of inputs as it approaches either 1 or 0. This may limit the ability of BPN networks to make reasonable extrapolations outside the extreme values of inputs and outputs used in model training. On the contrary, HON networks use non-asymptotic processing elements (i.e., high-order neurons) to overcome such a problem. The input data used for the HON network consisted of the average value of SPT along the pile shaft, the SPT value at the pile base, the pile stiffness, the shaft and base area, and the pile load. Other parameters used include soil type and installation method. Based on the coefficient of determination and root mean squared error, as well as the quality of load-settlement curves, a significant improvement was observed from the

comparison of HON model results with BPN, elastic and hyperbolic models. Also, the HON model was found to respond reasonably well to various input parameters in a manner consistent with the anticipated behaviour of axially loaded piles.

Ismail et al. (2013), soon after, developed a new load-deformation model for axially loaded piles by coupling the particle swarm optimisation (PSO) (Eberhart and Kennedy 1995) and back-propagation (BP) algorithms for model training. The results showed that the proposed PSO-BP hybrid model simulates the load-deformation curves of axially loaded piles more accurately than previous HON model. The PSO-BP model also turned out to be more accurate than traditional hyperbolic and t-z models.

Alkroosh and Nikraz (2011b) also developed artificial neural network (ANN) models for simulating the load-settlement behavior of pile foundations embedded in sand or mixed soils, subjected to axial loads. Three ANN models were developed, a model for bored piles and two models for driven piles (a model for each of concrete and steel piles). The data used for development of the ANN models comprised a series of in-situ pil load tests as well as cone penetration test (CPT) results. Predictions from the ANN models were comrade with the results of experimental data and with predictions of number of currently adopted load-transfer methods. The results indicated that the ANN models perform well and able to predict the pile-settlement behavior accurately.

4. Discussion and conclusions

In geotechnical engineering, it is most likely to encounter problems that are very complex and not well understood. In this regard, artificial intelligence (AI) provides several advantages over more traditional computing techniques. For most traditional mathematical models, the lack of physical understanding is usually supplemented by

either simplifying the problem or incorporating several assumptions into the models. Mathematical models also rely on assuming the structure of the model in advance, which may be sub-optimal. Consequently, many mathematical models fail to simulate the complex behavior of most geotechnical engineering problems. In contrast, AI techniques are data-driven approaches in which the model development is based on training of input-output data pairs to determine the structure and parameters of the model. In this case, there is less need to either simplify the problem or incorporate assumptions. Moreover, AI models can always be updated to obtain better results by presenting new training examples as new data become available. These factors combine to make AI a powerful modelling tool in geotechnical engineering.

It was evident from the review presented in this paper that AI techniques have been applied successfully to behavior of pile foundations including bearing capacity prediction, settlement estimation, and modeling of load-settlement response. However, most available applications focused on bearing capacity prediction and settlement estimation received less attention, which can be attributed to the fact that settlement of pile foundations is less significant than bearing capacity. In most reviewed AI applications in pile foundations, it was possible to provide simple formulations suitable for hand calculations for the relationships between the model inputs and the corresponding outputs. This helps to facilitate the use of the developed AI models and to make them accessible to the users. Based on the results of the reviewed applications, it can be concluded that AI techniques perform better than, or at least as good as, the most traditional methods.

Despite the success of AI techniques, they are still facing classical opposition due to some inherent shortcomings that need further attention in the future including the lack of

transparency, knowledge extraction, and model uncertainty. Detailed discussion of such shortcomings is beyond the scope of this paper but have been presented in detail by Shahin (2013). For example, special attention should be paid to incorporating prior knowledge about the underlying physical process based on engineering judgment or human expertise into the learning formulation. Improvements in such issues will greatly enhance the usefulness of AI techniques and will provide the next generation of applied AI models with the best way for advancing the field to the next level of sophistication and application. The author suggests that AI techniques for the time being might be treated as a complement to conventional computing techniques rather than as an alternative, or may be used as a quick check on solutions developed by more time-consuming and in-depth analyses.

References

Abu-Farsakh, M. Y., and Titi, H. H. (2004). "Assessment of direct cone penetration test methods for predicting the ultimate capacity of friction driven piles." Journal of Geotechnical and Geoenvironmental Engineering, 130(9), 935-944.

Abu-Kiefa, M. A. (1998). "General regression neural networks for driven piles in cohesionless soils." Journal of Geotechnical & Geoenvironmental Engineering, 124(12), 1177-1185.

Alkroosh, I., and Nikraz, H. (2011a). "Correlation of pile axial capacity and CPT data using gene expression programming." Geotechnical and Geological Engineering, 29, 725-748.

Alkroosh, I., and Nikraz, H. (2011b). "Simulating pile load-settlement behavior from CPT data using intelligent computing." Central European Journal of Engineering, 1(3), 295-305.

Alkroosh, I., and Nikraz, H. (2012). "Predicting axial capacity of driven piles in cohesive soils using intelligent computing." Engineering Applications of Artificial Intelligence, 25(3), 618-627.

Alkroosh, I., and Nikraz, H. (2014). "Predicting pile dynamic capacity via application of an evolutionary algorithm." Soils and Foundations, 54(2), 233-242.

Alsamman, O. M. (1995). "The use of CPT for calculating axial capacity of drilled shafts."PhD Thesis, University of Illinois-Champaign, Urbana, Illinois.

American Petroleum, I. (1984). RP2A: Recommended practice for planning, designing and constructing fixed offshore platforms, Washington, DC.

Ardalan, H., Eslami, A., and Nariman-Zadeh, N. (2009). "Piles shaft capacity from CPT and CPTU data by polynomial neural networks and genetic algorithms." Computers and Geotechnics, 36(4), 616-625.

Bowles, J. E. (1997). Foundation analysis and design, McGraw-Hill, New York.

Broms, B. B., and Lim, P. C. (1988). "A simple pile driving formula based on stress-wave measurements." Proc., Proceedings of the 3rd International Conference on the Application of Stress-Wave Theory to Piles, BiTech Publishers, Vancouver, 591-600.

Brown, M., and Harris, C. J. (1995). "A perspective and critique of adaptive neurofuzzy systems used for modelling and control applications." International Journal of Neural Systems, 6(2), 197-220.

Burland, J. B. (1973). "Shaft friction of piles in clay." Ground Engineering, 6(3), 1-15.

Bustamante, M., and Gianeselli, L. (1982). "Pile bearing capacity prediction by means of static penetrometer CPT." Proc., Proceedings of the 2nd European Symposium on Penetration Testing, Amsterdam, 493-500.

Chan, W. T., Chow, Y. K., and Liu, L. F. (1995). "Neural network: An alternative to pile driving formulas." Computers and Geotechnics, 17, 135-156.

Chen, Y. J., and Kulhawy, F. H. (1994). "Case history evaluation of the behavior of drilled shafts under axial and lateral loading." Report No. TR-104601, Electric Power Research Institute, Palo Alto, Calif.

Coyle, H. M., and Castello, R. R. (1981). "New design correlations for piles in sand." Journal of Geotechnical Engineering, 107(7), 965-986.

Cramer, N. L. (1985). "A representation for the adaptive generation of simple sequential programs." Proc., Proceedings of the international conference on genetic algorithms and their applications, Carnegie-Mellon University, Pittsburgh, PA, 183-187.

Das, B. M. (1995). Principles of foundation engineering, 3rd Edition, PWS Publishing Company, Boston, MA.

Das, S. K., and Basudhar, P. K. (2006). "Undrained lateral load capacity of piles in clay using artificial neural network." Computers and Geotechnics, 33(8), 454-459.

de Ruiter, J., and Beringen, F. L. (1979). "Pile foundation for large North Sea structures." Marine Geotechnology, 3(3), 267-314.

Eberhart, R. C., and Kennedy, J. (1995). "A new optimizer using particle swarm theory." Proc., Proceedings of the Sixth International Symposium on Micro-Machine and Human Science, IEEE, Nagoya, 39-43.

Elshorbagy, A., Corzo, G., Srinivasulu, S., and Solomatine, D. P. (2010). "Experimental investigation of the predictive capabilities of data driven modeling techniques in hydrology-part 1: concepts and methodology." Hydrology and Earth System Science 14, 1931-1941.

Eslami, A., and Fellenius, B. H. (1997). "Pile capacity by direct CPT and CPTu methods applied to 102 case histories." Canadian Geotechnical Journal, 34(6), 886-904.

Fausett, L. V. (1994). Fundamentals neural networks: Architecture, algorithms, and applications, Prentice-Hall, Englewood Cliffs, New Jersey.

Flood, I. (2008). "Towards the next generation of artificial neural networks for civil engineering." Advanced Engineering Informatics, 22(1), 4-14.

Gandomi, A. H., and Alavi, A. H. (2012). "A new multi-gene genetic programming approach to non-linear system modeling. Part II: geotechnical and earthquake engineering problems." Neural Computing Applications, 21, 189-201.

Gardner, M. W., and Dorling, S. R. (1998). "Artificial neural networks (the multilayer perceptron) - A review of applications in the atmospheric sciences." Atmospheric Environment, 32(14/15), 2627-2636.

Giustolisi, O., and Savic, D. A. (2006). "A symbolic data-driven technique based on evolutionary polynomial regression." Journal of Hydroinformatics, 8(3), 207-222.

Goh, A. T., Kulhawy, F. H., and Chua, C. G. (2005). "Bayesian neural network analysis of undrained side resistance of drilled shafts." Journal of Geotechnical and Geoenvironmental Engineering, 131(1), 84-93.

Goh, A. T. C. (1994). "Nonlinear modelling in geotechnical engineering using neural networks." Australian Civil Engineering Transactions, CE36(4), 293-297.

Goh, A. T. C. (1995a). "Back-propagation neural networks for modeling complex systems." Artificial Intelligence in Engineering, 9, 143-151.

Goh, A. T. C. (1995b). "Empirical design in geotechnics using neural networks." Geotechnique, 45(4), 709-714.

Goh, A. T. C. (1996). "Pile driving records reanalyzed using neural networks." Journal of Geotechnical Engineering, 122(6), 492-495.

Goldberg, D. E. (1989). Genetic Algorithms in Search Optimization and Machine Learning, Addison - Wesley, Mass.

Hiley, A. (1922). "The efficiency of the hammer blow, and its effects with reference to piling." Engineering, 2, 673.

Holland JH. (1975). "Adaptation in natural and artificial systems." University of Michigan

Ismail, A., and Jeng, D.-S. (2011). "Modelling load-settlement behaviour of piles using high-order neural network (HON-PILE model)." Engineering Applications of Artificial Intelligence, 24(5), 813-821.

Ismail, A., Jeng, D.-S., and Zhang, L. L. (2013). "An optimised product-unit neural network with a novel PSO-BP hybrid training algorithm: Applications to load-deformation analysis of axially loaded piles." Engineering Applications of Artificial Intelligence, 26(10), 2305-2314.

Janbu, N. (1953). "Une analyse energetique du battage des pieux a l'aide de parametres sans dimension." Proc., Norwegian Geotechnical Institute, Oslo, Norway, 63-64.

Koza, J. R. (1992). Genetic programming: on the programming of computers by natural selection, MIT Press, Cambridge (MA).

Lee, I. M., and Lee, J. H. (1996). "Prediction of pile bearing capacity using artificial neural networks." Computers and Geotechnics, 18(3), 189-200.

Maier, H. R., and Dandy, G. C. (2000). "Applications of artificial neural networks to forecasting of surface water quality variables: issues, applications and challenges." Artificial Neural Networks in Hydrology, R. S. Govindaraju, and A. R. Rao, eds.Kluwer, Dordrecht, The Netherlands, 287-309.

McCulloch, W. S., and Pitts, W. (1943). "A logical calculus of ideas imminent in nervous activity." Bulletin and Mathematical Biophysics, 5, 115-133.

Meyerhof, G. G. (1976). "Bearing capacity and settlement of pile foundations." Journal of Geotechnical Engineering, 102(3), 196-228.

Nawari, N. O., Liang, R., and Nusairat, J. (1999). "Artificial intelligence techniques for the design and analysis of deep foundations." Electronic Journal of Geotechnical Engineering, http://geotech.civeng.okstate.edu/ejge/ppr9909/index.html.

Nejad, F. P., Jaksa, M. B., Kakhi, M., and McCabe, B. A. (2009). "Prediction of pile settlement using artificial neural networks based on standard penetration test data." Computers andGeotechnics, 36(7), 1125-1133.

Pal, M., and Deswal, S. (2010). "Modelling pile capcity using Gaussian process regression." Computers and Geotechnics, 37(7-8), 942-947.

Park, H. I., and Cho, C. W. (2010). "Neural network model for predicting the resistance of driven piles." Marine Georesources and Geotechnology, 28(4), 324-344.

Poulos, H. G., and Davis, E. H. (1980). Pile foundation analysis and design, Wiley, New York.

Randolph, M. F. (1985). "Capacity of piles driven into dense sand." Rep. Soils TR 171, Engineering Depatment of Cambridge University, Cambridge.

Rausche, F., Moses, F., and Goble, G. G. (1972). "Soil resistance predictions from pile dynamics." Journal of Soil Mechanics and Foundation Engineering Division, 98, 917-937.

Reese, L. C., Isenhower, W. M., and Wang, S. T. (2006). Analysis and design of shallow and deep foundations, John Wiley & Sons, New York.

Rezania, M., Faramarzi, A., and Javadi, A. (2011). "An evolutionary based approach for assessment of earthquake-induced soil liquefaction and lateral displacement." Engineering Applications of Artificial Intelligence, 24(1), 142-153.

Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). "Learning internal representation by error propagation." Parallel Distributed Processing, D. E. Rumelhart, and J. L. McClelland, eds., MIT Press, Cambridge, 318-362.

Savic, D. A., Giutolisi, O., Berardi, L., Shepherd, W., Djordjevic, S., and Saul, A. (2006). "Modelling sewer failure by evolutionary computing." Porceedings of the Institution of Engineers, Water Management, 159(WM2), 111-118.

Schmertmann, J. H. (1978). "Guidelines for cone penetration test, performance and design." FHWA-TS-78-209, U. S. Department of Transportation, Washington, D. C., 145.

Semple, R. M., and Rigden, W. J. (1986). "Shaft capacity of driven pipe piles in clay." Ground Engineering, 19(1), 11-17.

Shahin, M. A. (2010). "Intelligent computing for modelling axial capacity of pile foundations." Canadian Geotechnical Journal, 47(2), 230-243.

Shahin, M. A. (2013). "Artificial intelligence in geotechnical engineering: applications, modeling aspects, and future directions." Metaheuristics in Water, Geotechnical and Transport Engineering, X. Yang, A. H. Gandomi, S. Talatahari, and A. H. Alavi, eds., Elsevier Inc., London, 169-204.

Shahin, M. A. (2014a). "Load-settlement modeling of axially loaded drilled shafts using CPT-based recurrent neural networks." International Journal of Geomechanics, http://dx.doi.org/10.1061/(ASCE)GM.1943-5622.0000370.

Shahin, M. A. (2014b). "Load-settlement modeling of axially loaded steel driven piles using CPT-based recurrent neural networks." Soils and Foundations, 54(3), 515522.

Shahin, M. A. (2014c). "Use of evolutionary computing for modelling some complex problems in geotechnical engineering." Geomechanics and Geoengineering: An International Journal, doi: 10.1080/17486025.2014.921333, 1-17.

Shahin, M. A., and Jaksa, M. B. (2005). "Neural network prediction of pullout capacity of marquee ground anchors." Computers and Geotechnics, 32(3), 153-163.

Shahin, M. A., and Jaksa, M. B. (2006). "Pullout capacity of small ground anchors by direct cone penetration test methods and neural networks." Canadian Geotechnical Journal, 43(6), 626-637.

Shahin, M. A., Jaksa, M. B., and Maier, H. R. (2001). "Artificial neural network applications in geotechnical engineering." Australian Geomechanics, 36(1), 4962.

Shahin, M. A., Jaksa, M. B., and Maier, H. R. (2009). "Recent advances and future challenges for artificial neural systems in geotechnical engineering applications." Journal of Advances in Artificial Neural Systems, doi: 10.1155/2009/308239.

Solomatine, D. P., and Ostfeld, A. (2008). "Data-driven modelling: some past experience and new approaches." Journal of Hydroinformatics, 10(1), 3-22.

Tarawneh, B. (2013). "Pipe pile setup: database and prediction model using artificial neural network." Soils and Foundations, 53(4), 607-615.

Tarawneh, B., and Imam, R. (2014). "Regression versus artificial neural networks: predicting pile setup from empirical sata." KSCE Journal of Civil Engineering, 18(4), 1018-1027.

Teh, C. I., Wong, K. S., Goh, A. T. C., and Jaritngam, S. (1997). "Prediction of pile capacity using neural networks." Journal of Computing in Civil Engineering, 11(2), 129-138.

Tian, X., Wei, W., and Xiaoni, W. (2010). "Artificial neural network model for time-dependent vertical beraing capacity of preformed concrete pile." Advanced Materials Research, 29-32(Part 1), 226-230.

Vesic, A. S. (1977). "Design of pile foundations." National Cooperative Highway Research, Synthetic of Practice No. 42, Washington DC: Transportation Research Board.

Wellington, A. M. (1892). "The iron wharf at Fort Monroe, Va." ASCE Transactions, 27, 129-137.

Zurada, J. M. (1992). Introduction to artificial neural systems, West Publishing Company, St. Paul.

Figure Captions:

Figure 1. Linear regression versus artificial intelligence (AI) modeling: (a) linear regression modeling (after Shahin et al., 2001); (b) AI data-driven modeling (adapted from Solomatine and Ostfeld, 2008).

Figure 2. Typical structure and operation of artificial neural networks (ANNs) (after Shahin et al., 2009).

Figure 3. Main steps in artificial neural network (ANN) model development (Maier and Dandy 2000).

Figure 4. Typical example of genetic programming (GP) tree representation for the function: [(4 - x1) /(x2 + x3)]2.

Figure 5. Typical flow diagram of the evolutionary polynomial regression (EPR) procedure (after Rezania et al., 2011).

Figure 6. Comparison between theoretical settlements and artificial neural network (ANN) predictions for pile foundations (after Goh, 1994)

Figure 7. Some simulation results of the recurrent neural network (RNN) model in the training and validation sets for drilled shafts (after Shahin, 2014a)

Figure 8. Some simulation results of the recurrent neural network (RNN) model in the training and validation sets for steel driven piles (after Shahin, 2014b)

Performance of artificial neural network (ANN) model and traditional methods for predicting friction capacity of piles in clays (Goh, 1995b).

Method Coefficient of correlation, r_Error rate (kPa)

_Training_Testing_Training_Testing

ANN (Goh, 1995b) 0.985 0.956 1.016 1.194 Semple and Rigden (1986) 0.976 0.885 1.318 1.894 P method (Burland, 1973) 0.731_0.704_4.824_3.096

Performance of artificial neural network (ANN) model and traditional methods for predicting ultimate load capacity of driven piles in cohesionless soils (Goh, 1995a).

Method Coefficient of correlation (r)

Training data Testing data

ANN (Goh, 1995a) 0.96 0.97

Engineering News (Wellington, 1892) 0.69 0.61

Hiley (1922) 0.48 0.76

Janbu (1953) 0.82 0.89

Analytical performance of EPR model for pull-out capacity of ground anchors (Shahin, 2014a).

Performance measure Training set Validation set

r 0.789 0.872

R2 0.619 0.753

RMSE (kN) 0.46 0.43

MAE (kN) 0.34 0.37

1.02 0.99

Comparison of EPR model and other methods in the validation set for pull-out capacity of ground anchors (Shahin, 2014a).

Performance Method

measure EPR ANNs LCPC Das Bowles

(Shahin, 2014a) (Shahin and Jaksa, 2005) (1982) (1995) (1997)

r 0.872 0.845 0.489 0.857 0.550

R2 0.753 0.705 -0.455 -1.844 -0.102

RMSE (kN) 0.43 0.47 1.03 1.45 0.90

MAE (kN) 0.37 0.37 0.88 0.98 0.61

m 0.99 0.95 1.84 0.72 1.86

Artificial neural network Processing element

Choice of performance criteria

Training Speed

— Processing speed during recall

— Prediction accuracy

Choice of data sets

— Nurrt)er of data sets (e.g. two. three, holdoii method)

— Method for data (i/ision

Data pre-processing

— Scaling

Transformation to normality Remcwal of non-stSionarities

Choice of model inputs

Choice of variables — Choice of lags

Choice of model architecture

Connection type (e g feedforward, feedback)

— Degree of connectivity (e.g. fully connected)

— Number of layers

Number of nodes per layer (trial & error, constructs or pruning methods)

Choice of stopping criteria

Fixed number of iterations

— Training error

— Cross-validation

Choice of optimization method

Local first order (e g back-propagation)

— Local second order (e.g. Levenberg-Maquardt, Conjugate Gradient)

— Global (e.g. simulated annealing, genetic algorithm)

Validation

I S Ian I

I Initialize (he inpul matrix I 1

Create initial population of exponent vectors, randomly

Assign exponent vectors to the corresponding columns of the inpul matrix (to create a population of mathematical structures)

Evaluate coefficients using least square method (to create a population of equations)

Evaluate fitness of equations in the population

Select individuals from mating poo! of exponent vectors

Select two exponent vectors (to perform crossover)

Select one exponent vector (to perform mutation)

Create offspring generation of exponent vectors -1- |

I End I

GAtool

2 4 6 8 10 12 14 16 18 20 22 Pile settlement/pile diameter (%)

Pile settlement/pile diameter (%)

Pile settlement/pile diameter (%)

Research highlights:

• Review of artificial intelligence (AI) applications in pile foundations

• Salient features associated with the AI modeling development

• AI has superior predictive ability compared to traditional methods.