0International Journal of Advanced Robotic Systems
OPEN V!/ ACCESS ARTICLE
Multi-AUV Hunting Algorithm Based on Bio-inspired Neural Network in Unknown Environments
Regular Paper
Daqi Zhu1, Ruofan Lv1*, Xiang Cao1 and Simon X. Yang2
1 Laboratory of Underwater Vehicles and Intelligent Systems, Shanghai Maritime University, Shanghai, China
2 The Advanced Robotics and Intelligent Systems Laboratory, School of Engineering, University of Guelph, Guelph, Canada Corresponding author(s) E-mail: shadowkiller028@163.com
Received 19 May 2014; Accepted 17 September 2015
DOI: 10.5772/61555
© 2015 Author(s). Licensee InTech. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
The multi-AUV hunting problem is one of the key issues in multi-robot system research. In order to hunt the target efficiently, a new hunting algorithm based on a bio-inspired neural network has been proposed in this paper. Firstly, the AUV's working environment can be represented, based on the biological-inspired neural network model. There is one-to-one correspondence between each neuron in the neural network and the position of the grid map in the underwater environment. The activity values of biological neurons then guide the AUV's sailing path and finally the target is surrounded by AUVs. In addition, a method called negotiation is used to solve the AUV's allocation of hunting points. The simulation results show that the algorithm used in the paper can provide rapid and highly efficient path planning in the unknown environment with obstacles and non-obstacles.
Keywords Multi-AUV (Autonomous Underwater Vehicle), Bio-Inspired Neural Network Algorithm, Hunting, Path Planning
1. Introduction
An autonomous underwater vehicle (AUV) is a type of intelligent robot that can travel in the underwater environment without requiring input from an operator [1-2]. AUVs have been studied by many scientists and applied in a variety of tasks such as underwater rescue, detection, location, etc. Many achievements in single AUV research have been made. However, many complicated tasks nowadays go beyond the single AUV's capability. Multi-AUV systems, in recent years, have been studied in areas such as formation [3-4], localization [5], cooperative hunting [6-8], cooperation searching [9], path planning [10-11], task assignment and cooperation [12-15], due to their outstanding robustness and high efficiency of coordination and collaboration. Among the areas mentioned above, the multi-AUV hunting problem has attracted much attention, because it can be applied in military tasks and is a good verification of cooperation and coordination of a multi-AUV system.
Much research has been carried out recently on the multirobot hunting issue and some approaches are proposed in
this paper. The hunting algorithm can essentially be classified into two categories: centralized control methodology and distributed control methodology. The difference is whether or not there is a supervisor. Various methods have been proposed, which include behaviour based, virtual structure based, leader-follower, artificial potential based and graph theory based methods. Grinton [16] presented a mechanism of commitments and conventions to guide the multi-robots' cooperation in a hunting task. Sauter [17] used a reinforcement learning method with animal behaviour to conduct research on the hunting problem. Cai [18] proposed an improved auction algorithm for multi-robot hunting cooperative behaviour.
However, all of the above articles concentrate on the known environment for a cooperative robot hunting task. In reality, tne wnrkiog environmont for robota is often unknasrn.InsMrrto deal rath a multi-ntbol: huntingtnsk in for unknettan enrirorcmenk Nghot [19] nroposod a ^ntingtteategy mfrswatmintemgencefor guntihg robotato ekaiucktthgtarget.Fe^pOJ^esented sninpul:-notpht -eedbacklcnegrizationalhorithm to caiculate the velocity of hunting robots in order to execute a hunting tariekhonn [Ul] hen pacneseOa metggdaeoehondttg-sion gdae^ketedn over neiwarksioconductresoarchog IntaUi-tent ki^o^^ohs huntm- fur schools of tish. Huweaet, the krec^lei^^ Coer paperr did neCconstdei mauiboilding er^eshtehennec^^y and Chada avoldanco was not uiucIIi sousi-erea in the ii-erature.
Recently, some researchers have approached the hunting process with simpleobstacles. Yamaguchi [22] proposeda method based on making troop formations for enclosing the target ad presented a smooth time-varying feedback ontrol law for coordinating the motions of multi-robots. Pan [ae] applied tfae improved slgoritnm to
themuni-sopat Anting groblom[ -nuse
stupes tha hinting ta^et is often otetic antl iHr na^ cols/rtentwii1 thereal envtronmeaU
Taiadde tha sWe-tcs>mmao discessod aOove, hie r^^i psoposad acleperatikn huntmdstraiegywith dimamic aHiancs tocVace amoviug harnet. This ae^aCi^t^dc^ Shecomp/et1on tima Co ems cotent.Wangpa] dropnsek a new 1gltingmethod ftth ncwdefimfiol eencektsef dscupp cut s>veriaтdlpn anta and finally coiai1ated an egtimizek paft tor multi-robht hentmg OiS She eueironmaut is too open and iha initiai ioaation ot tim Ttlnfinure1etsirtok doseto ihemovingternai. Nee-, Nl andYa^^] proposed an algorititmbasan on ai^ies inspired neural network model with formation strategy that was applied in a hunting task with good communication among several neurons; good coordination can be viewed during the whole hunting task. However, in the catching stage, the robots depend on the formation strategy and do not need the guidance of the neural network. Therefore, although there have been many approaches applied to the multi-robot hunting problem, ¡hehmifcitipns in terms of coordination, robustness and
effectiveness of a robot team mean that these methods cannot be fully applicable for a multi-AUV cooperative hunting problem in underwater circumstances.
This paper focuses on the situation in which the environment is unknown and the target is intelligent, with unpredictable and irregular motions. The multi-AUV hunting algorithm based on the bio-inspired neural network is presented. The hunting AUVs' paths are guided through the bio-inspired neural network and the results show that it can achieve the desired hunting result efficiency.
This paper is organized as follows: Section 2 describes the map of the hunting process and four kinds of hunting final states are given. In Section 3, the bio-inspired neural netwOTk alusrithmla dosiunad. He strategy stf vsTi p-k/trung andahewnole1untinkprecess arc daemiOadia deialLSimul atikdsareconducted in Section 4 and Section concludes the whole paper.
2. ProClemStatemagf
In this paper, a cooperative hunting task of multi-AUV in an unknown environment is studied. The multi-AUV system has no information about the underwater environment. The hunting task will be accomplished when the target is encircled by hunting AUVs. The underwater environment model is presented by using a discrete grid map. The grid map divides the working condition into cells of the same size, while every cell has two states - obstacles and free space, as shown in Figure 1.
The two-dnnensiongrid mapis iabeiiades V. TPo enine andiho iimu of atehuTtiwu region lsalsodisccatineP. T^i^^, tPo Wuntmg araaconbadefiueh tst iairf grid maps. The number of AUVr is denoted as Ac = {n Pc, •••Vc a a^d in this paper the research work focn ses on the conkiSion that oaiy one aargei is Uhnted bo malti-AUVc. Hen cc the ^^^hei: isiabellcd as Vc and the pbctasies nredenote(n as O^/O^ O^-AC The target has the same ipteilidani abilities as the hunting AUVs. Ea ch AUV h as 360 degree visual capability. The detection angle of each AUV and the target is 3600/ 8 = 450 respectively.
Obstacle j j Free space
Figure 1. Two dimension map
Hunting AUV
Target
2 r ^r ^
Figure 2. Target is hunted by AUVs in four conditions (a) Hunted state in corner (b) Hunted state in boundary (c) Hunted state with help of obstacles (d) Hunted state by four hunting AUVs
When the hunting process begins, the hunting AUVs will move towards the moving target. During the process, the hunting AUVs can avoid the obstacles and find a short path to catch the target. The target will judge whether there are any AUVs lying in the neighbouring cells. If so, then the target will try to modify its moving direction and run to the free space. Figure 2 shows the conditions where the target is successfully surrounded by hunting AUVs. The final hunting state can be divided into four situations, which are, respectively: the target surrounded by AUVs at a corner, at a boundary, with help of obstacles and without any help.
3. Hunting Algorithm based on Bio-inspired Neural Network
The neural network model, as a highly parallel distributed system, has shown its superiority in the mobile robot path planning and trajectory tracking research. On the whole, the study process is the essential part when a neural network is applied, but timeliness and efficiency cannot be guaranteed. The bio-inspired neural network model was proposed by Hodgkin and Huxley in 1952, by using a circuit element to describe the electric current of membrane [27]. Grossberg [28] summarized and improved this model into a "shunting model", which is based on the Hodgkin-Huxley model. The bio-inspired neural network model was applied to complete coverage path planning by Yang and Luo [29]. Pichevar and Rouat applied the approach to solve the sound source segregation problem [30]. The bio-inspired network model applied in the multi-robot cooperative hunting area does not need any learning process and the external excitation and inhibition will lead the robot to select every step to reach the goal.
In his 2011 paper [26], Ni used the bio-inspired neural network model with a formation and dynamic alliance algorithm to chase targets. Unusually, in this paper, the bio-inspired neural network is directly used in an AUV hunting task without the assistance of any other algorithm. This means that the hunting process can be completed with the
proposed bio-inspired neural network algorithm and the negotiation method, without a further synchronization method. The synchronization strategy will be considered in the further multi-AUV hunting research, in order to improve the hunting efficiency.
The hunting problem for AUVs and mobile robots is theoretically the same, so this work is a preliminary study for the multi-AUV hunting problem. In this paper, our starting point is to try to apply this method to the AUV system, which has not been considered in previous work. Unlike the mobile robot or an Unmanned Aerial Vehicle (UAV), due to the complicated underwater environment, the obstacles are assumed to be unknown and will be detected by underwater sensors, especially sonar, which is very different from mobile robots or UAVs. The work of underwater map building has been examined in the author's former work [31]; therefore in this paper only a simple conclusion is given as a fundamental part of the AUV hunting problem. Here, the target is assumed to be moving on a set path; when it detects the risk of hunting AUVs, it will move to avoid the hunting. In this paper, since studies have already been carried out on map building and localization, we have concentrated on the hunting process.
3.1 Bio-inspired Neural Network Algorithm
The "shunting model" proposed by Grossberg is shown in the following formulation:
= - Ax. + (B - x. )S+ - (D + x. )S-
dt ' ' ' ' '
This function is called the shunting equation. In this equation, xt is the neural activity of the i-th neuron; A, B and D represent the passive decay rate and the upper and lower bounds of the neural activity respectively, which are nonnegative constants; S+ and S- are the excitatory and inhibitory inputs to the neuron. In the hunting process, the hunting AUVs' motions are guided by the dynamic landscape of the neural network. The excitatory input St+ results from the target and its neighbouring AUVs and the input S - only results from the obstacles. In this context, the dynamic of the i-th neuron in the neural network can be characterized by a shunting equation as
= -Ax + ( B - x )| [I ]++X w [Xj ]+ |-( D + x )[I ]
where k is the number of neural connections of the i-th neuron to its neighbouring neurons. The terms
Uil + + Z + and [I°]- are the S + and S - in equation (1),
respectively.
The term [a]+ is a linear-above-threshold function defined as [a]+ = max{a, 0} ; similarly the term [a] ~ = mini - a, 0}. [IJ+ and [IJ" are the variables that represent the input to the ith neuron from the target and obstacle, respectively. They are defined as
E if it is a neighboring cell to target -E ifitisanobstacle 0 otherwise( free space)
where E > > B, which is a very large positive constant. In equation (2), the term wj is defined as
the AUV to move. Under the neural network guidance, the AUV's moving strategy can be followed as
Path = {P I x = maxjx, ,l = 1,2,...m},P = P,P = P } (6)
1 n pn ' ' ' J'p c' c n' V /
where x^ represents the activity value of neighbouring cells for the AUV's current position. m is the number of neighbouring cells. Pc, Pn and Pp represent the AUV's current position, next position and previous position respectively. In the AUV's moving process, the neighbouring cell of maximum activity value will be chosen as the next position. Simultaneously, the activity value of the whole neural network will be refreshed.
w = f(q-qi|)
(4) 3.3 Strategy of Intelligent Target
where^ and^ are twovectorsand 1%-°;! istheEuclidean distancebetween them. IT he fenation/(a) is a monotoni-cally decreasing function, which ks define d as
Si /a if hea <Rn tfa>R
where f and Rn are positive constants. Obviously, the wwi..1: connaotian coefficients ara cyeemetocel, w^s^.Fioeare 0 cOiawe tfenaurat neewoekmodeline 2;D environmer^t [32].
.On the basis of the mechanism of the bio-inspired neural network model in the multi-AUV hunting process, the target's motion is also limited by the control of the neuron's value. As mentioned above, the target is intelligent and the escape runaway choice for the target is random. To put it asimply, the moving target will run to an open space with few obstacles or hunting AUVs. When the hunting AUVs block or impede its route, the escape direction will be immediately affected. If the hunting AUV occupies one of ethe grids around the target, the movement direction will be limited. When the surrounding grids are all occupied by hunting AUVs,thetargetwill stopmoving.Generally, the maximum speed of the moving target is less than that of thehunting AUV.
Figure 3. 2-D model of neural network
In this structure, each neuron is connected by adjacent neurons, which form the whole network for transmission of activity.
3.2 Strategy of Path Selection
An AUV's moving path is guided by the activity of the neural network. Let us assume that an AUV's current position is k and the constant m represents the number of neighbouring neural cells. Thus, there are m selections for
3.4 HuntingProcess
When the hunting process begins, all the AUVs will pursue the moving target together. In order to show clear results of hunting, a matrix Trace (k *2) is defined to memorize the position of the AUV for path planning. The k represents the number of hunting steps for each AUV. The hunting task of the AUVs is to move towards encircling the target in a few steps. The whole process of the hunting behaviour can besummarized withthefollowingprocedures:
Step 1: Initialize the whole activity values of cells to zero.
Step 2: Set the initial position of each AUV to the current location.
Step 3: AUV will find the next step by choosing the maximumactivityvalueofeightneighbouringcells.
Step4:Storethe current AUVposition to matrix Trace.
Step 5: Set the activity value of the cells that the AUV has travelled through to zero.
Step 6: Judge whether the current position is the neighbouring cell of the target. If it is, set the activity value of the current position to - E in order to prevent other AUVs from moving to the same cell by mistake; otherwise go to Step 3.
Set the number of AUV and hunting points to M (Non-Negative
Compute the distance between each
the hunting point to AUV by choosing nearest distance
Choosing the furthest AUV to allocate to the hunting point that is allocated occasionally, and set the neuron value of this point to -E
Figure 4. Method of negotiation
Step 7: If the target continues to move, the AUV will follow the target until the up, down, left and right positions of the target are occupiedbyhunting AUVs. Then it will stop moving.
The hunting process will show that the AUV can encircle the targetuntil it cannot escape. The AUV will move towarOs thetergut directly audwitt avoid the varioue obsOaelee.
3.5 NegotiatiohMathod
In the hunting processydueio lhe cltractihnol tha mahi-mum neuron vaiueothe hustihgAUVs will yccupytae hunting pointsrandtmly. Howyvec,withouto methanism for anocatm° .hohuutingpointfor each AUV m balance, the eefeativenem a! tlie hanting task wtH be oteodenod. TVtaa, e methoS c^ntteO nonoV^i^ioa^ Is presented id this fapee to solve the ^oblem that ismentkceti above.The wfoola method can ba summarized es y-gure 4.
Under the guidance ofthe negotiaiis>h meUlod, tie hunting points will be allocated by AUVs automatically; hence it aaomanef°c AUV finieh ^shuntu^ task aukkly and shorten an unsiecessarysailmg As shownmF-guie 5, the four AUVs rente ihe moymg farget rnapactivem and the four hu nting points marked as 1 to 4 are allocated with efoncgotiatiay meSlfod. TUo^ AUm san tsian t-niala thv
huntinf egta ^im
af-ho hunting tff.U"a-o are mosctfmn teeded 14 . UVo ehnwn in Fiaute5), the eatk ^a^tttnave!n1: can tee conducteoC-hst.Tha taska-locae]ov nan be ^ven scccudmu crr .he ditianto wad the AUVsthct are nutt ccslgned at ask w iil stand e ti-LSome work on tasnatlacatthn hasheencarrle d out by the authors
in [33]. It will be a separate work, which needs further research.
hiuote 5. ehiscdemtteniagrcm o! the
h. Simhlatlcm SUetiet
To demonstratetnaleastb-lith and etfectivsnesc ot the proposed algarithm, eomo rimulatioh ehperimentshove been conducted. In this section, the simulation can be divided into two parts. Hunting with and without obstacles in an unknown environment is simulated and compared with another approach. In addition, the growth in activity value of each AUV will be displayed in a chart. The s^ntf^^ta ennironmont used is Windows 7, in-tdKR^rethM^ Duo CPU E8400 O.OOGH^ Ot nrumroy. The compilation tool is MATOAB 2011a.
4.1 SimulationDesign
tn these experimcntc1 a tecn igginen fcft a foartL ot AUVs PC = {Pc,, PCj ...PC} with only one target, Ev. The environ-
ment of the hunting area is a NS x NS =20x 20 grid map. The hunting task can be divided into two conditions: hunting without obstacles and with different shapes of obstacles. The boundaries of the area are known to the AUVs as well as to the target, while the environmental information of the whole area is unknown to both.
The number of AUVs is set at four and their movement is based on the bio-inspired neural network model in the sections above. The target is intelligent and moves randomly until it is surrounded by hunting AUVs. The speed of the hunting AUVs is set at 1 second / grid and the target speed is 4 second / grid. The parameters are set at A =2, B =1, D =1, ^ =0.6, E =100, Rn =2.
4.2 Hunting Simulation Experiment without Obstacles
The first simulation is conducted to test the cooperative hunting process without obstacles. For easy discussion, it is assumed that there are four hunting AUVs with only one target. The initial location of the target is (11,6). The hunting AUVs are PC,, PC2, PC^, PC^ and the initial positions of them are PC = {(1, 20), (1, 0), (20, 1), (20, 20)}, respectively. Figure 4(a) shows the initial locations and state of the hunting condition. Figure 6(b) shows the hunting process for the first seven steps. The target has already found the hunting AUVs moving towards it and it starts to escape from its initial location.
Figure 6(c) shows that in the final state of the hunting process, the moving target is surrounded by hunting AUVs in (11,9) and, through the hunting task, it is easy to see that the AUVs are moving directly to the target and do not collide with each other.
Table 1 lists the activity value of the neuron at each step in the hunting process of AUV PC^ under the circumstance with obstacles. The red data show the value corresponding to the next position that the AUV chooses. Obviously, the AUV selects the cell with the biggest activity value from the neighbouring eight cells.
In Table 1, in the initial stage of the hunting process, PC3 is located in the position (20,1), which is adjacent to the corner and boundary, so the number of neighbouring neurons does not equal eight. PC3 then chooses Pc3(x,y+1) to be the
next position, corresponding to (20,2), because the activity value of the neuron in this position is 6.945e-13, which is the largest of the neighbouring cells.
After the AUV runs one step, the whole system of the neural network will be refreshed immediately; then PC3 will judge whether the next position is the neighbouring cell of the target or not. Obviously, the answer is no. Therefore it chooses Pc3(x-1, y+1), which corresponds to (19,3). The activity value in that position is 4.396e-12, which is the largest of five neighbouring values. Similarly, PC3 chooses
the next position (18,4) by the same mechanism of path
Pc □ Pc, 0 target #
20 ZZ_ k) " 18 ■__
16 ZZZ!
14 zzz
12 ZZZ! io ZZZ! 8 ZZZ! 6 ZZZ! 4 ZZZ 2 ZZZ!
10 (a)
Pa, ♦ target "ù
10 (b)
« 0 0 Ù ♦ ♦ ♦ ♦
0 0 0 ♦ ♦ ♦
0 0 0 ♦ ♦ ♦
0 • • o Ï ■ ■ ■
■ C' • • ? ■ ■ ■ —I
5 10 15 20
Figure 6. Hunting process of the simulation (a) initial locations and the state of hunting condition (b) hunting process - first seven steps (c) final locations with trajectories of AUVs
planning. Now tite number of neurons is eight, because the. position of Pc, is not close to ths boundary. Wish the same.
method of choosing the maximum activity valye of neighbouring neurons, when PC3 sails to (11,8), it finds that the position is next to the moving target and the other position. oO neiyhbouring cells of the target aoe oceupied by other AUVs. It then stops hunting and finishes its hunting task. The results shown in Table 1 correspond to the hunting process in Figure 6 and confirm that it is effective to apply the bio-inspired neural network algorithm to the multi-AUV hunting task.
PC1 0 P
PC 0 PC
Pa * PC2 • Pa
Pa 0 target w
Current position (x,y) (20,1) (20,2) (19,3) (18,4) (17,5) (16,6) (15,7) (14,7) (13,7) (12,8)
Neighbouring cells
Pc3(x+1,y) - - 1.530e-13 1.627e-09 8.603e-07 7.879e-05 0.0021 0.0234 0.1291 0.4016
Pc3(x-1,y) 3.848e-14 1.493e-12 1.833e-10 7.500e-07 0.0001 0.0060 0.0719 0.3138 0.6023 0.9809
Pc3(x,y-1) - 3.848e-14 2.643e-12 1.547e-08 6.226e-06 0.0005 0.0116 0.0793 0.2566 0.5678
Pc3(x,y+1) 6.945e-13 1.530e-13 9.944e-12 7.156e-08 1.873e-05 0.0009 0.0119 0.0881 0.3306 0.9807
Pc3(x+1,y+1) - - 3.065e-13 2.796e-09 1.221e-06 8.953e-05 0.0019 0.0206 0.1173 0.3237
Pc3(x-1,y+1) 6.373e-14 4.396e-12 4.782e-10 1.561e-06 0.0002 0.0072 0.0611 0.2935 0.9807 -0.9195
Pc3(x+1,y-1) - - 6.654e-14 7.446e-10 4.736e-07 5.346e-05 0.0018 0.0197 0.3055 0.3865
Pc3(x-1,y-1) - 5.276e-14 4.878e-11 2.529e-07 6.502e-05 0.0035 0.0565 0.2286 0.4206 0.7320
Next position (20,2) (19,3) (18,4) (17,5) (16,6) (15,7) (14,7) (13,7) (12,8) (11,8)
Table 1. The changing activity values of the neurons of Pc in the hunting process of Figure 6(c)
4.3 Hunting Simulation Experiment with Obstacles
To prove the robustness of the proposed approach, some obstacles are added to this part of the simulation. The shapes of the obstacles are varied, comprising U-shape, polygon-shape, square-shape and rectangle-shape, in order to increase the difficulty of the hunting task. In Figure 5, the yellow pentagram represents the target and the black blocks are the static obstacles in the simulation. The hunting AUVs are PC , PC^, PC3, PCj,which still start moving from the location of PC = {(1, 20), (1, 0), (20, 1), (20, 20)} respectively. With the guidance of the neural network, the AUVs will move directly to the target and avoid the obstacles. Figure 7(a) shows the initial state of the hunting process. Figure 7(b) shows the hunting process of each AUV and moving target and Figure 7(c) shows the final state and the whole trajectories of the target and AUVs. Figure 7(d) shows that the AUVs can complete the hunting task with different shapes of obstacles.
Table 2 reflects the whole hunting process of one of the hunting AUVs (PC3). The dynamic changing values of the neurons also show the mechanism of path planning for each hunting AUV. The sign "-"represents a cell that is out of boundary and whose activity value does not exist. The data that are marked in a red colour represent the maximum value of the neighbouring eight cells of the current position.
In order to further prove the robustness of the proposed approach, the hunting process with a wider U-shaped obstacle has been simulated in Figure 8. It can be clearly seen that when the AUV is inside a U-shaped obstacle and the target is on the other side, the hunting AUV can navigate back and move around the obstacle to reach the target successfully.
4.4 Comparison with Different Method
To further test the priority of the proposed method applied to the hunting process, this paper conducts a comparison
with the artificial potential field method [34-35] applied in the hunting process. The potential fieldwork was proposed 15 years ago and has been applied in many areas. However, the application in a multi-agent system is still a new area, especially for the multi-AUV system, and a number of research papers on this topic are being published every year. In the artificial potential field method, a gravitational field to a target and a repulsive field to obstacles are built to work together, to lead the AUVs to move towards the target step by step. The direction of the hunting AUV is decided by a composition of forces, which include the gravitational pull from the target and the repulsion from the other hunting AUVs.
A brief description of the artificial potential field method can be summarized as follows: first, construct a distance function between the AUV and the target:
r(r, g) = 1 Kv y1) - g( y2)\
The generated gravitational field can then be given as:
UatJp(r, g)) = x||r(r, g)||m
where m is a positive constant. The attractive force of the target is:
Fa«! =-™«n(p(r, g))
:mX\\r( XV yù - g(X2' y 2 )|| "
The distance function between the AUVs can be given as:
Ureps U reps (.Pi ' 0))
20 0 18 16 m * ♦ ♦
14 11
12 10 ■ in i
8 ■ 6 ■ M IT
4 2 ■ 2 i i 1 1 ■ -----3
P o P ■ P ♦ target tV
5 10 15 20
pc, ^ PC] • Pp ■ Pc, ♦ target A
•IIIIIIIIIIIIII
P o P o P
j c, 1 CC-, 1 C
PC ♦target "uT
10 (c)
Pc t target tV
5 10 15 20
Figure 7. Hunting task with obstacles (a) initial location (b) hunting process of first six steps (c) final locations with trajectories of AUVs (d) hunting with different types of obstacles
20 18 16 14 12 10 8 6 4
5 10 15 20
Figure 8. Hunting task with U-shaped obstacle
The repulsive force is generated between the hunting AUVs themselves:
0 ♦
0 ♦
if O • o ♦
O 0 o o o t ■ ♦
O 0 0 0 ♦ i *
• ♦ ■
O 1 1 ■
o 1 1 i
• ■
O ■
O ■
i f ■
I m ■ ■
O ■ ■
• ■
F (p (r, o)) = -VU
reps^' i\ ' // re
s (Pi(r, o))
The hunting AUVs will then be guided by the total forces ofattractionand repulsion.
Figure 9 shows the simulation result of the artificial potential field method in a hunting experiment. Four AUVs labelled as {1,2,3,4} start from locations {(25,25), (1,25), (25,1),(1,1)} respectively and hunt the red moving target, which starts from (13,13) simultaneously. The target is finally caught in (13,24). Figure 10 shows the simulation result based on the method proposed in this paper. Four hunting AUVs start from locations {(25,25), (1,25),(25,1), (1,1)} respectively and the target escapes from (13,13), which is finally hunted in (13,25). The number of step for each AUV in the hunting process under the proposed method is shown in Figure 11. The result shows that the average number of steps for the hunting process is reduced by 45%. Therefore the method proposed in this paper appliedtothehuntingprocessismuchmoreefficient.
The reason for the superior performance can be explained as follows: the potential field method is basically designed on the modelling of a gravitational field and a repulsive field. Different designs of the gravitational field and the repulsive field will affect the hunting performance, but cannot directly move to the target like the proposed bio-inspired neural network method. Furthermore, one important issue ithat has not been discussed is that the potential field method has a shortage of local minimization (called deadlock); hence, for the U-shaped obstacle, it may fall into the obstacle inside without any other strategy, while the bio-inspired neural network method can solve it very well, as shown in Figure 9.
Current position (x,y) (20,1) (20,2) (19,3) (18,4) (17,4) (16,4) (15,5) (14,6) (13,7) (12,8)
Neighbouring cells
Pc3(x+1,y) - - 2.782e-14 1.203e-10 4.124e-08 2.427e-06 -0.9389 0.0031 0.0236 0.0937
Pc3(x-1,y) 1.947e-14 2.543e-14 6.987e-11 1.330e-07 2.059e-05 0.0004 0.0070 0.0559 0.2136 0.5257
Pc3(x,y-1) - 9.876e-13 6.905e-13 4.028e-09 6.658e-07 1.672e-05 0.0005 0.0077 0.0496 0.1043
Pc3(x,y+1) 4.549e-13 6.905e-13 1.056e-12 -0.9389 4.582e-07 -0.9389 0.0024 0.0225 0.1170 0.5287
Pc3(x+1,y+1) - - 2.586e-14 5.404e-11 -0.9389 1.410e-06 -0.9389 0.0039 0.0241 0.1271
Pc3(x-1,y+1) 1.054e-13 3.193e-12 8.593e-11 7.994e-08 -0.9389 0.0009 0.0147 0.1085 0.9396 0.9401
Pc3(x+1,y-1) - - 2.432e-14 1.379e-10 3.5652e-08 2.130e-06 8.799e-05 0.0018 0.0162 0.0446
Pc3(x-1,y-1) - 2.226e-14 3.193e-11 1.111e-07 6.3491e-06 8.531e-05 0.0021 0.0224 0.1208 0.1410
Next position (20,2) (19,3) (18,4) (17,4) (16,4) (15,5) (14,6) (13,7) (12,8) (11,8)
Table 2. The changing activity values of the neurons of Pc in the hunting process of Figure 6(c)
For the power consumption problem, since this work is based on the design of path planning, the power consumption can simply be in linear correlation with the hunting path length. From this point of view, it is easy to conclude that the bio-inspired neural network method is superior to potential fieldwork.
In order to further show the priority of the proposed method used in the hunting process, a chart describing the comparison of the two methods is shown in Table 3.
AUV PC1 pc -2 Pc3 pc C4 Average steps
Algorithm
Steps for Artificial potential method 42 42 42 42 42
Steps for Bio-inspired neural network algorithm 26 22 25 19 23
Table 3. The comparison of step numbers between the two methods
4.5 Hunting Simulation Extended to 3-D Environment
In this section, some preliminary work on multi-AUV hunting in the three-dimensional (3-D) environment is introduced. The proposed hunting algorithm based on the bio-inspired neural network is extended to the 3-D case, while the basic idea is essentially the same. In the 3-D simulation experiment, the hunting map is also selected as the discretization grid map. Six AUVs are selected for the dynamic hunting of an escapee. It should be noted that the complex current situation in the actual three-dimensional environment is not considered.
In this simulation experiment, static obstacles are added to the 3-D map, as shown in Figure 12(a), where a blue cube represents an obstacle.The six AUVs aw labelled as PC1,PC ,PC3, PC t PCsiPCs xnd they etast from ma tnitial poaLleon {(1,1,0)^0,0,2),(10,10,10),(3,10,10),(1,10,0),
i 10,1,10)}. The AUVs approach the target according to the masimum neuron(s activity sxtestion vechamrmand perfesm obstade anor°once. xtes ^ntingtask^ima^ sampletxd aMta pamt pnp.ftrshouMbcootesl thatwhen the target is surrounded by six hunting AUVs, the succoba-frd nubtmg state can be accomplished. Figure 12(b) shxws the Vnal stage of successful hunting and the moving trajectoi p ot cmch AUV. The target is rounded up by six AUVs, which are all in the red cube. In order to show the final state clearly, an enlarged view of the hunting AUVs anp rate taoge1 at the psint of bsing caught is demonstsated {n 0ine)o bC(bb
PI 1 11 El iE n CE E i: E
E E E E E H 1 E ¡i EMZ H i
E 4 EEE i
li 1 E J FB 3 i
LE r E 3 i
TO E li T| M 7 IL E E E T
Ë 111 E -1 L E E
|4 E L 2 1 m E 3
li ■r m 3 3
E ■r m
Li m 3
EE H m 3 E L
4 L E
4 E
7 E E E
li iM 4 E E E E E
7 L
4 Ë
4 E
E 4 EE
4 LE
0 5 10 15 20 25
Figure9. Hunting process in artificial potential field method
0 0 ÏtT < > O
a <' > o
<' > o
0 <' > ♦
o < > ♦
0 <' > ♦
0 < > o
<' > O
0 < > <>
<' > o
• ■ ■
• ■
• ■
« ■
• ■
• ■
* ■
o ■
• ■
« ■
O ( ( ( ! s
traget
5 10 15 20 25
Figure 10. Hunting process in method proposed in this paper
5 35 <
g> 30 T*
fc 15 £
Z 10 5 0
J Artificial potential method I Bio-inspired neural network method
*(n) X(n)
(a) Initial hunting state under obstacles environment task
partial enlarged detail
Pc1 Pc2 Pc3 Pc4 average
(b) Final hunting state under obstacles environment task and partial enlarged detail
Figure 12. The multi-AUV hunting simulation experiment under static obstacles
Figure11. Hunting efficiency comparison between the two methods
5. Conclusion
Cooperative hunting by multi-AUVs in an unknown environment is investigated and a bio-inspired neural network is proposed for application to the whole hunting process. By choosing the maximum activity value of the neural network of neighbouring cells, the hunting AUV will select a direct path to the moving target and finish the hunting task. The proposed approach can deal with various situations automatically and catch the moving target effectively. In addition, it can deal with hunting tasks in the environment with different shapes of obstacles. The parameters in the hunting experiment are decided by real-world applications. However, from simulation results, it can be seen that sometimes there will be a collision between AUVs. This indicates that the cooperative and collaborative mechanism among AUV team members is not built properly. Thus, further study will continue to focus on how to avoid collision between hunting AUV team members and how to complete the hunting task in a 3-D environment under the proposed method. In addition, a further important problem that needs to be discussed is the ocean current effect in the underwater environment.
6. Acknowledgements
This project is supported by the National Namral Science Foundation s>f China(5ty79098, 515e53eOu 6Seoe2S9), Creaiive AstientyPlan for Scieoceand Toc9nology Corno mi9sioo od Se^Sml ( lTJCtnC280t).
7. Rsteoences
Tt P. A. Moller, p.A.Famll, Y.Zh)e andV. Djepic, "f.^o^l^o^^h^p^oos UnderwateoVeh.cle Nsoigation/-(£££ OodnalufOcsanlcU<nginoering, Vol.30, No.P, ae-P63-T78, 2UOO.
jO] E. FicrelA, N.E.LsonarT,P.Bhatta, D. Pate°,D.R. Bacdmaye5 etd D.M. Oralentoni, "Moth-AUV conCtofand ..deceive csmuling On Monterey Bsp," AulonsmoueU9de7U88аr Oshicles, Vsl.]). No.O, 0,. 035-9Ce, 2004..
t3]R.Cui, S.SamGe, B.V. E. Htw aaR Y. Choo, "LeaPe4-fhПtwep foemeCioo control ef nette a8taaOe0ovtovomout unUsrweeecvehictes," Oarsn angineprioa,Voi.ЗP,No.l7pp0,aU■ PR91-PaSRiSeS7.
,4] Y.Yang, SiWang Z.Ptu ani YVCsu^'eMcticc planning for multi-HUG formation in an environment with obstacles," Ocean Engineering, Vol.38, No. 17-18, pp. 2262-2269, 2011.
[5] M. Liu, W. Li and P. Xuan, "Convex Optimization Algorithms for Cooperative Localization in Autonomous Underwater Vehicles," Acta Automatica Sinica, Vol.36, No.5, pp.704-710, 2010.
[6] J. Li, Q. Pan, B. Hong and M. Li, "Multi-robot Cooperative Pursuit Based on Association Rule Data Mining," Sixth International Conference on Fuzzy Systems and Knowledge Discovery, pp. 14-16, 2009.
[7] M. Wu, F. Huang, L. Wang and J. Sun, "A Distributed Multi-Robot Cooperative Hunting Algorithm Based on Limit-cycle," International Asia Conference on Informatics in Control, Automation and Robotics, pp. 156-160, 2009.
[8] R. R. McCune and G. R. Madey, "Swarm Control of UAVs for cooperative hunting with DDDAS," Procedia Computer Science, Vol. 18, pp. 2537-2544, 2013.
[9] S. Yoon and C. Qiao, "Cooperative Search and Survey Using Autonomous Underwater Vehicles (AUVs)," IEEE Transactions on Parallel and Distributed Systems, Vol.22, No.3, pp. 364-379, 2011.
[10] M. P. Aghababa, "3D path planning for underwater vehicles using five evolutionary optimization algorithms avoiding static and energetic obstacles," Applied Ocean Research, Vol. 38, pp. 48-62, 2012.
[11] G. Zhang and H. Jia, "3D path planning of AUV based on improved ant colony optimization," China Control Conference (CCC), pp. 5017-5022, 2013.
[12] D. Zhang, G. Xie, J. Yu and L. Wang, "An Adaptive task assignment for multiple mobile robots via swarm intelligence approach," IEEE International Symposium on Computational Intelligence in Robotics and Automation, pp. 415-420, 2007.
[13] Z. Liu, Q. Luo, D. Wen, S. Qiao, J. Shi and W. Zhang, "A Task Assignment Algorithm for Multiple Aerial Vehicles to Attack Targets With Dynamic Values," IEEE Transactions on Intelligent Transportation Systems, Vol.14, No.1, pp. 236-248, 2013.
[14] A. T. Tolmidis and L. Petrou, "Multi-objective optimization for dynamic task allocation in a multirobot system," Engineering Applications of Artificial Intelligence, Vol.26, No. 5-6, pp. 1458-1468, 2013.
[15] S. Hunt, Q. Meng, C. Hinde and T. Huang, "A consensus-based grouping algorithm for multiagent cooperative task allocation with complex requirements," Cognitive Computation, Vol. 6, No. 3, pp. 338-350, 2014.
[16] C. Grinton, "A Testbed for Investigation Agent Effectiveness in a Multi-agent Pursuit Game," The University of Melbourne, 1996.
[17] M. Z. Sauter, D. Shi and J. D. Kralik, "Multi-agent reinforcement learning and chimpanzee hunting," IEEE International Conference on Robotics and Biomi-metics, pp. 622-626, 2009.
[18] Z. Cai, L. Sun, H. Gao and P. Zhou, "Multi-robot Cooperative Pursuit Based on Combinatorial
Auction Mechanism Under Dynamic Environment," 2nd International Symposium on Systems and Control in Aerospace and Astronautics, pp.1-6, 2008.
[19] M. K. Nighot, V. H. Patil and G. S. Mani, "Multirobot hunting based on Swarm Intelligence," International Conference on Hybrid Intelligent Systems, pp. 203-206, 2012.
[20] S. Feng and T. Wang, "Implementation of control algorithm for interception and hunting by Amigo-Bot robots," International Conference on Mechatronic Science, Electric Engineering and Computer (MEC), pp. 2429-2433, 2011.
[21] Y. Sheng, A. H. Sayed, "Cooperative prey herding based on diffusion adaptation," IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3752-3755, 2011.
[22] H. Yamaguchi, "A distributed motion coordination strategy for multiple nonholonomic mobile robots in cooperative hunting operations," Robotics and Autonomous Systems, Vol. 43, No. 4, pp. 257-282, 2003.
[23] Y. Pan and D. Li, "Improvement with Joint Rewards on Multi-agent Cooperative Reinforcement Learning," International Conference on Computer Science and Software Engineering, pp. 536-539, 2008.
[24] Y. Ma, Z. Cao, X. Dong, C. Zhou and M. Tan, "A multi-robot coordinated hunting strategy with dynamic alliance," Control and Decision Conference, pp. 2338-2342, 2009.
[25] C. Wang, T. Zhang, K. Wang, S. Lv and H. Ma, "A new approach of multi-robot cooperative pursuit," China Control Conference (CCC), pp. 7252-7256, 2013.
[26] J. Ni and S. X. Yang, "Bio-inspired Neural Network for Real-Time Cooperative Hunting by Multi-robots in Unknown Environments," IEEE Transactions on Neural Networks, Vol.22, No.12, 2011.
[27] A. L. Hodgkin and A. F. Huxley, "A quantitative description of membrane current and its application to conduction and excitation in nerve," Journal of Physiology, Vol.117, No.4, pp. 500-544, 1952.
[28] S. Grossberg, "Nonlinear neural networks: Principles, mechanisms, and architecture," Neural Networks, Vol. 1, No.1, pp. 17-61, 1988.
[29] S. X. Yang and C. Luo, "A Neural Network Approach to Complete Coverage Path Planning," IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics, Vol.34, No.1, pp. 718-725, 2004.
[30] R. Pichevar and J. Rouat, "Binding of audio elements in the sound source segregation problem via a two-layered bio-inspired neural network," IEEE Canadian Conference on Electrical and Computer Engineering, pp. 1151-1154, 2003.
[31] M. Yan, D. Zhu, S. X. Yang, "Complete coverage path planning in an unknown underwater environment based on D-S data fusion real-time map
building," International Journal of Distributed Sensor Networks, Vol.9, No.2, pp. 1-11, 2013.
[32] C. Luo and S. X. Yang, "A Bioinspired Neural Network for Real-Time Concurrent Map Building and Complete Coverage Robot Navigation in Unknown Environments," IEEE Transactions on Neural Networks, Vol.19, No.7, pp. 1279-1298, 2008.
[33] D. Zhu, H. Huang, S. X. Yang, "Dynamic Task Assignment and path planning of Multi-AUV System Based on an Improved Self-organizing Map and Velocity Synthesis Method in 3D Underwater
Workspace," IEEE Transactions on Cybernetics, Vol. 43, No.2, pp. 504-514, 2013.
[34] N. V. Dounskaia, "Artificial potential method for control of constrained robot motion," IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics, Vol.28, No.3, pp. 447-453, 1998.
[35] E. P. Lopes, E. P. L. Aude, J. T. C. Silveira and H. Serdeira, "Obstacle avoidance strategy based on adaptive potential fields generated by an electronic stick," IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2626-2631, 2005.
