SlideShare a Scribd company logo
6
Recurrent Neural Network-Based Adaptive
Controller Design for Nonlinear
Dynamical Systems
Hong Wei Ge and Guo Zhen Tan*
College of Computer Science and Technology, Dalian University Of Technology, Dalian,
China
1. Introduction
The design goal of a control system is to influence the behavior of dynamic systems to
achieve some pre-determinate objectives. A control system is usually designed on the
premise that an accurate knowledge of a given object and environment cannot be obtained
in advance. It usually requires suitable methods to address the problems related to
uncertain and highly complicated dynamic system identification. As a matter of fact, system
identification is an important branch of research in the automatic control domain. However,
the majority of methods for system identification and parameters' adjustment are based on
linear analysis: therefore it is difficult to extend them to complex non-linear systems.
Normally, a large amount of approximations and simplifications have to be performed and,
unavoidably, they have a negative impact on the desired accuracy. Fortunately the
characteristics of the Artificial Neural Network (ANN) approach, namely non-linear
transformation and support to highly parallel operation, provide effective techniques for
system identification and control, especially for non-linear systems [1-9]. The ANN
approach has a high potential for identification and control applications mainly because: (1)
it can approximate the nonlinear input-output mapping of a dynamic system [10]; (2) it
enables to model the complex systems’ behavior and to achieve an accurate control through
training, without a priori information about the structures or parameters of systems. Due to
these characteristics, there has been a growing interest, in recent years, in the application of
neural networks to dynamic system identification and control.
“Depth” and “resolution ratio” are the main characteristics to measure the dynamic memory
performance of neural networks [11]. “Depth” denotes how far information can be
memorized; “resolution ratio” denotes how much information in input sequences of neural
networks can be retained. The memory of time-delay units is of lower depth and higher
resolution ratio, while most recurrent neural networks, such as Elman neural networks, are
higher depth and lower resolution ratio. The popular neural networks have much defect on
dynamic memory performance. This chapter proposed a novel time-delay recurrent
network model which has far more “depth” and “resolution ratio” in memory for
Corresponding author

*

www.intechopen.com
116

Recurrent Neural Networks and Soft Computing

identifying and controlling dynamic systems. The proposed identification and control
schemes are examined by the numerical experiments for identifying and controlling some
typical nonlinear systems.
The rest of this chapter is organized as follows. Section 2 proposes a novel time-delay
recurrent neural network (TDRNN) by introducing the time-delay and recurrent
mechanism; moreover, a dynamic recurrent back propagation algorithm is developed
according to the gradient descent method. Section 3 derives the optimal adaptive learning
rates to guarantee the global convergence in the sense of discrete-type Lyapunov stability.
Thereafter, the proposed identification and control schemes based on TDRNN models are
examined by numerical experiments in Section 4. Finally, some conclusions are made in
Section 5.

2. Time-delay recurrent neural network (TDRNN)
Figure 1 depicts the proposed time-delay recurrent neural network (TDRNN) by
introducing the time-delay and recurrent mechanism. In the figure, Z 1 denotes a one-step
time delay, the notation “฀” represents the memory neurons in the input layer with selffeedback gain  (0    1) , which improves the resolution ratio of the inputs.



-1

Z





z(k )

x(k )

W1

-1

Z

W2



u(k )

-1

Z





-1

Z

1




y(k )

W

3

1

Fig. 1. Architecture of the modified Elman network
It is a type of recurrent neural network with different layers of neurons, namely: input
nodes, hidden nodes, output nodes and, specific of the approach, context nodes. The input
and output nodes interact with the outside environment, whereas the hidden and context
nodes do not. The context nodes are used only to memorize previous activations of the
output nodes. The feed-forward connections are modifiable, whereas the recurrent
connections are fixed. More specifically, the proposed TDRNN possesses self-feedback links

www.intechopen.com
Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems

117

with fixed coefficient  in the context nodes. Thus the output of the context nodes can be
described by
yCl ( k )   yCl ( k  1)  yl ( k  1)

(l  1, 2, , m) .

(1)

where yCl ( k ) and y l ( k ) are, respectively, the outputs of the lth context unit and the lth

output unit and  ( 0    1 ) is the self-feedback coefficient. If we assume that there are r
nodes in the input layer, n nodes in the hidden layer, and m nodes in the output layer and
context layers respectively, then the input u is an r dimensional vector, the output x of the
hidden layer is n dimensional vector, the output y of the output layer and the output yC of

the context nodes are m dimensional vectors, and the weights W 1 , W 2 and W 3 are n  r,
m  n and m  m dimensional matrices, respectively.
The mathematical model of the proposed TDRNN can be described as follows.

y( k )  g( W 2 x( k )  W 3 yC ( k )) ,

(2)

yC ( k )   yC ( k  1)  y( k  1) ,

(3)

x( k )  f ( W 1 z( k )) ,

(4)

z( k )  u( k )    u( k  i )   z( k  1) .


i 1

(5)

where 0   ,  ,   1,     1, z(0)  0 , and  is the step number of time delay. f ( x ) is
often taken as the sigmoidal function
f (x) 

1
.
1  ex

(6)

and g( x ) is often taken as a linear function, that is
y( k )  W 2 x( k )  W 3 yC ( k ) .

Taking expansion for z( k  1) , z( k  2) ,…, z(1) by using Eq.(5), then we have
z( k )   u( k  i )    i u( k    i )   k u(0) .


k 

i 0

i 1

(7)

(8)

From Eq.(8) it can be seen that the memory neurons in the input layer include all the
previous input information and the context nodes memorize previous activations of the
output nodes, so the proposed TDRNN model has far higher memory depth than the
popular neural networks. Furthermore, the neurons in the input layer can memory
accurately the inputs from time k   to time k , and this is quite different from the memory
performance of popular recurrent neural networks. If the delay step  is moderate large, the
TDRNN possesses higher memory resolution ratio.

www.intechopen.com
118

Recurrent Neural Networks and Soft Computing

Let the kth desired output of the system be y d ( k ) . We can then define the error as
1
E  ( y d ( k )  y( k ))T ( y d ( k )  y( k )) .
2

(9)

Differentiating E with respect to W 3 , W 2 and W 1 respectively, according to the gradient
descent method, we obtain the following equations
3
wil   3 i0 yC ,l ( k )

2
3
wij  2 i0 ( x j ( k )  wii

(i  1, 2, , m ; l  1, 2, , m) ,

yC ,i ( k )
2
wij

2
w1  1   t0 wtj f j()zq ( k )
jq
n

t 1

( i  1, 2, , m ; j  1, 2, , n) ,

)

( j  1, 2, , n ; q  1, 2, , r ) .

(10)
(11)

(12)

which form the learning algorithm for the proposed TDRNN, where 1 , 2 and 3 are
learning steps of W 1 , W 2 and W 3 , respectively, and

 i0  ( y d , i ( k )  yi ( k )) g () ,
i
yC ,i ( k )
2
wij



yC ,i ( k  1)
2
wij



yi ( k  1)
2
wij

(13)
.

(14)

If g( x ) is taken as a linear function, then g ()  1 . Clearly, Eqs. (11) and (14) possess
i
recurrent characteristics.

3. Convergence of proposed time-delay recurrent neural network
In Section 2, we have proposed a TDRNN model and derived its dynamic recurrent back
propagation algorithm according to the gradient descent method. But the learning rates in the
update rules have a direct effect on the stability of dynamic systems. More specifically, a large
learning rate can make the modification of weights over large in each update step, and this will
induce non-stability and non-convergence. On the other hand, a small learning rate will induce
a lower learning efficiency. In order to train neural networks more efficiently, we propose three
criterions of selecting proper learning rates for the dynamic recurrent back propagation
algorithm based on the discrete-type Lyapunov stability analysis. The following theorems give
sufficient conditions for the convergence of the proposed TDRNN when the functions f () and
g() in Eqs. (4) and (2) are taken as sigmoidal function and linear function respectively.

Suppose that the modification of the weights of the TDRNN is determined by Eqs. (10-14).
For the convergence of the TDRNN we have the following theorems.
Theorem 1. The stable convergence of the update rule (12) on W 1 is guaranteed if the
learning rate 1 ( k ) satisfies that

www.intechopen.com
Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems

0  1 ( k ) 

8
nr max zk ( k )
k

.

119

(15)

2
max( Wij ( k ))
ij

Proof. Define the Lyapunov energy function as follows.
E( k ) 

Where

1 m 2
 ei ( k ) .
2 i 1

(16)

ei ( k )  y d ,i ( k )  y i ( k ) .

(17)

And consequently, we can obtain the modification of the Lyapunov energy function
E( k )  E( k  1)  E( k ) 

1 m 2
  ei ( k  1)  ei2 ( k ) .

2 i 1 

(18)

Then the error during the learning process can be expressed as
ei ( k  1)  ei ( k )   

n r
y ( k )
ei ( k )
1
1
W jq  ei ( k )    i 1 W jq .
1
W jq
j 1 q 1
j  1 q  1 W jq
n

r

(19)

Furthermore, the modification of weights associated with the input and hidden layers is
1
W jq ( k )  1 ( k )ei ( k )

y ( k )
ei ( k )
 1 ( k )ei ( k ) i 1 .
1
W jq
W jq

(20)

Hence, from Eqs.(18-20) we obtain

2


T
1 m 2 
 y ( k )   y ( k )  
ei ( k )  1  1 ( k )  i 1   i 1    1



2 i 1
 W   W  







2 2
y ( k )
1 m
  ei2 ( k )  1  1 ( k ) i 1   1



2 i 1
W






E( k ) 

(21)

  ei2 ( k )i1 ( k )
m

i 1

Where
 

2
y ( k ) 
1
 i1 ( k )  1   1  1 ( k ) i 1  
2






2

 
 


y ( k )
y ( k ) 
1
 1 ( k ) i 1  2  1 ( k ) i 1 


2
W
W



www.intechopen.com

2

W

2

.

(22)
120

Recurrent Neural Networks and Soft Computing

W 1 represents an n  r dimensional vector and  denotes the Euclidean norm.

Notice that the activation function of the hidden neurons in the TDRNN is the sigmoidal
type, we have 0  f ( x )  1 / 4 . Thus,
y i ( k )
1
W jq

2
 Wij ( k ) f j ' ()zq ( k ) 

1
2
max zq ( k ) max( Wij ( k ))
ij
4 q
.

(i  1, 2, , m ; j  1, 2, , n ; q  1, 2, , r )

(23)

According to the definition of the Euclidean norm we have
y( k )



W 1

nr
2
max zq ( k ) max( Wij ( k )) .
ij
4 q

Therefore, while 0  1 ( k ) 

8
nr max zq ( k )
q

2
max( Wij ( k ))
ij

(24)

, we have  i1 ( k )  0 , then from

Eq.(21) we obtain E( k )  0 . According to the Lyapunov stability theory, this shows that
the training error will converges to zero as t   . This completes the proof.

Theorem 2. The stable convergence of the update rule (11) on W 2 is guaranteed if the
learning rate 2 ( k ) satisfies that
0  2 ( k ) 

2
.
n

(25)

Proof. Similarly, the error during the learning process can be expressed as
ei ( k  1)  ei ( k )  
n

ei ( k )

2
j  1 Wij

2
Wij  ei ( k )  
n

j 1

y i ( k )
2
Wij

2
Wij .

(26)

Therefore,
2


T

 

1 m 2 
 1   ( k )  y i ( k )   y i ( k )    1
E( k )   ei ( k ) 
2
2
2


2 i 1
 Wi   Wi  

 







2 2
y ( k )
1 m


  ei2 ( k )  1   2 ( k ) i 2   1
.


2 i 1
Wi






  ei2 ( k ) i2 ( k )
m

i 1

www.intechopen.com

(27)
Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems

121

Where

 i2 ( k ) 


y ( k )
1 
1   1  2 ( k ) i 2
2 
Wi
 


2 2 






.



(28)

Notice that the activation function of the hidden neurons in the TDRNN is the sigmoidal
2
type, and neglect the dependence relation between yC ( k ) and the weights wij , we obtain
E
  i0 x j ( k ) .
2
wij

(29)

Hence,
yi ( k )
2
Wij

 x j (k)  1

(i  1, 2, , m ; j  1, 2, , n) .

(30)

According to the definition of the Euclidean norm we have

yi ( k )
Wi2

 n.

(31)

2
, we have  i2 ( k )  0 , then from Eq.(27) we obtain E( k )  0 .
n
According to the Lyapunov stability theory, this shows that the training error will converges
to zero as t   . This completes the proof.

Therefore, while 0   2 ( k ) 

Theorem 3. The stable convergence of the update rule (10) on W 3 is guaranteed if the
learning rate 3 ( k ) satisfies that

0  3 ( k ) 

2
2

.

(32)

m max( yC , l ( k ))
l

Proof. Similarly, as the above proof, we have
2


T
 yi ( k )   yi ( k )  
1 m 2 
  1
E( k )   ei ( k )  1   3 ( k ) 
3  
3 


2 i 1
 W   W  

 





2


2
y ( k ) 
1 m
.
  ei2 ( k )  1   3 ( k ) i 3   1



2 i 1
W






  ei2 ( k ) i3 ( k )
m

i 1

www.intechopen.com

(33)
122

Recurrent Neural Networks and Soft Computing

Where

 i3 ( k ) 


y ( k )
1 
1   1  3 ( k ) i 3
2 
W
 


2 2 

 .
 
 


(34)

Furthermore, according to the learning algorithm we have
ys ( k )
3
Wil

  is yC , h ( k )   is yC ,l ( k )   is max( yC ,l ( k ))
l

(i  1, 2, , m ; s  1, 2, , m ; l  1, 2, , m)

.

(35)

Where
1 i  s
.
0 i  s

 is  

(36)

According to the definition of the Euclidean norm we have
y( k )
W 3

 m max( yC ,l ( k )) .

(37)

l

Therefore, from Eq.(34), we have  i3 ( k )  0 , while 0   3 ( k ) 

2
2

. Then from

m max( yC , l ( k ))

Eq.(33) we obtain E( k )  0 . According to the Lyapunov stability theory, this shows that the
l

training error will converges to zero as t   . This completes the proof.

4. Numerical results and discussion
The performance of the proposed time-delay recurrent neural network for identifying and
controlling dynamic systems is examined by some typical test problems. We provide four
examples to illustrate the effectiveness of the proposed model and algorithm.
4.1 Nonlinear time-varying system identification

We have carried out the identification for the following nonlinear time-varying system
using the TDRNN model as an identifier.
y( k  1) 

y( k )

1  0.68sin(0.0005 k )y 2 ( k )

 0.78u3 ( k )  v( k ) .

(38)

Where v( k ) is Gauss white noise with zero mean and constant variance 0.1. The input of
system is taken as u( k )  sin(0.01 k ) .

To evaluate the performance of the proposed algorithm, the numerical results are compared
with those obtained by using Elman neural network (ENN). The Elman network is a typical

www.intechopen.com
Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems

123

recurrent network proposed by Elman [12]. Some parameters on the TDRNN in our
experiments are taken as follows. The number of hidden nodes is taken as 6, the weights are
initialized in the interval [-2, 2] randomly, besides,  ,  and  are set as 0.4, 0.6, 0.4
respectively. The number of hidden nodes in the ENN is also taken as 6.
Figure 2 shows the identification result, where the “Actual curve” is the real output curve of
the dynamic system, represented by the solid line; the “Elman curve” is the output curve
identified using the ENN model, and represented by the dash line; the “TDRNN curve” is
the output curve identified by the proposed TDRNN model, and represented by the dash
dot line. Figure 3 shows the identification error curves obtained with the TDRNN and ENN
respectively, in which the error is the absolute value of the difference between identification
result and the actual output. From the two figures it can be seen that the proposed method is
superior to the ENN method. These results demonstrate the power and potential of the
proposed TDRNN model for identifying nonlinear systems.

3.5
3.0
2.5

y( k )

2.0
1.5
1.0

Actual curve
Elman curve
TDRNN curve

0.5
0.0
-0.5
0

20

40

60

k

Fig. 2. Identification curves with different methods

www.intechopen.com

80

100
124

Recurrent Neural Networks and Soft Computing

0.08

Error

0.06

0.04

Elman error curve
TDRNN error curve

0.02

0.00

-0.02
0

20

40

60

80

100

k

Fig. 3. Comparison of error curves obtained by different methods
4.2 Bilinear DGP system control

In this section, we control the following bilinear DGP system using the TDRNN model as a
controller.
z(t )  0.5  0.4 z(t  1)  0.4 z(t  1)u(t  1)  u(t ) .

(39)

The system output at an arbitrary time is influenced by all the past information. The control
reference curves are respectively taken as:
1.

2.

Line type:

z(t )  1.0 ;

(40)

Quadrate wave:
0.0 (2 k  5  t  (2 k  1)  5, ( k  0,1, 2,))
z(t )  
1.0 ((2 k  1)  5  t  2 k  5, ( k  1, 2, 3,))

(41)

The parameters on the TDRNN in the experiments are taken as follows. The number of
hidden nodes is taken as 6, the weights are initialized in the interval [-2, 2] randomly,
besides,  ,  and  are set as 0.3, 0.6, 0.4 respectively. Figures 4 and 5 show the control
results. Figure 4 shows the control curve using the proposed TDRNN model when the
control reference is taken as a line type. Figure 5 shows the control curve when the reference
is taken as a quadrate wave type. From these results it can be seen that the proposed control
model and algorithm possess a satisfactory control precision.

www.intechopen.com
Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems

125

1.2

z( t)

1.1

Reference Curve
Control Curve

1.0

0.9

0.8

0

2

4

6

8

10

t
Fig. 4. Control curves with line type reference

1.0

z( t)

0.8
0.6

Reference Curve
Control Curve

0.4
0.2
0.0
0

5

10

15

20

25

30

t
Fig. 5. Control curves with quadrate wave reference
4.3 Inverted pendulum control

The inverted pendulum system is one of the classical examples used in many experiments
dealing with classical as well as modern control, and it is often used to test the effectiveness
of different controlling schemes [13-16]. So in this chapter, to examine the effectiveness of
the proposed TDRNN model, we investigate the application of the TDRNN to the control of
inverted pendulums.
The inverted pendulum system used here is shown in Fig.6, which is formed from a cart, a
pendulum and a rail for defining position of cart. The Pendulum is hinged on the center of
the top surface of the cart and can rotate around the pivot in the same vertical plane with the
rail. The cart can move right or left on the rail freely.

www.intechopen.com
126

Recurrent Neural Networks and Soft Computing

2l



F

m

M

x
0

-2
-3

2
3

Fig. 6. Schematic diagram of inverted pendulum system
The dynamic equation of the inverted pendulum system can be expressed as the following
two nonlinear differential equations.





g sin   cos [ F  ml 2 sin   c sgn( x )  (m  M )1 ] 

4 m cos 2 
l
l
3
m M


x




F  ml( 2 sin    cos  )  c sgn( x )
.
m M


 p

ml ,

(42)

(43)

Where the parameters, M and m are respectively the mass of the cart and the mass of the

pendulum in unit (kg), g  9.81 m s 2 is the gravity acceleration, l is the half length of the

pendulum in unit (m), F is the control force in the unit (N) applied horizontally to the
cart, uc is the friction coefficient between the cart and the rail, up is the friction

 
coefficient between the pendulum pole and the cart. The variables  ,  ,  represent the
angle between the pendulum and upright position, the angular velocity and the angular
acceleration of the pendulum, respectively. Moreover, given that clockwise direction is
 
positive. The variables x , x , x denote the displacement of the cart from the rail origin,
its velocity, its acceleration, and right direction is positive.

We use the variables  and x to control inverted pendulum system. The control goal is to
make  approach to zero by adjusting F, with the constraint condition that x is in a given
interval. The control block diagram of the inverted pendulum system is shown in Figure 7.
The TDRNN controller adopts variables  and x as two input items.

www.intechopen.com
Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems







W1

Z-1



x

Inverted
pendulum

W2

F



Z-1

127

x

1

W3

Fig. 7. Control block diagram of inverted pendulum system
In the numerical experiments, the motion of the inverted pendulum system is simulated by
numerical integral. The parameter setting is listed in the Table 1.
parameter
value

g
9.81

M
1.0

m
0.1

μc
0.002

l
0.6

μp
0.00002




φ
5°

0

x
0


x
0

Table 1. Parameter Setting of Inverted Pendulum
Besides, the number of hidden nodes is taken as 6, the weights are initialized in the interval
[-3, 3] randomly, the parameters  ,  and  on the TDRNN are set as 0.3, 0.6, 0.4
respectively. The control goals are to control the absolute value of  within 10º and make it
approximate to zero as closely as possible, with the constraint condition of the absolute
value of x within 3.0m.

6
4
2
0

-2
-4
-6
0

1

Fig. 8. Control curve of the angle 

www.intechopen.com

2

3

Time(s)

4

5
128

Recurrent Neural Networks and Soft Computing

0.12
0.10

x( m)

0.08
0.06
0.04
0.02
0.00
0

1

2

3

4

5

Time(s)

Fig. 9. Control curve of the displacement x
The control results are shown in Figures 8 and 9, and the sampling interval is taken as
T  1ms . Figures 8 and 9 respectively show the control curve of the angle  and the control
curve of the displacement x . From Figure 8, it can be seen that the fluctuation degree of 
is large at the initial stage, as time goes on, the fluctuation degree becomes smaller and
smaller, and it almost reduces to zero after 3 seconds. Figure 9 shows that the change trend
of x is similar to that of  except that it has a small slope. These results demonstrate the
proposed control scheme based on the TDRNN can effectively perform the control for
inverted pendulum system.
4.4 Ultrasonic motor control

In this section, a dynamic system of the ultrasonic motor (USM) is considered as an example
of a highly nonlinear system. The simulation and control of the USM are important
problems in the applications of the USM. According to the conventional control theory, an
accurate mathematical model should be set up. But the USM has strongly nonlinear speed
characteristics that vary with the driving conditions and its operational characteristics
depend on many factors. Therefore, it is difficult to perform effective control to the USM
using traditional methods based on mathematical models of systems. Our numerical
experiments are performed using the model of TDRNN for the speed control of a
longitudinal oscillation USM [17] shown in Figure 10.

www.intechopen.com
Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems

Piezoelectric vibrator

129

Direction of the rotation

~

Vibratory piece

Rotor

Fig. 10. Schematic diagram of the motor
Some parameters on the USM model are taken as: driving frequency 27.8 kHZ , amplitude
of driving voltage 300 V , allowed output moment 2.5 kg  cm , rotation speed 3.8 m / s .
Besides, the number of hidden nodes of the TDRNN is taken as 5, the weights are initialized
in the interval [-3, 3] randomly, the parameters  ,  and  on the TDRNN are taken as 0.4,
0.6, 0.4 respectively. The input of the TDRNN is the system control error in the last time, and
the output of the TDRNN, namely the control parameter of the USM is taken as the
frequency of the driving voltage.
Figure 11 shows the speed control curves of the USM using the three different control
strategies when the control speed is taken as 3.6 m / s . In the figure the dotted line a
represents the speed control curve based on the method presented by Senjyu et al.[18], the
solid line b represents the speed control curve using the method presented by Shi et al.[19]
and the solid line c represents the speed curve using the method proposed in this paper.
Simulation results show the stable speed control curves and the fluctuation amplitudes
obtained by using the three methods. The fluctuation degree is defined as

  (Vmax  Vmin ) / Vave  100%

(44)

where Vmax , Vmin and Vave represent the maximum, minimum and average values of the
speeds. From Figure 11 it can be seen that the fluctuation degrees when using the methods
proposed by Senjyu and Shi are 5.7% and 1.9% respectively, whereas, it is just 1.1% when
using the method in this paper. Figure 12 shows the speed control curves of the reference
speeds vary as step types. From the figures it can be seen that this method possesses good
control precision.

www.intechopen.com
130

Recurrent Neural Networks and Soft Computing

Speed( m s)
/

3.8

a

1.9%

3.7

3.6

5.7%
3.5

1.1%

b

c

a -- obtained by Senjyu
b -- obtained by Shi
c -- obtained in this paper

3.4

3.3
10.00

10.05

10.10

10.15

10.20

Time(s)

Fig. 11. Comparison of speed control curves using different schemes

Speed (m/s)

3.6

3.0
2.4

1.8
1.2
0

0.12

0.24
Time(s)

0.36

0.48

Fig. 12. Speed control curve for step type

5. Conclusions
This chapter proposes a time-delay recurrent neural network (TDRNN) with better
performance in memory than popular neural networks by employing the time-delay and
recurrent mechanism. Subsequently, the dynamic recurrent back propagation algorithm for
the TDRNN is developed according to the gradient descent method. Furthermore, to train
neural networks more efficiently, we propose three criterions of selecting proper learning
rates for the dynamic recurrent back propagation algorithm based on the discrete-type
Lyapunov stability analysis. Besides, based on the TDRNN model, we have described,
analyzed and discussed an identifier and an adaptive controller designed to identify and

www.intechopen.com
Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems

131

control nonlinear systems. Our numerical experiments show that the TDRNN has good
effectiveness in the identification and control for nonlinear systems. It indicates that the
methods described in this chapter can provide effective approaches for nonlinear dynamic
systems identification and control.

6. Acknowledgment
The authors are grateful to the support of the National Natural Science Foundation of China
(61103146) and (60873256), the Fundamental Research Funds for the Central Universities
(DUT11SX03).

7. References
[1] M. Han, J.C. Fan and J. Wang, A Dynamic Feed-forward Neural Network Based on
Gaussian Particle Swarm Optimization and its Application for Predictive Control,
IEEE Transactions on Neural Networks, 22 (2011) 1457-1468.
[2] G. Puscasu and B. Codres, Nonlinear System Identification and Control Based on
Modular Neural Networks, International Journal of Neural Systems, 21(2011), 319334.
[3] L.Li, G. Song, and J.Ou, Nonlinear Structural Vibration Suppression Using Dynamic
Neural Network Observer and Adaptive Fuzzy Sliding Mode Control. Journal of
Vibration and Control, 16 (2010), 1503-1526.
[4] T. Hayakawa, W.M. Haddad, J.M. Bailey and N. Hovakimyan, Passivity-based Neural
Network Adaptive Output Feedback Control for Nonlinear Nonnegative
Dynamical Systems, IEEE Transactions on Neural Networks, 16 (2005) 387-398.
[5] M. Sunar, A.M.A. Gurain and M. Mohandes, Substructural Neural Network Controller,
Computers & Structures, 78 (2000) 575-581.
[6] D. Wang and J. Huang, Neural Network-based Adaptive Dynamic Surface Control for a
Class of Uncertain Nonlinear Systems in Strict-feedback Form, IEEE Transactions
on Neural Networks, 16 (2005) 195-202.
[7] Y.M. Li, Y.G. Liu and X.P. Liu, Active Vibration Control of a Modular Robot Combining
a Back-propagation Neural Network with a Genetic Algorithm, Journal of
Vibration and Control, 11 (2005) 3-17.
[8] J.C. Patra and A.C. Kot, Nonlinear Dynamic System Identification Using Chebyshev
Functional Link Artificial Neural Networks, IEEE Transactions on Systems, Man,
and Cybernetics, Part B: Cybernetics, 32 (2002) 505-511.
[9] R.J. Wai, Hybrid Fuzzy Neural-network Control for Nonlinear Motor-toggle
Servomechanism, IEEE Transactions on Control Systems Technology, 10 (2002) 519532.
[10] G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control
Signals and System, 2 (1989) 303–314.
[11] S. Haykin, Neural Networks: A Comprehensive Foundation (Englewood Cliffs, NJ:
Prentice Hall, 1999).
[12] J.L. Elman, Finding Structure in Time, Cognitive Science, 14 (1990) 179-211.
[13] C.H. Chiu, Y.F. Peng, and Y.W. Lin, Intelligent backstepping control for wheeled
inverted pendulum, Expert Systems With Applications, 38 (2011) 3364-3371.

www.intechopen.com
132

Recurrent Neural Networks and Soft Computing

[14] M.I. El-Hawwary, A.L. Elshafei and H.M. Emara, Adaptive Fuzzy Control of the
Inverted Pendulum Problem, IEEE Transactions on Control Systems Technology,
14 (2006) 1135-1144.
[15] P.J. Gawthrop and L.P. Wang, Intermittent Predictive Control of An Inverted
Pendulum, Control Engineering Practice, 14 (2006) 1347-1356.
[16] R.J. Wai and L.J. Chang, Stabilizing and Tracking Control of Nonlinear Dual-axis
Inverted-pendulum System Using Fuzzy Neural Network, IEEE Transactions on
Fuzzy systems, 14 (2006) 145-168.
[17] X. Xu, Y.C. Liang, H.P. Lee, W.Z. Lin, S.P. Lim and K.H. Lee, Mechanical modeling of a
longitudinal oscillation ultrasonic motor and temperature effect analysis, Smart
Materials and Structures, 12 (2003) 514-523.
[18] T. Senjyu, H. Miyazato, S. Yokoda, and K. Uezato, Speed control of ultrasonic motors
using neural network, IEEE Transactions on Power Electronics, 13 (1998) 381-387.
[19] X.H. Shi, Y.C. Liang, H.P. Lee, W.Z. Lin, X.Xu and S.P. Lim, Improved Elman networks
and applications for controlling ultrasonic motors, Applied Artificial Intelligence 18
(2004) 603-629.

www.intechopen.com
Recurrent Neural Networks and Soft Computing
Edited by Dr. Mahmoud ElHefnawi

ISBN 978-953-51-0409-4
Hard cover, 290 pages
Publisher InTech

Published online 30, March, 2012

Published in print edition March, 2012

How to reference

In order to correctly reference this scholarly work, feel free to copy and paste the following:
Hong Wei Ge and Guo Zhen Tan (2012). Recurrent Neural Network-Based Adaptive Controller Design For
Nonlinear Dynamical Systems, Recurrent Neural Networks and Soft Computing, Dr. Mahmoud ElHefnawi (Ed.),
ISBN: 978-953-51-0409-4, InTech, Available from: http://www.intechopen.com/books/recurrent-neuralnetworks-and-soft-computing/recurrent-neural-network-based-adaptive-controller-design-for-nonlinearsystems

InTech Europe

University Campus STeP Ri
Slavka Krautzeka 83/A
51000 Rijeka, Croatia
Phone: +385 (51) 770 447
Fax: +385 (51) 686 166
www.intechopen.com

InTech China

Unit 405, Office Block, Hotel Equatorial Shanghai
No.65, Yan An Road (West), Shanghai, 200040, China
Phone: +86-21-62489820
Fax: +86-21-62489821

More Related Content

What's hot (18)

PDF
neural networksNnf
Sandilya Sridhara
 
PDF
Neural Networks: Rosenblatt's Perceptron
Mostafa G. M. Mostafa
 
PDF
The impact of visual saliency prediction in image classification
Universitat Politècnica de Catalunya
 
PPTX
Unit ii supervised ii
Indira Priyadarsini
 
PPTX
Batch normalization presentation
Owin Will
 
PDF
Two Bit Arithmetic Logic Unit (ALU) in QCA
idescitation
 
PDF
Neural Networks: Introducton
Mostafa G. M. Mostafa
 
PDF
Boundness of a neural network weights using the notion of a limit of a sequence
IJDKP
 
PDF
Quantum brain a recurrent quantum neural network model to describe eye trac...
Elsa von Licy
 
PPTX
Introduction to Neural networks (under graduate course) Lecture 9 of 9
Randa Elanwar
 
PDF
Neural Networks: Self-Organizing Maps (SOM)
Mostafa G. M. Mostafa
 
PPTX
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
Simplilearn
 
PPTX
Unit iii update
Indira Priyadarsini
 
PDF
Paper id 26201482
IJRAT
 
PDF
Spiking Neural Networks As Continuous-Time Dynamical Systems: Fundamentals, E...
IDES Editor
 
PDF
RNN and its applications
Sungjoon Choi
 
PDF
Artificial Neural Networks Lect2: Neurobiology & Architectures of ANNS
Mohammed Bennamoun
 
PDF
From neural networks to deep learning
Viet-Trung TRAN
 
neural networksNnf
Sandilya Sridhara
 
Neural Networks: Rosenblatt's Perceptron
Mostafa G. M. Mostafa
 
The impact of visual saliency prediction in image classification
Universitat Politècnica de Catalunya
 
Unit ii supervised ii
Indira Priyadarsini
 
Batch normalization presentation
Owin Will
 
Two Bit Arithmetic Logic Unit (ALU) in QCA
idescitation
 
Neural Networks: Introducton
Mostafa G. M. Mostafa
 
Boundness of a neural network weights using the notion of a limit of a sequence
IJDKP
 
Quantum brain a recurrent quantum neural network model to describe eye trac...
Elsa von Licy
 
Introduction to Neural networks (under graduate course) Lecture 9 of 9
Randa Elanwar
 
Neural Networks: Self-Organizing Maps (SOM)
Mostafa G. M. Mostafa
 
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
Simplilearn
 
Unit iii update
Indira Priyadarsini
 
Paper id 26201482
IJRAT
 
Spiking Neural Networks As Continuous-Time Dynamical Systems: Fundamentals, E...
IDES Editor
 
RNN and its applications
Sungjoon Choi
 
Artificial Neural Networks Lect2: Neurobiology & Architectures of ANNS
Mohammed Bennamoun
 
From neural networks to deep learning
Viet-Trung TRAN
 

Similar to Neural Network Dynamical Systems (20)

PDF
Analysis of intelligent system design by neuro adaptive control no restriction
iaemedu
 
PDF
Analysis of intelligent system design by neuro adaptive control
iaemedu
 
PPT
02-11-2005.ppt
GeraudRusselGouneChe1
 
PDF
505 260-266
idescitation
 
PDF
JAISTサマースクール2016「脳を知るための理論」講義04 Neural Networks and Neuroscience
hirokazutanaka
 
DOCX
introduction to machine learning unit III
GayathriRHICETCSESTA
 
PDF
NETWORK LEARNING AND TRAINING OF A CASCADED LINK-BASED FEED FORWARD NEURAL NE...
ijaia
 
PDF
Optimization of Number of Neurons in the Hidden Layer in Feed Forward Neural ...
IJERA Editor
 
PDF
Artificial Neural Network Seminar Report
Todd Turner
 
PDF
test
guest52789d
 
PPTX
Deep learning (2)
Muhanad Al-khalisy
 
PDF
Introduction to Recurrent Neural Network
Knoldus Inc.
 
PDF
ANN based STLF of Power System
Yousuf Khan
 
PDF
5ACChapter5DeepLearning23EN UC Coimbra PT
antoniodouradopc
 
PDF
Ijciet 10 01_153-2
IAEME Publication
 
PDF
Artificial Neural Networks In Control And Optimization
Jim Webb
 
PDF
Recurrent Time-Delay Neural Network - Fabio Greco - ICASSP 91
Fabio Greco
 
PPT
Artificial neutral network cousre of AI.ppt
attaurahman
 
PPT
lec10newwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
khushbu maurya
 
PDF
A New Classifier Based onRecurrent Neural Network Using Multiple Binary-Outpu...
iosrjce
 
Analysis of intelligent system design by neuro adaptive control no restriction
iaemedu
 
Analysis of intelligent system design by neuro adaptive control
iaemedu
 
02-11-2005.ppt
GeraudRusselGouneChe1
 
505 260-266
idescitation
 
JAISTサマースクール2016「脳を知るための理論」講義04 Neural Networks and Neuroscience
hirokazutanaka
 
introduction to machine learning unit III
GayathriRHICETCSESTA
 
NETWORK LEARNING AND TRAINING OF A CASCADED LINK-BASED FEED FORWARD NEURAL NE...
ijaia
 
Optimization of Number of Neurons in the Hidden Layer in Feed Forward Neural ...
IJERA Editor
 
Artificial Neural Network Seminar Report
Todd Turner
 
Deep learning (2)
Muhanad Al-khalisy
 
Introduction to Recurrent Neural Network
Knoldus Inc.
 
ANN based STLF of Power System
Yousuf Khan
 
5ACChapter5DeepLearning23EN UC Coimbra PT
antoniodouradopc
 
Ijciet 10 01_153-2
IAEME Publication
 
Artificial Neural Networks In Control And Optimization
Jim Webb
 
Recurrent Time-Delay Neural Network - Fabio Greco - ICASSP 91
Fabio Greco
 
Artificial neutral network cousre of AI.ppt
attaurahman
 
lec10newwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
khushbu maurya
 
A New Classifier Based onRecurrent Neural Network Using Multiple Binary-Outpu...
iosrjce
 
Ad

More from M Reza Rahmati (20)

PDF
Geometric Control System and Fault-Diagnosis
M Reza Rahmati
 
PDF
Fault detection and_diagnosis
M Reza Rahmati
 
PDF
Wind river NASA Real-Time Constrained Control Systems
M Reza Rahmati
 
PDF
Differential geometry vision systems
M Reza Rahmati
 
PDF
Automata based programming
M Reza Rahmati
 
PDF
Component Based Control System Design
M Reza Rahmati
 
PDF
Robust reachability analysis NASA
M Reza Rahmati
 
PDF
Reachability Analysis Control of Non-Linear Dynamical Systems
M Reza Rahmati
 
PDF
Reachability Analysis "Control Of Dynamical Non-Linear Systems"
M Reza Rahmati
 
PDF
Symbolic Computation and Automated Reasoning in Differential Geometry
M Reza Rahmati
 
PDF
Modeling and Prototyping Physical Neural Network and On-Line Learning Perform...
M Reza Rahmati
 
PDF
HyperSpace
M Reza Rahmati
 
PDF
Planning and Control Algorithms Model-Based Approach (State-Space)
M Reza Rahmati
 
PDF
Geometrical control theory
M Reza Rahmati
 
PDF
Optimal Control System Design
M Reza Rahmati
 
PDF
Geometry methods dynamical_system_theory
M Reza Rahmati
 
PDF
The Existence and Laws of Our Universe
M Reza Rahmati
 
PDF
Modelling Time in Computation (Dynamic Systems)
M Reza Rahmati
 
PDF
Finite Elements libmesh
M Reza Rahmati
 
PPTX
Trilinos progress, challenges and future plans
M Reza Rahmati
 
Geometric Control System and Fault-Diagnosis
M Reza Rahmati
 
Fault detection and_diagnosis
M Reza Rahmati
 
Wind river NASA Real-Time Constrained Control Systems
M Reza Rahmati
 
Differential geometry vision systems
M Reza Rahmati
 
Automata based programming
M Reza Rahmati
 
Component Based Control System Design
M Reza Rahmati
 
Robust reachability analysis NASA
M Reza Rahmati
 
Reachability Analysis Control of Non-Linear Dynamical Systems
M Reza Rahmati
 
Reachability Analysis "Control Of Dynamical Non-Linear Systems"
M Reza Rahmati
 
Symbolic Computation and Automated Reasoning in Differential Geometry
M Reza Rahmati
 
Modeling and Prototyping Physical Neural Network and On-Line Learning Perform...
M Reza Rahmati
 
HyperSpace
M Reza Rahmati
 
Planning and Control Algorithms Model-Based Approach (State-Space)
M Reza Rahmati
 
Geometrical control theory
M Reza Rahmati
 
Optimal Control System Design
M Reza Rahmati
 
Geometry methods dynamical_system_theory
M Reza Rahmati
 
The Existence and Laws of Our Universe
M Reza Rahmati
 
Modelling Time in Computation (Dynamic Systems)
M Reza Rahmati
 
Finite Elements libmesh
M Reza Rahmati
 
Trilinos progress, challenges and future plans
M Reza Rahmati
 
Ad

Recently uploaded (20)

PDF
How Startups Are Growing Faster with App Developers in Australia.pdf
India App Developer
 
PDF
HubSpot Main Hub: A Unified Growth Platform
Jaswinder Singh
 
PDF
Achieving Consistent and Reliable AI Code Generation - Medusa AI
medusaaico
 
PDF
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
PDF
July Patch Tuesday
Ivanti
 
PDF
Blockchain Transactions Explained For Everyone
CIFDAQ
 
PDF
Fl Studio 24.2.2 Build 4597 Crack for Windows Free Download 2025
faizk77g
 
PDF
Windsurf Meetup Ottawa 2025-07-12 - Planning Mode at Reliza.pdf
Pavel Shukhman
 
PPTX
Top iOS App Development Company in the USA for Innovative Apps
SynapseIndia
 
PDF
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
PDF
Predicting the unpredictable: re-engineering recommendation algorithms for fr...
Speck&Tech
 
PDF
Building Resilience with Digital Twins : Lessons from Korea
SANGHEE SHIN
 
PPTX
Building Search Using OpenSearch: Limitations and Workarounds
Sease
 
PDF
Persuasive AI: risks and opportunities in the age of digital debate
Speck&Tech
 
PDF
Smart Air Quality Monitoring with Serrax AQM190 LITE
SERRAX TECHNOLOGIES LLP
 
PPTX
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
PDF
Complete Network Protection with Real-Time Security
L4RGINDIA
 
PDF
Presentation - Vibe Coding The Future of Tech
yanuarsinggih1
 
PDF
SWEBOK Guide and Software Services Engineering Education
Hironori Washizaki
 
PPTX
MSP360 Backup Scheduling and Retention Best Practices.pptx
MSP360
 
How Startups Are Growing Faster with App Developers in Australia.pdf
India App Developer
 
HubSpot Main Hub: A Unified Growth Platform
Jaswinder Singh
 
Achieving Consistent and Reliable AI Code Generation - Medusa AI
medusaaico
 
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
July Patch Tuesday
Ivanti
 
Blockchain Transactions Explained For Everyone
CIFDAQ
 
Fl Studio 24.2.2 Build 4597 Crack for Windows Free Download 2025
faizk77g
 
Windsurf Meetup Ottawa 2025-07-12 - Planning Mode at Reliza.pdf
Pavel Shukhman
 
Top iOS App Development Company in the USA for Innovative Apps
SynapseIndia
 
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
Predicting the unpredictable: re-engineering recommendation algorithms for fr...
Speck&Tech
 
Building Resilience with Digital Twins : Lessons from Korea
SANGHEE SHIN
 
Building Search Using OpenSearch: Limitations and Workarounds
Sease
 
Persuasive AI: risks and opportunities in the age of digital debate
Speck&Tech
 
Smart Air Quality Monitoring with Serrax AQM190 LITE
SERRAX TECHNOLOGIES LLP
 
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
Complete Network Protection with Real-Time Security
L4RGINDIA
 
Presentation - Vibe Coding The Future of Tech
yanuarsinggih1
 
SWEBOK Guide and Software Services Engineering Education
Hironori Washizaki
 
MSP360 Backup Scheduling and Retention Best Practices.pptx
MSP360
 

Neural Network Dynamical Systems

  • 1. 6 Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems Hong Wei Ge and Guo Zhen Tan* College of Computer Science and Technology, Dalian University Of Technology, Dalian, China 1. Introduction The design goal of a control system is to influence the behavior of dynamic systems to achieve some pre-determinate objectives. A control system is usually designed on the premise that an accurate knowledge of a given object and environment cannot be obtained in advance. It usually requires suitable methods to address the problems related to uncertain and highly complicated dynamic system identification. As a matter of fact, system identification is an important branch of research in the automatic control domain. However, the majority of methods for system identification and parameters' adjustment are based on linear analysis: therefore it is difficult to extend them to complex non-linear systems. Normally, a large amount of approximations and simplifications have to be performed and, unavoidably, they have a negative impact on the desired accuracy. Fortunately the characteristics of the Artificial Neural Network (ANN) approach, namely non-linear transformation and support to highly parallel operation, provide effective techniques for system identification and control, especially for non-linear systems [1-9]. The ANN approach has a high potential for identification and control applications mainly because: (1) it can approximate the nonlinear input-output mapping of a dynamic system [10]; (2) it enables to model the complex systems’ behavior and to achieve an accurate control through training, without a priori information about the structures or parameters of systems. Due to these characteristics, there has been a growing interest, in recent years, in the application of neural networks to dynamic system identification and control. “Depth” and “resolution ratio” are the main characteristics to measure the dynamic memory performance of neural networks [11]. “Depth” denotes how far information can be memorized; “resolution ratio” denotes how much information in input sequences of neural networks can be retained. The memory of time-delay units is of lower depth and higher resolution ratio, while most recurrent neural networks, such as Elman neural networks, are higher depth and lower resolution ratio. The popular neural networks have much defect on dynamic memory performance. This chapter proposed a novel time-delay recurrent network model which has far more “depth” and “resolution ratio” in memory for Corresponding author * www.intechopen.com
  • 2. 116 Recurrent Neural Networks and Soft Computing identifying and controlling dynamic systems. The proposed identification and control schemes are examined by the numerical experiments for identifying and controlling some typical nonlinear systems. The rest of this chapter is organized as follows. Section 2 proposes a novel time-delay recurrent neural network (TDRNN) by introducing the time-delay and recurrent mechanism; moreover, a dynamic recurrent back propagation algorithm is developed according to the gradient descent method. Section 3 derives the optimal adaptive learning rates to guarantee the global convergence in the sense of discrete-type Lyapunov stability. Thereafter, the proposed identification and control schemes based on TDRNN models are examined by numerical experiments in Section 4. Finally, some conclusions are made in Section 5. 2. Time-delay recurrent neural network (TDRNN) Figure 1 depicts the proposed time-delay recurrent neural network (TDRNN) by introducing the time-delay and recurrent mechanism. In the figure, Z 1 denotes a one-step time delay, the notation “฀” represents the memory neurons in the input layer with selffeedback gain  (0    1) , which improves the resolution ratio of the inputs.  -1 Z   z(k ) x(k ) W1 -1 Z W2  u(k ) -1 Z   -1 Z 1   y(k ) W 3 1 Fig. 1. Architecture of the modified Elman network It is a type of recurrent neural network with different layers of neurons, namely: input nodes, hidden nodes, output nodes and, specific of the approach, context nodes. The input and output nodes interact with the outside environment, whereas the hidden and context nodes do not. The context nodes are used only to memorize previous activations of the output nodes. The feed-forward connections are modifiable, whereas the recurrent connections are fixed. More specifically, the proposed TDRNN possesses self-feedback links www.intechopen.com
  • 3. Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 117 with fixed coefficient  in the context nodes. Thus the output of the context nodes can be described by yCl ( k )   yCl ( k  1)  yl ( k  1) (l  1, 2, , m) . (1) where yCl ( k ) and y l ( k ) are, respectively, the outputs of the lth context unit and the lth output unit and  ( 0    1 ) is the self-feedback coefficient. If we assume that there are r nodes in the input layer, n nodes in the hidden layer, and m nodes in the output layer and context layers respectively, then the input u is an r dimensional vector, the output x of the hidden layer is n dimensional vector, the output y of the output layer and the output yC of the context nodes are m dimensional vectors, and the weights W 1 , W 2 and W 3 are n  r, m  n and m  m dimensional matrices, respectively. The mathematical model of the proposed TDRNN can be described as follows. y( k )  g( W 2 x( k )  W 3 yC ( k )) , (2) yC ( k )   yC ( k  1)  y( k  1) , (3) x( k )  f ( W 1 z( k )) , (4) z( k )  u( k )    u( k  i )   z( k  1) .  i 1 (5) where 0   ,  ,   1,     1, z(0)  0 , and  is the step number of time delay. f ( x ) is often taken as the sigmoidal function f (x)  1 . 1  ex (6) and g( x ) is often taken as a linear function, that is y( k )  W 2 x( k )  W 3 yC ( k ) . Taking expansion for z( k  1) , z( k  2) ,…, z(1) by using Eq.(5), then we have z( k )   u( k  i )    i u( k    i )   k u(0) .  k  i 0 i 1 (7) (8) From Eq.(8) it can be seen that the memory neurons in the input layer include all the previous input information and the context nodes memorize previous activations of the output nodes, so the proposed TDRNN model has far higher memory depth than the popular neural networks. Furthermore, the neurons in the input layer can memory accurately the inputs from time k   to time k , and this is quite different from the memory performance of popular recurrent neural networks. If the delay step  is moderate large, the TDRNN possesses higher memory resolution ratio. www.intechopen.com
  • 4. 118 Recurrent Neural Networks and Soft Computing Let the kth desired output of the system be y d ( k ) . We can then define the error as 1 E  ( y d ( k )  y( k ))T ( y d ( k )  y( k )) . 2 (9) Differentiating E with respect to W 3 , W 2 and W 1 respectively, according to the gradient descent method, we obtain the following equations 3 wil   3 i0 yC ,l ( k ) 2 3 wij  2 i0 ( x j ( k )  wii (i  1, 2, , m ; l  1, 2, , m) , yC ,i ( k ) 2 wij 2 w1  1   t0 wtj f j()zq ( k ) jq n t 1 ( i  1, 2, , m ; j  1, 2, , n) , ) ( j  1, 2, , n ; q  1, 2, , r ) . (10) (11) (12) which form the learning algorithm for the proposed TDRNN, where 1 , 2 and 3 are learning steps of W 1 , W 2 and W 3 , respectively, and  i0  ( y d , i ( k )  yi ( k )) g () , i yC ,i ( k ) 2 wij  yC ,i ( k  1) 2 wij  yi ( k  1) 2 wij (13) . (14) If g( x ) is taken as a linear function, then g ()  1 . Clearly, Eqs. (11) and (14) possess i recurrent characteristics. 3. Convergence of proposed time-delay recurrent neural network In Section 2, we have proposed a TDRNN model and derived its dynamic recurrent back propagation algorithm according to the gradient descent method. But the learning rates in the update rules have a direct effect on the stability of dynamic systems. More specifically, a large learning rate can make the modification of weights over large in each update step, and this will induce non-stability and non-convergence. On the other hand, a small learning rate will induce a lower learning efficiency. In order to train neural networks more efficiently, we propose three criterions of selecting proper learning rates for the dynamic recurrent back propagation algorithm based on the discrete-type Lyapunov stability analysis. The following theorems give sufficient conditions for the convergence of the proposed TDRNN when the functions f () and g() in Eqs. (4) and (2) are taken as sigmoidal function and linear function respectively. Suppose that the modification of the weights of the TDRNN is determined by Eqs. (10-14). For the convergence of the TDRNN we have the following theorems. Theorem 1. The stable convergence of the update rule (12) on W 1 is guaranteed if the learning rate 1 ( k ) satisfies that www.intechopen.com
  • 5. Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 0  1 ( k )  8 nr max zk ( k ) k . 119 (15) 2 max( Wij ( k )) ij Proof. Define the Lyapunov energy function as follows. E( k )  Where 1 m 2  ei ( k ) . 2 i 1 (16) ei ( k )  y d ,i ( k )  y i ( k ) . (17) And consequently, we can obtain the modification of the Lyapunov energy function E( k )  E( k  1)  E( k )  1 m 2   ei ( k  1)  ei2 ( k ) .  2 i 1  (18) Then the error during the learning process can be expressed as ei ( k  1)  ei ( k )    n r y ( k ) ei ( k ) 1 1 W jq  ei ( k )    i 1 W jq . 1 W jq j 1 q 1 j  1 q  1 W jq n r (19) Furthermore, the modification of weights associated with the input and hidden layers is 1 W jq ( k )  1 ( k )ei ( k ) y ( k ) ei ( k )  1 ( k )ei ( k ) i 1 . 1 W jq W jq (20) Hence, from Eqs.(18-20) we obtain 2   T 1 m 2   y ( k )   y ( k )   ei ( k )  1  1 ( k )  i 1   i 1    1    2 i 1  W   W          2 2 y ( k ) 1 m   ei2 ( k )  1  1 ( k ) i 1   1    2 i 1 W      E( k )  (21)   ei2 ( k )i1 ( k ) m i 1 Where    2 y ( k )  1  i1 ( k )  1   1  1 ( k ) i 1   2     2      y ( k ) y ( k )  1  1 ( k ) i 1  2  1 ( k ) i 1    2 W W   www.intechopen.com 2 W 2 . (22)
  • 6. 120 Recurrent Neural Networks and Soft Computing W 1 represents an n  r dimensional vector and  denotes the Euclidean norm. Notice that the activation function of the hidden neurons in the TDRNN is the sigmoidal type, we have 0  f ( x )  1 / 4 . Thus, y i ( k ) 1 W jq 2  Wij ( k ) f j ' ()zq ( k )  1 2 max zq ( k ) max( Wij ( k )) ij 4 q . (i  1, 2, , m ; j  1, 2, , n ; q  1, 2, , r ) (23) According to the definition of the Euclidean norm we have y( k )  W 1 nr 2 max zq ( k ) max( Wij ( k )) . ij 4 q Therefore, while 0  1 ( k )  8 nr max zq ( k ) q 2 max( Wij ( k )) ij (24) , we have  i1 ( k )  0 , then from Eq.(21) we obtain E( k )  0 . According to the Lyapunov stability theory, this shows that the training error will converges to zero as t   . This completes the proof. Theorem 2. The stable convergence of the update rule (11) on W 2 is guaranteed if the learning rate 2 ( k ) satisfies that 0  2 ( k )  2 . n (25) Proof. Similarly, the error during the learning process can be expressed as ei ( k  1)  ei ( k )   n ei ( k ) 2 j  1 Wij 2 Wij  ei ( k )   n j 1 y i ( k ) 2 Wij 2 Wij . (26) Therefore, 2   T     1 m 2   1   ( k )  y i ( k )   y i ( k )    1 E( k )   ei ( k )  2 2 2   2 i 1  Wi   Wi             2 2 y ( k ) 1 m     ei2 ( k )  1   2 ( k ) i 2   1 .   2 i 1 Wi        ei2 ( k ) i2 ( k ) m i 1 www.intechopen.com (27)
  • 7. Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 121 Where  i2 ( k )   y ( k ) 1  1   1  2 ( k ) i 2 2  Wi    2 2      .   (28) Notice that the activation function of the hidden neurons in the TDRNN is the sigmoidal 2 type, and neglect the dependence relation between yC ( k ) and the weights wij , we obtain E   i0 x j ( k ) . 2 wij (29) Hence, yi ( k ) 2 Wij  x j (k)  1 (i  1, 2, , m ; j  1, 2, , n) . (30) According to the definition of the Euclidean norm we have yi ( k ) Wi2  n. (31) 2 , we have  i2 ( k )  0 , then from Eq.(27) we obtain E( k )  0 . n According to the Lyapunov stability theory, this shows that the training error will converges to zero as t   . This completes the proof. Therefore, while 0   2 ( k )  Theorem 3. The stable convergence of the update rule (10) on W 3 is guaranteed if the learning rate 3 ( k ) satisfies that 0  3 ( k )  2 2 . (32) m max( yC , l ( k )) l Proof. Similarly, as the above proof, we have 2   T  yi ( k )   yi ( k )   1 m 2    1 E( k )   ei ( k )  1   3 ( k )  3   3    2 i 1  W   W           2   2 y ( k )  1 m .   ei2 ( k )  1   3 ( k ) i 3   1    2 i 1 W        ei2 ( k ) i3 ( k ) m i 1 www.intechopen.com (33)
  • 8. 122 Recurrent Neural Networks and Soft Computing Where  i3 ( k )   y ( k ) 1  1   1  3 ( k ) i 3 2  W    2 2   .      (34) Furthermore, according to the learning algorithm we have ys ( k ) 3 Wil   is yC , h ( k )   is yC ,l ( k )   is max( yC ,l ( k )) l (i  1, 2, , m ; s  1, 2, , m ; l  1, 2, , m) . (35) Where 1 i  s . 0 i  s  is   (36) According to the definition of the Euclidean norm we have y( k ) W 3  m max( yC ,l ( k )) . (37) l Therefore, from Eq.(34), we have  i3 ( k )  0 , while 0   3 ( k )  2 2 . Then from m max( yC , l ( k )) Eq.(33) we obtain E( k )  0 . According to the Lyapunov stability theory, this shows that the l training error will converges to zero as t   . This completes the proof. 4. Numerical results and discussion The performance of the proposed time-delay recurrent neural network for identifying and controlling dynamic systems is examined by some typical test problems. We provide four examples to illustrate the effectiveness of the proposed model and algorithm. 4.1 Nonlinear time-varying system identification We have carried out the identification for the following nonlinear time-varying system using the TDRNN model as an identifier. y( k  1)  y( k ) 1  0.68sin(0.0005 k )y 2 ( k )  0.78u3 ( k )  v( k ) . (38) Where v( k ) is Gauss white noise with zero mean and constant variance 0.1. The input of system is taken as u( k )  sin(0.01 k ) . To evaluate the performance of the proposed algorithm, the numerical results are compared with those obtained by using Elman neural network (ENN). The Elman network is a typical www.intechopen.com
  • 9. Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 123 recurrent network proposed by Elman [12]. Some parameters on the TDRNN in our experiments are taken as follows. The number of hidden nodes is taken as 6, the weights are initialized in the interval [-2, 2] randomly, besides,  ,  and  are set as 0.4, 0.6, 0.4 respectively. The number of hidden nodes in the ENN is also taken as 6. Figure 2 shows the identification result, where the “Actual curve” is the real output curve of the dynamic system, represented by the solid line; the “Elman curve” is the output curve identified using the ENN model, and represented by the dash line; the “TDRNN curve” is the output curve identified by the proposed TDRNN model, and represented by the dash dot line. Figure 3 shows the identification error curves obtained with the TDRNN and ENN respectively, in which the error is the absolute value of the difference between identification result and the actual output. From the two figures it can be seen that the proposed method is superior to the ENN method. These results demonstrate the power and potential of the proposed TDRNN model for identifying nonlinear systems. 3.5 3.0 2.5 y( k ) 2.0 1.5 1.0 Actual curve Elman curve TDRNN curve 0.5 0.0 -0.5 0 20 40 60 k Fig. 2. Identification curves with different methods www.intechopen.com 80 100
  • 10. 124 Recurrent Neural Networks and Soft Computing 0.08 Error 0.06 0.04 Elman error curve TDRNN error curve 0.02 0.00 -0.02 0 20 40 60 80 100 k Fig. 3. Comparison of error curves obtained by different methods 4.2 Bilinear DGP system control In this section, we control the following bilinear DGP system using the TDRNN model as a controller. z(t )  0.5  0.4 z(t  1)  0.4 z(t  1)u(t  1)  u(t ) . (39) The system output at an arbitrary time is influenced by all the past information. The control reference curves are respectively taken as: 1. 2. Line type: z(t )  1.0 ; (40) Quadrate wave: 0.0 (2 k  5  t  (2 k  1)  5, ( k  0,1, 2,)) z(t )   1.0 ((2 k  1)  5  t  2 k  5, ( k  1, 2, 3,)) (41) The parameters on the TDRNN in the experiments are taken as follows. The number of hidden nodes is taken as 6, the weights are initialized in the interval [-2, 2] randomly, besides,  ,  and  are set as 0.3, 0.6, 0.4 respectively. Figures 4 and 5 show the control results. Figure 4 shows the control curve using the proposed TDRNN model when the control reference is taken as a line type. Figure 5 shows the control curve when the reference is taken as a quadrate wave type. From these results it can be seen that the proposed control model and algorithm possess a satisfactory control precision. www.intechopen.com
  • 11. Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 125 1.2 z( t) 1.1 Reference Curve Control Curve 1.0 0.9 0.8 0 2 4 6 8 10 t Fig. 4. Control curves with line type reference 1.0 z( t) 0.8 0.6 Reference Curve Control Curve 0.4 0.2 0.0 0 5 10 15 20 25 30 t Fig. 5. Control curves with quadrate wave reference 4.3 Inverted pendulum control The inverted pendulum system is one of the classical examples used in many experiments dealing with classical as well as modern control, and it is often used to test the effectiveness of different controlling schemes [13-16]. So in this chapter, to examine the effectiveness of the proposed TDRNN model, we investigate the application of the TDRNN to the control of inverted pendulums. The inverted pendulum system used here is shown in Fig.6, which is formed from a cart, a pendulum and a rail for defining position of cart. The Pendulum is hinged on the center of the top surface of the cart and can rotate around the pivot in the same vertical plane with the rail. The cart can move right or left on the rail freely. www.intechopen.com
  • 12. 126 Recurrent Neural Networks and Soft Computing 2l  F m M x 0 -2 -3 2 3 Fig. 6. Schematic diagram of inverted pendulum system The dynamic equation of the inverted pendulum system can be expressed as the following two nonlinear differential equations.     g sin   cos [ F  ml 2 sin   c sgn( x )  (m  M )1 ]  4 m cos 2  l l 3 m M  x    F  ml( 2 sin    cos  )  c sgn( x ) . m M   p ml , (42) (43) Where the parameters, M and m are respectively the mass of the cart and the mass of the pendulum in unit (kg), g  9.81 m s 2 is the gravity acceleration, l is the half length of the pendulum in unit (m), F is the control force in the unit (N) applied horizontally to the cart, uc is the friction coefficient between the cart and the rail, up is the friction   coefficient between the pendulum pole and the cart. The variables  ,  ,  represent the angle between the pendulum and upright position, the angular velocity and the angular acceleration of the pendulum, respectively. Moreover, given that clockwise direction is   positive. The variables x , x , x denote the displacement of the cart from the rail origin, its velocity, its acceleration, and right direction is positive. We use the variables  and x to control inverted pendulum system. The control goal is to make  approach to zero by adjusting F, with the constraint condition that x is in a given interval. The control block diagram of the inverted pendulum system is shown in Figure 7. The TDRNN controller adopts variables  and x as two input items. www.intechopen.com
  • 13. Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems    W1 Z-1  x Inverted pendulum W2 F  Z-1 127 x 1 W3 Fig. 7. Control block diagram of inverted pendulum system In the numerical experiments, the motion of the inverted pendulum system is simulated by numerical integral. The parameter setting is listed in the Table 1. parameter value g 9.81 M 1.0 m 0.1 μc 0.002 l 0.6 μp 0.00002   φ 5° 0 x 0  x 0 Table 1. Parameter Setting of Inverted Pendulum Besides, the number of hidden nodes is taken as 6, the weights are initialized in the interval [-3, 3] randomly, the parameters  ,  and  on the TDRNN are set as 0.3, 0.6, 0.4 respectively. The control goals are to control the absolute value of  within 10º and make it approximate to zero as closely as possible, with the constraint condition of the absolute value of x within 3.0m. 6 4 2 0 -2 -4 -6 0 1 Fig. 8. Control curve of the angle  www.intechopen.com 2 3 Time(s) 4 5
  • 14. 128 Recurrent Neural Networks and Soft Computing 0.12 0.10 x( m) 0.08 0.06 0.04 0.02 0.00 0 1 2 3 4 5 Time(s) Fig. 9. Control curve of the displacement x The control results are shown in Figures 8 and 9, and the sampling interval is taken as T  1ms . Figures 8 and 9 respectively show the control curve of the angle  and the control curve of the displacement x . From Figure 8, it can be seen that the fluctuation degree of  is large at the initial stage, as time goes on, the fluctuation degree becomes smaller and smaller, and it almost reduces to zero after 3 seconds. Figure 9 shows that the change trend of x is similar to that of  except that it has a small slope. These results demonstrate the proposed control scheme based on the TDRNN can effectively perform the control for inverted pendulum system. 4.4 Ultrasonic motor control In this section, a dynamic system of the ultrasonic motor (USM) is considered as an example of a highly nonlinear system. The simulation and control of the USM are important problems in the applications of the USM. According to the conventional control theory, an accurate mathematical model should be set up. But the USM has strongly nonlinear speed characteristics that vary with the driving conditions and its operational characteristics depend on many factors. Therefore, it is difficult to perform effective control to the USM using traditional methods based on mathematical models of systems. Our numerical experiments are performed using the model of TDRNN for the speed control of a longitudinal oscillation USM [17] shown in Figure 10. www.intechopen.com
  • 15. Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems Piezoelectric vibrator 129 Direction of the rotation ~ Vibratory piece Rotor Fig. 10. Schematic diagram of the motor Some parameters on the USM model are taken as: driving frequency 27.8 kHZ , amplitude of driving voltage 300 V , allowed output moment 2.5 kg  cm , rotation speed 3.8 m / s . Besides, the number of hidden nodes of the TDRNN is taken as 5, the weights are initialized in the interval [-3, 3] randomly, the parameters  ,  and  on the TDRNN are taken as 0.4, 0.6, 0.4 respectively. The input of the TDRNN is the system control error in the last time, and the output of the TDRNN, namely the control parameter of the USM is taken as the frequency of the driving voltage. Figure 11 shows the speed control curves of the USM using the three different control strategies when the control speed is taken as 3.6 m / s . In the figure the dotted line a represents the speed control curve based on the method presented by Senjyu et al.[18], the solid line b represents the speed control curve using the method presented by Shi et al.[19] and the solid line c represents the speed curve using the method proposed in this paper. Simulation results show the stable speed control curves and the fluctuation amplitudes obtained by using the three methods. The fluctuation degree is defined as   (Vmax  Vmin ) / Vave  100% (44) where Vmax , Vmin and Vave represent the maximum, minimum and average values of the speeds. From Figure 11 it can be seen that the fluctuation degrees when using the methods proposed by Senjyu and Shi are 5.7% and 1.9% respectively, whereas, it is just 1.1% when using the method in this paper. Figure 12 shows the speed control curves of the reference speeds vary as step types. From the figures it can be seen that this method possesses good control precision. www.intechopen.com
  • 16. 130 Recurrent Neural Networks and Soft Computing Speed( m s) / 3.8 a 1.9% 3.7 3.6 5.7% 3.5 1.1% b c a -- obtained by Senjyu b -- obtained by Shi c -- obtained in this paper 3.4 3.3 10.00 10.05 10.10 10.15 10.20 Time(s) Fig. 11. Comparison of speed control curves using different schemes Speed (m/s) 3.6 3.0 2.4 1.8 1.2 0 0.12 0.24 Time(s) 0.36 0.48 Fig. 12. Speed control curve for step type 5. Conclusions This chapter proposes a time-delay recurrent neural network (TDRNN) with better performance in memory than popular neural networks by employing the time-delay and recurrent mechanism. Subsequently, the dynamic recurrent back propagation algorithm for the TDRNN is developed according to the gradient descent method. Furthermore, to train neural networks more efficiently, we propose three criterions of selecting proper learning rates for the dynamic recurrent back propagation algorithm based on the discrete-type Lyapunov stability analysis. Besides, based on the TDRNN model, we have described, analyzed and discussed an identifier and an adaptive controller designed to identify and www.intechopen.com
  • 17. Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 131 control nonlinear systems. Our numerical experiments show that the TDRNN has good effectiveness in the identification and control for nonlinear systems. It indicates that the methods described in this chapter can provide effective approaches for nonlinear dynamic systems identification and control. 6. Acknowledgment The authors are grateful to the support of the National Natural Science Foundation of China (61103146) and (60873256), the Fundamental Research Funds for the Central Universities (DUT11SX03). 7. References [1] M. Han, J.C. Fan and J. Wang, A Dynamic Feed-forward Neural Network Based on Gaussian Particle Swarm Optimization and its Application for Predictive Control, IEEE Transactions on Neural Networks, 22 (2011) 1457-1468. [2] G. Puscasu and B. Codres, Nonlinear System Identification and Control Based on Modular Neural Networks, International Journal of Neural Systems, 21(2011), 319334. [3] L.Li, G. Song, and J.Ou, Nonlinear Structural Vibration Suppression Using Dynamic Neural Network Observer and Adaptive Fuzzy Sliding Mode Control. Journal of Vibration and Control, 16 (2010), 1503-1526. [4] T. Hayakawa, W.M. Haddad, J.M. Bailey and N. Hovakimyan, Passivity-based Neural Network Adaptive Output Feedback Control for Nonlinear Nonnegative Dynamical Systems, IEEE Transactions on Neural Networks, 16 (2005) 387-398. [5] M. Sunar, A.M.A. Gurain and M. Mohandes, Substructural Neural Network Controller, Computers & Structures, 78 (2000) 575-581. [6] D. Wang and J. Huang, Neural Network-based Adaptive Dynamic Surface Control for a Class of Uncertain Nonlinear Systems in Strict-feedback Form, IEEE Transactions on Neural Networks, 16 (2005) 195-202. [7] Y.M. Li, Y.G. Liu and X.P. Liu, Active Vibration Control of a Modular Robot Combining a Back-propagation Neural Network with a Genetic Algorithm, Journal of Vibration and Control, 11 (2005) 3-17. [8] J.C. Patra and A.C. Kot, Nonlinear Dynamic System Identification Using Chebyshev Functional Link Artificial Neural Networks, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 32 (2002) 505-511. [9] R.J. Wai, Hybrid Fuzzy Neural-network Control for Nonlinear Motor-toggle Servomechanism, IEEE Transactions on Control Systems Technology, 10 (2002) 519532. [10] G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signals and System, 2 (1989) 303–314. [11] S. Haykin, Neural Networks: A Comprehensive Foundation (Englewood Cliffs, NJ: Prentice Hall, 1999). [12] J.L. Elman, Finding Structure in Time, Cognitive Science, 14 (1990) 179-211. [13] C.H. Chiu, Y.F. Peng, and Y.W. Lin, Intelligent backstepping control for wheeled inverted pendulum, Expert Systems With Applications, 38 (2011) 3364-3371. www.intechopen.com
  • 18. 132 Recurrent Neural Networks and Soft Computing [14] M.I. El-Hawwary, A.L. Elshafei and H.M. Emara, Adaptive Fuzzy Control of the Inverted Pendulum Problem, IEEE Transactions on Control Systems Technology, 14 (2006) 1135-1144. [15] P.J. Gawthrop and L.P. Wang, Intermittent Predictive Control of An Inverted Pendulum, Control Engineering Practice, 14 (2006) 1347-1356. [16] R.J. Wai and L.J. Chang, Stabilizing and Tracking Control of Nonlinear Dual-axis Inverted-pendulum System Using Fuzzy Neural Network, IEEE Transactions on Fuzzy systems, 14 (2006) 145-168. [17] X. Xu, Y.C. Liang, H.P. Lee, W.Z. Lin, S.P. Lim and K.H. Lee, Mechanical modeling of a longitudinal oscillation ultrasonic motor and temperature effect analysis, Smart Materials and Structures, 12 (2003) 514-523. [18] T. Senjyu, H. Miyazato, S. Yokoda, and K. Uezato, Speed control of ultrasonic motors using neural network, IEEE Transactions on Power Electronics, 13 (1998) 381-387. [19] X.H. Shi, Y.C. Liang, H.P. Lee, W.Z. Lin, X.Xu and S.P. Lim, Improved Elman networks and applications for controlling ultrasonic motors, Applied Artificial Intelligence 18 (2004) 603-629. www.intechopen.com
  • 19. Recurrent Neural Networks and Soft Computing Edited by Dr. Mahmoud ElHefnawi ISBN 978-953-51-0409-4 Hard cover, 290 pages Publisher InTech Published online 30, March, 2012 Published in print edition March, 2012 How to reference In order to correctly reference this scholarly work, feel free to copy and paste the following: Hong Wei Ge and Guo Zhen Tan (2012). Recurrent Neural Network-Based Adaptive Controller Design For Nonlinear Dynamical Systems, Recurrent Neural Networks and Soft Computing, Dr. Mahmoud ElHefnawi (Ed.), ISBN: 978-953-51-0409-4, InTech, Available from: http://www.intechopen.com/books/recurrent-neuralnetworks-and-soft-computing/recurrent-neural-network-based-adaptive-controller-design-for-nonlinearsystems InTech Europe University Campus STeP Ri Slavka Krautzeka 83/A 51000 Rijeka, Croatia Phone: +385 (51) 770 447 Fax: +385 (51) 686 166 www.intechopen.com InTech China Unit 405, Office Block, Hotel Equatorial Shanghai No.65, Yan An Road (West), Shanghai, 200040, China Phone: +86-21-62489820 Fax: +86-21-62489821