J Control Theory Appl 2012 10 (3) 349–353DOI 10.1007/s11768-012-0076-0

Identification for temperature model ofaccelerometer based on proximal SVR and particle

swarm optimization algorithms

Xiangtao YU, Lan ZHANG, Linrui GUO, Feng ZHOUAerospace Science and Industry Inertial Technology Co., Ltd., Beijing 100074, China

Abstract: The impact of temperature on accelerometer will directly influence the precision of the inertial naviga-tion system (INS). To eliminate the measurement error of accelerometer, this paper proposes a proximal support vectorregression (PSVR) algorithm for generating a linear or nonlinear regression which requires the solution to single systemof linear equations. PSVR is used to identify the static temperature model of the accelerometer. In order to improve theidentifying performance, the kernel parameters and penalty factors of PSVR are optimized by the canonical particle swarmoptimization (CPSO). The experiments under different temperature conditions were conducted. The experimental resultsshow that the proposed PSVR can correctly identify the static temperature model of quartz flexure accelerometer and ismore efficient than those of the standard SVR and least square algorithm.

Keywords: Proximal support vector regression; Particle swarm optimization; System identification; Quartz flexureaccelerometer; Inertial navigation system

1 IntroductionAs one of the important inertial instruments, accelerom-

eter has been used successfully in measuring the state ofmotion of the vehicle. The precision of accelerometer has adirect effect on the inertial navigation system (INS). Envi-ronment temperature is an important factor of the precisionof inertial instrument [1]. The accelerometer output is af-fected by the temperature, which is often a problem. Thereare two methods for solving the problem. One method is touse a temperature control system particularly designed tooffer steady temperature environment. The disadvantage ofthis method is that the accelerometer volume will be largerthan usual and the cost will be increased. The other methodis to use a temperature compensation model to compensateerror of the accelerometer. Compared with the first method,this method is simple and efficient.

Neural networks (NNs) [2] could be a good solution tomodeling for many problems. However, the learned solu-tions often fall into local minimum and a large number ofinstances are needed for training the networks. As a rela-tively new method, support vector machine (SVM) [3] usestructural risk minimization (SRM) instead of empirical riskminimization (ERM). In SVM, the generalization of smallsample model is strengthened. To date, this method has beensuccessfully applied to many fields such as the fault detec-tion and diagnostics [4], time series prediction [5], and pat-tern recognition [6].

However, it generally takes a long time for the standardSVM to solve a quadratic program or a linear program.Some methods have been presented to improve the SVMperformance, such as sequential minimal optimization arith-metic [7], least square support vector machine [8], and the

generalized support vector machine [9], and so on. Espe-cially, proximal support vector machine (PSVM) [10] pro-posed by Fung which classifies points depending on prox-imity to one of two parallel planes only requires solving asingle system of linear equation. It takes very short time andis more efficient than the standard SVM. While PSVM hasbeen applied to the fault detection and diagnostics [11], theapplication of PSVM to system identification has seldombeen reported.

This paper uses proximal support vector regression(PSVR) to identify the temperature model of quartz flexureaccelerometer. The parameters of PSVR are optimized byparticle swarm optimization [12] algorithms to avoid depen-dence on initial parameters and training samples. The exper-iments show that the proposed PSVR can correctly identifythe model of quartz flexure accelerometer temperature withgreat efficiency.

2 Proximal support vector regressionIn order to extend PSVM for system identification, the

symbols that are introduced in this paper have the samemeaning as in [10].2.1 Linear proximal support vector regression

The linear PSVR (LPSVR) model for system identifica-tion could be written as:

f(x) = xTw − r, (1)where w is the normal of the bounding planes, r deter-mines the distance from the optimal hyperplane relative tothe original one.

A regression problem with m-points in n-dimensionalreal space R

n, represented by a matrix Am×n, is consid-

Received 9 April 2010; revised 6 April 2011.This work was supported by the National Key Basic Research and Development Program (No. 61388).

c© South China University of Technology and Academy of Mathematics and Systems Science, CAS and Springer-Verlag Berlin Heidelberg 2012

350 X. Yu et al. / J Control Theory Appl 2012 10 (3) 349–353

ered. f ∈ R denotes the output value in accordance with theith pattern input. The LPSVR is represented by the follow-ing quadratic programming with linear equality constraints:

min(w,r,y)∈Rn+1+m [

v

2‖y‖2 +

12(wTw + r2)],

s.t. Aw − er + y = f,(2)

where v is a regularization variable called trade-off param-eter, e is a vector with all variables of unity and yi denotesthe error variable for the ith sample.

LPSVR could be solved by setting equal zero the gradi-ents with respect to (w, r, y, u) of the Lagrangian :

L(w, r, y, u) =v

2‖y‖2 +

12(wTw + r2)

−uT(Aw − er + y − f). (3)The following Karush-Kuhn-Tucke (KKT) optimality

conditions could be obtained by setting the gradients of Lequal zeros:⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

∂L

∂w= w − ATu = 0,

∂L

∂γ= γ + eTu = 0,

∂L

∂y= vy − u = 0,

∂L

∂u= −Aw + eγ − y + f = 0.

(4)

The Lagrange multiplier could be obtained based on for-mula (4):

u = (I

v+ AAT + eeT)−1f. (5)

The LPSVR equation could be represented as follows:f(x) = xTw − r

= xTATu + eTu

= (xTAT + eT)(I

v+ AAT + eeT)−1f. (6)

2.2 Nonlinear proximal support vector regressionLinear separating planes are often not good enough for

the regression of the dataset satisfactorily, and the nonlinearPSVR (NLPSVR) could be obtained by replacing the initialvariables, w by its dual equivalent w = ATu, the linear ker-nel AAT by nonlinear kernel which is a kernel function thatis satisfied with the Mercer condition respectively.

min(u,r,y)∈Rn+1+m

v

2‖y‖2 +

12(uTu + r2),

s.t. K(A,AT)u − er + y = f.(7)

The Lagrangian for (7) can be written as:L(u, r, y, υ)

=v

2‖y‖2+

12‖[

u

r

]‖2 − υT(Ku−er+y−f). (8)

The following KKT optimality conditions could be ob-tained by setting the gradients of L equal to zero:⎧⎪⎪⎪⎨

⎪⎪⎪⎩u − KTυ = 0,

γ + eTυ = 0,

vy − υ = 0,

Ku − eγ + y = f.

(9)

Substituting these first three expressions in the last equa-

tion of the system of equations (9), the Lagrange multipliercould be obtained:

υ = (KKT + eeT +I

v)−1f. (10)

The NLPSVR regression equation could be representedas follows:

f(x) = (K(xT, AT)K(A,AT)T + eT)v. (11)

Equations (6) and (11) are the PSVR models we obtained.Contrast to [10], it could be seen that the gained model

is identical to those obtained in [10] except for the PSVRequation does not contain a diagonal matrix D.

3 Particle swarm optimization algorithmsThe choice of the penalty factor and the kernel function

of PSVR plays important role in learning correction andthe generalization of regression model. Particle swarm op-timization (PSO) algorithm has many advantages such assimpler implementation, faster convergence rate and fewerparameters to adjust over other evolutionary computationtechniques like genetic algorithms. In this paper, PSO isadopted to obtain the penalty factor and the kernel functionof PSVR.

The PSO algorithm randomly initializes m d-dimensional particles in the searching space, and then findsout the optimal parameters by iterative calculation. In everyiterative process, the particles renew themselves by trac-ing the individual optimal Pibest and the global optimaPgbest. The position and the velocity of ith particle areXi=[xi1 xi2 · · · xiD] and V i=[vi1 vi2 · · · viD], respec-tively. The individual optimal value Pibest of every particleis Pibest=[pi1 pi2 · · · piD], the global optimal value Pgbest

is Pgbest=[pg1 pg2 · · · pgD]. The position and the velocityof the particles at the current time step (k+1) are expressedin terms of the values at previous time step k as follows:

vij(k + 1) = ω · vij(k) + c1 · r1 · (pij − xij(k))+ c2 · r2 · (pgj − xij(k)), (12)

xi,j(k+1)=xi,j(k)+vi,j(k+1), j =1, 2, . . . , D, (13)

where ω is the inertia factor that keeps the diversity of theparticles by changing the momentum of the particles to pre-vent the particles from leading to the local minimum. A bigω is in favor of winkling the local minimum, while a smallw is in favor of the arithmetic convergence. c1 and c2 are thelearning factor. r1 and r2 are the random numbers between0 and 1. The last two items in equation (12) are the com-parison of the action of particles themselves and the com-parison of the differences of the actions. They are called the‘conscious part’. Generally, a velocity boundary should bedefined. When the velocity oversteps the boundary, the ve-locity is defined as the velocity boundary value.

Clerc [13] found that the velocity boundary limitationcould be eliminated by a compressing factor. The jth ve-locity of the ith particle can be renewed by equation (15):

vij(k+1)=

⎧⎪⎨⎪⎩

K[vij(k)+c1r1(pij−xij(k))+c2r1(pgj−xij(k))], xmin <xij <xmax,

0, others,(14)

X. Yu et al. / J Control Theory Appl 2012 10 (3) 349–353 351

xij(k+1)=

⎧⎪⎨⎪⎩

xij +vij(k+1), xmin <(xij +vij)<xmax,

xmax, (xij + vij) > xmax,

xmin, (xij + vij) < xmin,

(15)

where c1 and c2 are the learning factors. r1 and r2 are therandom numbers between 0 and 1.

K =2∣∣2 − C −√C2 − 4C

∣∣ , C = c1 + c2.

Typically, the cognition learning rate [14] is c1 = 2.8, so-cial learning rate c2 = 1.3, and the total iteration number isN = 30.

4 Experimental research4.1 Function validation

1) The validation of LPSVR.In order to verify the correctness of LPSVR, the regres-

sion of a simple linear function as formula (16) is evaluatedwith LPSVR:

y = x + 0.2 × n(x). (16)

n(x) generates arrays of random numbers whose elementsare normally distributed with 0-mean, and 1-variance (σ2 =1). The regression results are shown as Fig. 1.

Fig. 1 Linear regression problem.The solid line indicates the LPSVR outputs; the dotted

line represents the linear function. The stars indicate thetraining data points. It can be seen from Fig. 1 that the lin-ear function regression result is well associated with the truefunction.

2) The validation of NLPSVR.A regression task as formula (17) expressed is solved us-

ing nonlinear NLPSVR:

y = sinc(x) + 0.1 × n(x), (17)

where sinc computes the sinc function of an input vector orarray, the sinc function is shown as

sinc(x) =

⎧⎨⎩

1, x = 0,sin(πx)

πx, x �= 0.

(18)

The Gauss function is chosen as the kernel function ofNLPSVR:

K(x, y) = exp(−μ ‖x − y‖2). (19)

The result is displayed in Fig. 2.

Fig. 2 Nonlinear regression problem.It is clear that NLPSVR also has good regression effects

from Fig. 2. These results suggest that the proposed PSVRhas good regression ability no matter whether it is linear ornonlinear.4.2 Experimental verification

The procedures of identifying the temperature model ofquartz flexure accelerometer based on PSVR optimized withPSO algorithms are shown in Fig. 3.

Fig. 3 Chart of temperature model of optimizing PSVR parame-ters based on CPSO.

As shown in Fig. 3, the aim of this system is to optimizethe PSVR parameter values automatically:

Step 1 Setting the PSO parameters which include thenumber of CPSO population, the total iteration number, thecognition learning rate, and social learning rate.

Step 2 Initializing the PSVR parameters: the trade-offparameter v and the kernel parameter μ which controls theflexibility of the gauss function.

Step 3 Taking some experimental data are as the train-ing data.

352 X. Yu et al. / J Control Theory Appl 2012 10 (3) 349–353

Step 4 Computing objective function based on thetraining data. The equation of objective function is

M = {N∑

i=1

[y(i) − y(i)]2}/N, (20)

where y(i) is the output of the PSVR, y(i) is the targetvalue, N is the total number of samples.

Step 5 If stopping conditions are met; the optimizedPSVR parameter is obtained; otherwise, return to step 3.

The experiments under different temperature conditionswere conducted, in order to verify good regression perfor-mance of the proposed PSVR. The tested quartz flexure ac-celerometer is set as the state of the door which the inputaxis (IA) is horizontal with μ rad, the positive output axis(OA) downwards; the pendulous axis (PA) is parallel to therotation of the head within μ rad. The tested quartz flexureaccelerometer is installed in the rotary table equipped withtemperature box as Fig. 4 shows.

The detail heating process system is as shown in Fig. 5.

Fig. 4 Chart of the state of door.

Fig. 5 Chart of temperature model of optimizing PSVR parame-ters based on CPSO.

The heating range is from −40◦C to +60◦C and from+60◦C to −40◦C. Every heating position step is about10◦C, and the temperature is kept about 30 minutes in everyfixed temperature value. The four points rotary experimentthat included with 0◦, 90◦, 180◦ and 270◦ are conducted.The bias and scale factor could be calculated based on equa-tions (21) and (22):

K0 =(U0 + U180)

2K1, (21)

K1 =(U90 − U270)

2g, (22)

where K0 is the bias, K1 is the scale factor, U0, U90, U180

and U270 are the voltage values when θ is in the positions of0◦, 90◦, 180◦ and 270◦, respectively.

The temperature value and the corresponding bias can beutilized as the input and output of PSVR1. For example,we can take the temperature value and the correspondingscale factor as the input and output of PSVR2. The param-eters of kernel function μ and penalty factor v of PSVR areoptimized with CPSO. The parameters of CPSO are set ac-cording to the CPSO algorithms described in Section 3. Theoptimized results are shown as follows:

PSVR1 v = 221.880, μ = 76.665, MSE = 1.7985 ×10−6 g;

PSVR2 v =5096.276, μ= 0.12282, MSE = 4.0926 ×10−6 g/mA.

In order to validate the effective performance of the pro-posed method, the PSVR simulation results are shown inFig. 6.

It can be seen that the identification of PSVR is well as-sociated with the experimental results from Fig. 6 (a) andFig. 6 (c), and the absolute error precision of PSVR andSVR is better than that of least square from Fig. 6 (b) andFig. 6 (d). The identification accuracy of the model is im-proved about one order of magnitude. It takes about 0.015second for PSVR to identify the model, which is less thanSVR which needs about 0.359 second. PSVR is adaptedwell to identify the temperature model of quartz flexure ac-celerometer.

(a) Comparison between the experimental results and PSVRoutputs of K0

(b) Comparison of the identification absolute errors of K0

(c) Comparison between the experimental results and PSVRoutputs of K1

X. Yu et al. / J Control Theory Appl 2012 10 (3) 349–353 353

(d) Comparison of the identification absolute errors of K1

Fig. 6 The comparison of PSVR outputs.

5 ConclusionsIn this paper, an extension of PSVM for system identi-

fication has been proposed. The proposed PSVR methodis used to identify the temperature model of quartz flex-ure accelerometer. In order to get better identification per-formance, the PSO algorithm is introduced to optimize theparameters of PSVR. The experiments show that the de-signed PSVR can correctly identify the temperature modelof quartz flexure accelerometer in very short time. It demon-strates that the proposed PSVR is effective.

References[1] X. Yu, L. Zhang, L. Guo, et al. Temperature modeling and

compensation of accelerometer based on least squares waveletsupport vector machine. Journal of Chinese Inertial Technology,2011, 19(1): 95 – 98.

[2] S. Won, L. Song, S. Lee, et al. Identification of finite state automatawith a class of recurrent neural networks. IEEE Transactions onNeural Networks, 2010, 21(9): 1408 – 1421.

[3] V. N. Vapnik. An overview of statistical learning theory. IEEETransactions on Neural Networks, 1999, 10(5): 988 – 999.

[4] X. Yu, F. Chu, R. Hao. Fault diagnostics approach for rollingbearing based on soft morphological and support vector machine.Chinese Journal of Mechanical Engineering, 2009, 45(7): 78 – 80(in Chinese).

[5] N. Sapankevych, R. Sankar. Time series prediction using supportvector machines: a survey. IEEE Computational IntelligenceMagazine, 2009, 4(2): 24 – 38.

[6] C. J. C. Burges. A tutorial on support vector machines for patternrecognition. Data Mining and Knowledge Discovery, 1998, 2(2): 121– 167.

[7] J. Platt. Fast training of support vector machines using sequentialminimal optimization. Advances in Kernel Methods-Support VectorLearning, Cambridge: MIT Press, 1999: 185 – 208.

[8] J. A. K. Suykens, J. Vandewalle. Least squares support vectormachine classifiers. Neural Processing Letters, 1999, 9(3): 293 – 300.

[9] O. L. Mangasarian. Generalized support vector machines. Advancesin Large Margin Classifiers, Cambridge: MIT Press, 2000: 135 – 146.

[10] G. Fung, O. L. Mangasarian. Proximal support vector machineclassifiers. Proceedings KDD2001: Knowledge Discovery and DataMining, New York, 2001: 77 – 86.

[11] X. Yu, W. Lu, F Chu. Fault diagnostics based on pattern spectrumentropy and proximal support vector machine. Key EngineeringMaterials, 2009, 413/414: 607 – 612.

[12] J. Kennedy, R. Eberhart. Particle swarm optimization. Proceedingsof IEEE International Conference on Neural Networks, Piscataway:IEEE, 1995: 1942 – 1948.

[13] M. Clerc. The swarm and the queen: towards a deterministic andadaptive particle swarm optimization. Proceedings of the IEEECongress on Evolutionary Computation, Washington: IEEE, 1999:1951 – 1957.

[14] A. Carlisle, G. Dozier. An off-the-shelf PSO. Proceedings of theWorkshop on Particles Swarm Optimization, Indianapolis, 2001: 1 –6.

Xiangtao YU was born in Shandong, China, in1979. He received his Ph.D. degree in Control The-ory and Control Engineering from Beijing Instituteof Technology in 2007. He is a senior engineerin ASIT Co., Ltd. His research interests includecontrol system molding and optimization, inertialtechnology, and fault diagnosis. E-mail: yuxiang-tao@163.com.

Lan ZHANG was born in 1972. She is the direc-tor of the 5th research office in ASIT Co., Ltd. Herresearch interest is inertial technology. E-mail: zha-lahzhali@163.com.

Linrui GUO was born in 1979. He received hisPh.D. degree from Tsinghua University in 2006. Heis the deputy director of the 5th research office inASIT Co., Ltd. His research interest is inertial tech-nology. E-mail: linrui.guo@gmail.com.

Feng ZHOU was born in 1978. He is the deputydirector of the 5th research office in ASIT Co., Ltd.His research interest is inertial technology. E-mail:zhouf911@163.com. –– – –– – –– – –– – – –– –