Least Absolute Integral Method of Data Fitting Based on Algorithm of Simulated Annealing and Neural Network

Maolin Cheng

School of Mathematics and Physics, Suzhou University of Science and Technology, Suzhou, China

Email address:

To cite this article:

Maolin Cheng. Least Absolute Integral Method of Data Fitting Based on Algorithm of Simulated Annealing and Neural Network. Mathematics and Computer Science. Vol. 1, No. 3, 2016, pp. 61-65. doi: 10.11648/j.mcs.20160103.15

Received: September 5, 2016; Accepted: September 18, 2016; Published: October 9, 2016

Abstract: There are many methods related to data fitting, and each method has its distinctive features. The article discusses the method of data fitting function under integral criterion. Since the estimate fitting parameters are complicated, the article combines algorithm of simulated annealing and neural network algorithm to solve the integral with neural network algorithm and solve the unknown parameters with simulated annealing algorithm. By case analog computation of household per capita consumption expenditure of urban and the rural residents in China, it proves that the combination of simulated annealing algorithm and neural network algorithm has strong reliability and high accuracy in terms of new method for least absolute integral data fitting.

Keywords: Data Fitting, Simulated Annealing, Neural Network, Algorithm, Least Absolute Integral Method

1. Introduction

There are n groups of observational data  and they are the values of n nodes of .

The form of general linear regression model is , where  is a fit function, which could be a linear function or a nonlinear function.  is a p-dimensional parameter vector and  is a random error variable which meets

,

There are three frequently-used  norms:

(1)

(2)

(3)

Apparently,  pertains to least absolute method;  pertains to least square method; while  pertains to regression fitting under the least most rule.

Since we have regarded the observational data  as the points in the interval  defined by, we assume  are all continuous functions. Actually what we need is only integrabel and the fitting function  we are looking for are not discrete points. Due to , and the definition is .

Thus the parameters in  have been determined, let .

In general,  equal to 1, 2, or . Since we hereby want to study the situation in case, namely

reaches the minimum. This method is referred to as least absolute integral method in this article and the geometric meaning is the minimum area enclosed by curves of  within the interval of . Since the  is unknown and only the values at the nodes are available, in addition, the integrand is an absolute value function, therefore the commonly used methods for seeking extreme value are invalid. In case the  is substituted by interpolating function, the calculation of the minimal value will be complicated. Especially, in case the  is an essential nonlinearity function, the calculation of the minimal value will be more complicated. Consequently, the article provides simulated annealing and neural network algorithms to find the solution to the integration with neural network algorithm [1-5]. It is feasible to integrate the functions which are complicated and hard to be integrated as well as the functions which only have a series of data without clear analytic expressions. The unknown parameters can be solved with revised simulated annealing algorithm [6-10]. The validity, reliability and accuracy of this method have also been proved by computation example.

2. Method and Model of Simulated Annealing and Neural Network Algorithm

2.1. Integrate with Neural Network Algorithm

2.1.1. Trigonometric Function Neural Network Model

If  is given (the  optimizing process is derived from simulated annealing algorithm—refers to 2. 2), then,

has been confirmed and here the calculation of integral is done by using trigonometric function neural network model. Since the integrand is an error function  and it has the fluctuation characteristic of little variation, it can be approximated by superimposed trigonometric functions. Therefore, the trigonometric function neural network model is mentioned in the article [11-15], the integral value is approximated by output integration of the neural network based on trigonometric functions basis funciton.

In consideration of possible cycles, the fluctuation series could be expressed by

,

therefore, the hidden neuron excitation function is

.

The number of hidden neurons is .

The excitation matrix is

Set the weight matrix as .

Then the output of neural network:

The error function: .

The number of samples is , , set the error matrix as , the performance index is

where  is the square of Euclidean norm. Weight adjustment:

where  is the learning rate and .

2.1.2. Convergence Conditions of Trigonometric Function Neural Network Algorithm

Learning rate has significant influence on the convergence of trigonometric function neural network algorithm. Over low learning rate may slow down the convergence speed of trigonometric function neural network algorithm, while increasing the computation volume. By contrast, over high learning rate may lead to oscillation, rather than convergence. In order to ensure the absolute convergence of neural network algorithm, in the following, convergence condition of trigonometric function neural network algorithm is listed, acting as a theoretical basis for selection of learning rate.

Set as the learning rate, Lyapunov function is defined by

Then, ,

Moreover,

where is the square of Euclidean norm, so that:

In order to make the neural network algorithm convergent, there shall be, and the following inequality must be true:

As , i.e.:

As

Hereby

So that, when learning rate is set to be , the neural network algorithm would be convergent.

2.1.3. Calculation of Integral Based on Trigonometric Function Neural Network Algorithm

Since we know the weight of neural network  and the integral value will be:

2.2. Estimated Parameters of Simulated Annealing Algorithm

2.2.1. The General Steps of Simulated Annealing Algorithm

The simulated annealing algorithm i.e. SA algorithm [16-20] was put forwarded by Kirkpatrick and the annealing process of solid was made analogy with combinatorial optimization problems. The purpose of this method is to seek the optimal solution of function or approximate optimal solution by utilizing the method of achieving minimum energy balance during annealing process. The evaluation function is equivalent to energy  while the predefined one set of  parameters correspond to a physical state. The control parameters during the whole optimizing process are equivalent to the temperature  during annealing process, while the SA algorithm can be inverted for optimizing. The concrete steps of algorithm include:

(1)    Set the initial temperature  and minimum temperature. Provide the ranges of each parameter and select an initial model randomly within this range. The corresponding objective function value  will be calculated.

(2)    Disturb the current model parameters to generate a new model. Let . Similarly, the corresponding objective function value  will be calculated, such will get .

(3)    If , the new model can be accepted; if , the new model  will be accepted based on probability ,where  is the temperature. In case of the model being accepted, set .

(4)    At the temperature , reiterate the disturbance and acceptance processes for certain times, i.e. , repeat the steps of (2) and (3).

(5)    Decrease the temperature slowly.

(6)    Repeat steps (2) and (5) until  or , where the  is the predetermined smaller positive number.

(7)    The final solution obtained is the most optimal solution, i.e., the parameter values we are seeking for.

2.2.2. The Improvement of Simulated Annealing Algorithm

During the application of the algorithm, to get the parameters faster and more accurately following improvement could be made:

(1) Model disturbance

The author analyzes disturbance improvement strategy for simulated annealing algorithm and puts forward an algorithm which intensifies the local search ability. The algorithm drawlessonsfroms the idea of non-uniform mutation in genetic algorithm and generates new model parameters after disturbing the current model parameters with non-uniform mutation strategy, namely:

where  is No.  parameter in current model,  is No.  parameter after disturbance, is No.  parameter disturbance factor,  are values range of No.  parameter, is a random number within ,  is current temperature,  is the maximum iteration(related to maximum temperature and minimum temperature),  is the constant to confirm non-uniformity and  is a sign function.

The above equation has showed that the search range is relatively broad in case of high temperature; however, the search range narrows gradually with the decrease of the temperature. The value of  decreases gradually with the increase of iterations (i.e. the decrease of the temperature) thus the search range also narrows, which intensifies the local search ability. In the process of actual computation, the value of  must be controlled reasonably—it is prone to be fallen into local extremum in case the value is too big. Generally speaking it is better to be located within. The new accepted model will get closer to the actual model to be solved with the decrease of the temperature. Therefore, intensifying local search is more favorable to improve the convergence performance of the algorithm.

(2) Cooling method

As for annealing plan, the practice shows that the changing pattern of  exponentially is more suitable to the nature of annealing. In consideration of the fact that temperature drops very fast at the beginning then slow down at last in annealing method, the author adopts the modified index curve which is more suitable to the decreasing characteristic, namely

where  is the initial temperature,  is the given constant and ,  is the speed factor,  is the iterations. With the increase of , the temperature drops very fast at the beginning then slow down at last. Such the large-scaled rough search is utilized at the beginning and then refined search is utilized locally at last.

3. Example Application

At present, China economic is rapid growth. Researching on developing trend of household per capita consumption expenditure of urban and the rural residents in China is of vital significance. What the increasing regularity of household per capita consumption expenditure of urban and the rural residents in China and what kind of development tendency will be presented in the future? The article chooses Gaussian model based on the characteristics of growth for household per capita consumption expenditure of urban and the rural residents in China during 1991-2010. The information refers to Table 1.

The form of model:

Table 1. Information about household per capita consumption expenditure of urban and the rural residents in China and relevant outcome.

By utilizing the method in the article, the trigonometric function neural network model will take performance index . The number of hidden layers of neural network, the learning rate , the sample training rate is .

The parameter setting of simulated annealing algorithm is: initial temperature , minimum temperature , the length of Markov chains is 10, temperature drop factor ,the non-uniformity degree constant of model disturbance ,  and the  in cooling method.

By software programming with Matlab, we get

namely

4. Summary

The simulated annealing and neural network algorithm has been applied in data fitting least square integral method. The numerical simulation results show that the method is feasible and valid. The simulated results of simulated annealing neural network algorithm are desirable by judging from the test amount . However, the major defect of the algorithm is the contradiction between the precision of solution and long solving time. The solving time will increase greatly with the increase of inverting parameters. In addition, the scope of variation interval of each parameter has relatively great influence on the convergence rate of algorithm. Consequently, how to confirm effectively the ranges of parameters and some initial parameters of the algorithm so as to increase the convergence rate of algorithm is still to be further studied and discussed.

References

1. Zhang X.D, J.N.Yu, L.P.Guo, J.G.Zhang, Q.H.Ding(2011). Prediction of modeling of nonlinear times Series based on Improved BP neural network. Journal of Mianyang Normal University,30:84-88.
2. Gao X.J. (2011). Application of BP neural network in prediction of the liver cirrhosis's cure.Mathematical Theory and Applications,31:20-23.
3. Gavin J.J.B.Bowden, G.C.Nixon, H. R. Dandy, M. H.Maier(2006). Forecasting chlorine residuals in a water distribution system using a general regression neural network.Mathematical and Computer Modelling,44: 69-484.
4. Hu J.X,J.Zeng(2010).A Fast learning algorithm of global convergence for BP-neural network. Journal of Systems Science and Mathematical Sciences,30: 604-610.
5. Jia H.P. (2012). GRNN neural network in the application of power system load forecasting.Electronic Design Engineering, 20: 14-16.
6. Alrefaei M.H,A.H.Diabat(2009). A simulated annealing technique for multi-objective simulation optimization. Applied Mathematics and Computation,215: 3029-3035.
7. Anagnostopoulos K.P,L.Kotsikas(2010).Experimental evaluation of simulated annealing algorithms for the time-cost trade-off problem. Applied Mathematics and Computation, 217: 260-270.
8. Chen J,Y.Liu(2011).Traffic prediction research of neural network combined simulated annealing algorithm. Computer Engineering and Design, 32: 2138-2145.
9. Gao S. (2002).Research on annealing strategy in Simulated Annealing Algorithm.Aeronautical Computer Technique,32: 20-25.
10. Gu Y.X,B.W.Xiang, G. Z.Zhao(2005).An improved simulated annealing algorithm for global optimization problems with continuous variables. Systems Engineering Theory & Practice,25: 103-177.
11. Shi H.,Y.L.Wang, F.L.Weng(2011).Composite prediction model of BP neural networks and fuzzy time series and its application. Journal of Computer Application, 31: 90-92.
12. Xiao H., S.D.Liu, X.Y.Huang, L.Jin(2010). A study on neural network ensemble forecast model based on kernel principal component analysis.Computer Simulation,27: 163-167.
13. Qiu Q.R,T.Yu(2011).BP neural network forecast model based on principal component analysis for the real estate price of prediction. Journal of Hunan University of Arts and Science(Natural Science Edition),23: 24-27.
14. Li H.X,C.W.Li(2009). Application of artificial neural network in predicting the number of graduate students. Mathematics in Practice and Theory,39: 27-33.
15. Feng Z.Y,M.H.Wang(2011). Essential analysis on the generalization problem of feedforward neural network used in economic forcasting. Journal of Shaanxi University of Science & Technology(Natural Science Edition),29: 108-111.
16. Guo D.L,H.M.Xia,Y.Q.Zhou(2010). Hybrid simulation annealing and evolution strategy algorithm in non-linear parameter estimation application. Mathematics in Practice and Theory,40: 91-98.
17. Hao Z,F.Hong(2009).Using genetic/simulated annealing algorithm to solve disassembly sequence planning. Journal of Systems Ensnaring and Electronics, 20:906-912.
18. Jin L.X. H.W.Tang, B.Li, M.J. Ji, X.Z. Zhu(2005). A simulated annealing algorithm for continuous functions and its convergence properties.Mathematica Numerica Sinica,27:19-30.
19. Liu J.J,M.F.Chen, Z.P.Ye(2010).The estimation of the non-linear model parameter based on combination of damped least-squares method and simulated annealing method. Journal of Jinggangshan University(Natural Science),31: 10-14.
20. MichaelA.H,M.Brasel, J.Morig, F.Tusch, P.Werner, algorithms for minimizing mean flow time in an open shop.Mathematical and Computer Modelling,48:1279-1293.