Least Absolute Integral Method of Data Fitting Based on Algorithm of Simulated Annealing and Neural Network
School of Mathematics and Physics, Suzhou University of Science and Technology, Suzhou, China
To cite this article:
Maolin Cheng. Least Absolute Integral Method of Data Fitting Based on Algorithm of Simulated Annealing and Neural Network. Mathematics and Computer Science. Vol. 1, No. 3, 2016, pp. 61-65. doi: 10.11648/j.mcs.20160103.15
Received: September 5, 2016; Accepted: September 18, 2016; Published: October 9, 2016
Abstract: There are many methods related to data fitting, and each method has its distinctive features. The article discusses the method of data fitting function under integral criterion. Since the estimate fitting parameters are complicated, the article combines algorithm of simulated annealing and neural network algorithm to solve the integral with neural network algorithm and solve the unknown parameters with simulated annealing algorithm. By case analog computation of household per capita consumption expenditure of urban and the rural residents in China, it proves that the combination of simulated annealing algorithm and neural network algorithm has strong reliability and high accuracy in terms of new method for least absolute integral data fitting.
Keywords: Data Fitting, Simulated Annealing, Neural Network, Algorithm, Least Absolute Integral Method
There are n groups of observational data and they are the values of n nodes of .
The form of general linear regression model is , where is a fit function, which could be a linear function or a nonlinear function. is a p-dimensional parameter vector and is a random error variable which meets
There are three frequently-used norms:
Apparently, pertains to least absolute method; pertains to least square method; while pertains to regression fitting under the least most rule.
Since we have regarded the observational data as the points in the interval defined by, we assume are all continuous functions. Actually what we need is only integrabel and the fitting function we are looking for are not discrete points. Due to , and the definition is .
Thus the parameters in have been determined, let .
In general, equal to 1, 2, or . Since we hereby want to study the situation in case, namely
reaches the minimum. This method is referred to as least absolute integral method in this article and the geometric meaning is the minimum area enclosed by curves of within the interval of . Since the is unknown and only the values at the nodes are available, in addition, the integrand is an absolute value function, therefore the commonly used methods for seeking extreme value are invalid. In case the is substituted by interpolating function, the calculation of the minimal value will be complicated. Especially, in case the is an essential nonlinearity function, the calculation of the minimal value will be more complicated. Consequently, the article provides simulated annealing and neural network algorithms to find the solution to the integration with neural network algorithm [1-5]. It is feasible to integrate the functions which are complicated and hard to be integrated as well as the functions which only have a series of data without clear analytic expressions. The unknown parameters can be solved with revised simulated annealing algorithm [6-10]. The validity, reliability and accuracy of this method have also been proved by computation example.
2. Method and Model of Simulated Annealing and Neural Network Algorithm
2.1. Integrate with Neural Network Algorithm
2.1.1. Trigonometric Function Neural Network Model
If is given (the optimizing process is derived from simulated annealing algorithm—refers to 2. 2), then,
has been confirmed and here the calculation of integral is done by using trigonometric function neural network model. Since the integrand is an error function and it has the fluctuation characteristic of little variation, it can be approximated by superimposed trigonometric functions. Therefore, the trigonometric function neural network model is mentioned in the article [11-15], the integral value is approximated by output integration of the neural network based on trigonometric functions basis funciton.
In consideration of possible cycles, the fluctuation series could be expressed by
therefore, the hidden neuron excitation function is
The number of hidden neurons is .
The excitation matrix is
Set the weight matrix as .
Then the output of neural network:
The error function: .
The number of samples is , , set the error matrix as , the performance index is
where is the square of Euclidean norm. Weight adjustment:
where is the learning rate and .
2.1.2. Convergence Conditions of Trigonometric Function Neural Network Algorithm
Learning rate has significant influence on the convergence of trigonometric function neural network algorithm. Over low learning rate may slow down the convergence speed of trigonometric function neural network algorithm, while increasing the computation volume. By contrast, over high learning rate may lead to oscillation, rather than convergence. In order to ensure the absolute convergence of neural network algorithm, in the following, convergence condition of trigonometric function neural network algorithm is listed, acting as a theoretical basis for selection of learning rate.
Set as the learning rate, Lyapunov function is defined by
where is the square of Euclidean norm, so that:
In order to make the neural network algorithm convergent, there shall be, and the following inequality must be true:
As , i.e.:
So that, when learning rate is set to be , the neural network algorithm would be convergent.
2.1.3. Calculation of Integral Based on Trigonometric Function Neural Network Algorithm
Since we know the weight of neural network and the integral value will be:
2.2. Estimated Parameters of Simulated Annealing Algorithm
2.2.1. The General Steps of Simulated Annealing Algorithm
The simulated annealing algorithm i.e. SA algorithm [16-20] was put forwarded by Kirkpatrick and the annealing process of solid was made analogy with combinatorial optimization problems. The purpose of this method is to seek the optimal solution of function or approximate optimal solution by utilizing the method of achieving minimum energy balance during annealing process. The evaluation function is equivalent to energy while the predefined one set of parameters correspond to a physical state. The control parameters during the whole optimizing process are equivalent to the temperature during annealing process, while the SA algorithm can be inverted for optimizing. The concrete steps of algorithm include:
(1) Set the initial temperature and minimum temperature. Provide the ranges of each parameter and select an initial model randomly within this range. The corresponding objective function value will be calculated.
(2) Disturb the current model parameters to generate a new model. Let . Similarly, the corresponding objective function value will be calculated, such will get .
(3) If , the new model can be accepted; if , the new model will be accepted based on probability ,where is the temperature. In case of the model being accepted, set .
(4) At the temperature , reiterate the disturbance and acceptance processes for certain times, i.e. , repeat the steps of (2) and (3).
(5) Decrease the temperature slowly.
(6) Repeat steps (2) and (5) until or , where the is the predetermined smaller positive number.
(7) The final solution obtained is the most optimal solution, i.e., the parameter values we are seeking for.
2.2.2. The Improvement of Simulated Annealing Algorithm
During the application of the algorithm, to get the parameters faster and more accurately following improvement could be made:
(1) Model disturbance
The author analyzes disturbance improvement strategy for simulated annealing algorithm and puts forward an algorithm which intensifies the local search ability. The algorithm drawlessonsfroms the idea of non-uniform mutation in genetic algorithm and generates new model parameters after disturbing the current model parameters with non-uniform mutation strategy, namely:
where is No. parameter in current model, is No. parameter after disturbance, is No. parameter disturbance factor, are values range of No. parameter, is a random number within , is current temperature, is the maximum iteration(related to maximum temperature and minimum temperature), is the constant to confirm non-uniformity and is a sign function.
The above equation has showed that the search range is relatively broad in case of high temperature; however, the search range narrows gradually with the decrease of the temperature. The value of decreases gradually with the increase of iterations (i.e. the decrease of the temperature) thus the search range also narrows, which intensifies the local search ability. In the process of actual computation, the value of must be controlled reasonably—it is prone to be fallen into local extremum in case the value is too big. Generally speaking it is better to be located within. The new accepted model will get closer to the actual model to be solved with the decrease of the temperature. Therefore, intensifying local search is more favorable to improve the convergence performance of the algorithm.
(2) Cooling method
As for annealing plan, the practice shows that the changing pattern of exponentially is more suitable to the nature of annealing. In consideration of the fact that temperature drops very fast at the beginning then slow down at last in annealing method, the author adopts the modified index curve which is more suitable to the decreasing characteristic, namely
where is the initial temperature, is the given constant and , is the speed factor, is the iterations. With the increase of , the temperature drops very fast at the beginning then slow down at last. Such the large-scaled rough search is utilized at the beginning and then refined search is utilized locally at last.
3. Example Application
At present, China economic is rapid growth. Researching on developing trend of household per capita consumption expenditure of urban and the rural residents in China is of vital significance. What the increasing regularity of household per capita consumption expenditure of urban and the rural residents in China and what kind of development tendency will be presented in the future? The article chooses Gaussian model based on the characteristics of growth for household per capita consumption expenditure of urban and the rural residents in China during 1991-2010. The information refers to Table 1.
The form of model:
|Time (year)||(Yuan)||Predicted value||Relative error (%)||Time (year)||(Yuan)||Predicted value||Relative error (%)|
By utilizing the method in the article, the trigonometric function neural network model will take performance index . The number of hidden layers of neural network, the learning rate , the sample training rate is .
The parameter setting of simulated annealing algorithm is: initial temperature , minimum temperature , the length of Markov chains is 10, temperature drop factor ,the non-uniformity degree constant of model disturbance , and the in cooling method.
By software programming with Matlab, we get
The simulated annealing and neural network algorithm has been applied in data fitting least square integral method. The numerical simulation results show that the method is feasible and valid. The simulated results of simulated annealing neural network algorithm are desirable by judging from the test amount . However, the major defect of the algorithm is the contradiction between the precision of solution and long solving time. The solving time will increase greatly with the increase of inverting parameters. In addition, the scope of variation interval of each parameter has relatively great influence on the convergence rate of algorithm. Consequently, how to confirm effectively the ranges of parameters and some initial parameters of the algorithm so as to increase the convergence rate of algorithm is still to be further studied and discussed.