American Institute of Mathematical Sciences

Advanced
January  2019, 4: 8 doi: 10.1186/s41546-019-0042-6

Nonlinear regression without i.i.d. assumption

Qing Xu and  Xiaohua (Michael) Xuan ,

UniDT, Shanghai, China

In this paper, we consider a class of nonlinear regression problems without the assumption of being independent and identically distributed. We propose a correspondent mini-max problem for nonlinear regression and give a numerical algorithm. Such an algorithm can be applied in regression and machine learning problems, and yields better results than traditional least squares and machine learning methods.
Keywords: Nonlinear regression, Minimax, Independent, Identically distributed, Least squares, Machine learning, Quadratic programming.
Citation: Qing Xu, Xiaohua (Michael) Xuan. Nonlinear regression without i.i.d. assumption. Probability, Uncertainty and Quantitative Risk, 2019, 4 (0) : 8-. doi: 10.1186/s41546-019-0042-6
References:
[1]

Ben-Israel, A. and T.N.E. Greville. (2003). Generalized inverses:Theory and applications (2nd ed.), Springer, New York.,

[2]

Boyd, S., N. Parikh, E. Chu, B. Peleato, and J. Eckstein. (2010). Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Found. Trends Mach. Learn. 3, 1-122.,

[3]

Boyd, S. and L. Vandenberghe. (2004). Convex Optimization, Cambridge University Press. https://doi.org/10.1017/cbo9780511804441.005.,

[4]

Demyanov, V.F. and V.N. Malozemov. (1977). Introduction to Minimax, Wiley, New York.,

[5]

Jin, H. and S. Peng. (2016). Optimal Unbiased Estimation for Maximal Distribution. https://arxiv.org/abs/1611.07994.,

[6]

Kellogg, R.B. (1969). Nonlinear alternating direction algorithm, Math. Comp. 23, 23-38.,

[7]

Kendall, M.G. and A. Stuart. (1968). The Advanced Theory of Statistics, Volume 3:Design and Analysis, and Time-Series (2nd ed.), Griffin, London.,

[8]

Kiwiel, K.C. (1987). A Direct Method of Linearization for Continuous Minimax Problems, J. Optim. Theory Appl. 55, 271-287.,

[9]

Klessig, R. and E. Polak. (1973). An Adaptive Precision Gradient Method for Optimal Control, SIAM J. Control 11, 80-93.,

[10]

Legendre, A.-M. (1805). Nouvelles methodes pour la determination des orbites des cometes, F. Didot, Paris.,

[11]

Lin, L., Y. Shi, X. Wang, and S. Yang. (2016). k-sample upper expectation linear regression-Modeling, identifiability, estimation and prediction, J. Stat. Plan. Infer. 170, 15-26.,

[12]

Lin, L., P. Dong, Y. Song, and L. Zhu. (2017a). Upper Expectation Parametric Regression, Stat. Sin. 27, 1265-1280.,

[13]

Lin, L., Y.X. Liu, and C. Lin. (2017b). Mini-max-risk and mini-mean-risk inferences for a partially piecewise regression, Statistics 51, 745-765.,

[14]

Nocedal, J. and S.J. Wright. (2006). Numerical Optimization, Second Edition, Springer, New York.,

[15]

Panin, V.M. (1981). Linearization Method for Continuous Min-max Problems, Kibernetika 2, 75-78.,

[16]

Peng, S. (2005). Nonlinear expectations and nonlinear Markov chains, Chin. Ann. Math. 26B, no. 2, 159-184.,

[17]

Seber, G.A.F. and C.J. Wild. (1989). Nonlinear Regression, Wiley, New York.,

show all references

References:
[1]

Ben-Israel, A. and T.N.E. Greville. (2003). Generalized inverses:Theory and applications (2nd ed.), Springer, New York.,

[2]

Boyd, S., N. Parikh, E. Chu, B. Peleato, and J. Eckstein. (2010). Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Found. Trends Mach. Learn. 3, 1-122.,

[3]

Boyd, S. and L. Vandenberghe. (2004). Convex Optimization, Cambridge University Press. https://doi.org/10.1017/cbo9780511804441.005.,

[4]

Demyanov, V.F. and V.N. Malozemov. (1977). Introduction to Minimax, Wiley, New York.,

[5]

Jin, H. and S. Peng. (2016). Optimal Unbiased Estimation for Maximal Distribution. https://arxiv.org/abs/1611.07994.,

[6]

Kellogg, R.B. (1969). Nonlinear alternating direction algorithm, Math. Comp. 23, 23-38.,

[7]

Kendall, M.G. and A. Stuart. (1968). The Advanced Theory of Statistics, Volume 3:Design and Analysis, and Time-Series (2nd ed.), Griffin, London.,

[8]

Kiwiel, K.C. (1987). A Direct Method of Linearization for Continuous Minimax Problems, J. Optim. Theory Appl. 55, 271-287.,

[9]

Klessig, R. and E. Polak. (1973). An Adaptive Precision Gradient Method for Optimal Control, SIAM J. Control 11, 80-93.,

[10]

Legendre, A.-M. (1805). Nouvelles methodes pour la determination des orbites des cometes, F. Didot, Paris.,

[11]

Lin, L., Y. Shi, X. Wang, and S. Yang. (2016). k-sample upper expectation linear regression-Modeling, identifiability, estimation and prediction, J. Stat. Plan. Infer. 170, 15-26.,

[12]

Lin, L., P. Dong, Y. Song, and L. Zhu. (2017a). Upper Expectation Parametric Regression, Stat. Sin. 27, 1265-1280.,

[13]

Lin, L., Y.X. Liu, and C. Lin. (2017b). Mini-max-risk and mini-mean-risk inferences for a partially piecewise regression, Statistics 51, 745-765.,

[14]

Nocedal, J. and S.J. Wright. (2006). Numerical Optimization, Second Edition, Springer, New York.,

[15]

Panin, V.M. (1981). Linearization Method for Continuous Min-max Problems, Kibernetika 2, 75-78.,

[16]

Peng, S. (2005). Nonlinear expectations and nonlinear Markov chains, Chin. Ann. Math. 26B, no. 2, 159-184.,

[17]

Seber, G.A.F. and C.J. Wild. (1989). Nonlinear Regression, Wiley, New York.,

[1]

Karl-Peter Hadeler, Frithjof Lutscher. Quiescent phases with distributed exit times. Discrete & Continuous Dynamical Systems - B, 2012, 17 (3) : 849-869. doi: 10.3934/dcdsb.2012.17.849
[2]

Shangzhi Li, Shangjiang Guo. Permanence and extinction of a stochastic SIS epidemic model with three independent Brownian motions. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2693-2719. doi: 10.3934/dcdsb.2020201
[3]

Jean-François Biasse. Improvements in the computation of ideal class groups of imaginary quadratic number fields. Advances in Mathematics of Communications, 2010, 4 (2) : 141-154. doi: 10.3934/amc.2010.4.141
[4]

Marcelo Messias. Periodic perturbation of quadratic systems with two infinite heteroclinic cycles. Discrete & Continuous Dynamical Systems - A, 2012, 32 (5) : 1881-1899. doi: 10.3934/dcds.2012.32.1881
[5]

Ardeshir Ahmadi, Hamed Davari-Ardakani. A multistage stochastic programming framework for cardinality constrained portfolio optimization. Numerical Algebra, Control & Optimization, 2017, 7 (3) : 359-377. doi: 10.3934/naco.2017023
[6]

Raz Kupferman, Cy Maor. The emergence of torsion in the continuum limit of distributed edge-dislocations. Journal of Geometric Mechanics, 2015, 7 (3) : 361-387. doi: 10.3934/jgm.2015.7.361
[7]

Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399
[8]

Yanqin Fang, Jihui Zhang. Multiplicity of solutions for the nonlinear Schrödinger-Maxwell system. Communications on Pure & Applied Analysis, 2011, 10 (4) : 1267-1279. doi: 10.3934/cpaa.2011.10.1267
[9]

Deren Han, Zehui Jia, Yongzhong Song, David Z. W. Wang. An efficient projection method for nonlinear inverse problems with sparsity constraints. Inverse Problems & Imaging, 2016, 10 (3) : 689-709. doi: 10.3934/ipi.2016017
[10]

Olena Naboka. On synchronization of oscillations of two coupled Berger plates with nonlinear interior damping. Communications on Pure & Applied Analysis, 2009, 8 (6) : 1933-1956. doi: 10.3934/cpaa.2009.8.1933
[11]

Thierry Cazenave, Ivan Naumkin. Local smooth solutions of the nonlinear Klein-gordon equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (5) : 1649-1672. doi: 10.3934/dcdss.2020448
[12]

Scipio Cuccagna, Masaya Maeda. A survey on asymptotic stability of ground states of nonlinear Schrödinger equations II. Discrete & Continuous Dynamical Systems - S, 2021, 14 (5) : 1693-1716. doi: 10.3934/dcdss.2020450
[13]

Amit Goswami, Sushila Rathore, Jagdev Singh, Devendra Kumar. Analytical study of fractional nonlinear Schrödinger equation with harmonic oscillator. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021021
[14]

Jiangxing Wang. Convergence analysis of an accurate and efficient method for nonlinear Maxwell's equations. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2429-2440. doi: 10.3934/dcdsb.2020185
[15]

Manoel J. Dos Santos, Baowei Feng, Dilberto S. Almeida Júnior, Mauro L. Santos. Global and exponential attractors for a nonlinear porous elastic system with delay term. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2805-2828. doi: 10.3934/dcdsb.2020206
[16]

Pavel I. Naumkin, Isahi Sánchez-Suárez. Asymptotics for the higher-order derivative nonlinear Schrödinger equation. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021028
[17]

Pengfei Wang, Mengyi Zhang, Huan Su. Input-to-state stability of infinite-dimensional stochastic nonlinear systems. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021066
[18]

Irena PawŃow, Wojciech M. Zajączkowski. Global regular solutions to three-dimensional thermo-visco-elasticity with nonlinear temperature-dependent specific heat. Communications on Pure & Applied Analysis, 2017, 16 (4) : 1331-1372. doi: 10.3934/cpaa.2017065
[19]

Xiaoming Wang. Quasi-periodic solutions for a class of second order differential equations with a nonlinear damping term. Discrete & Continuous Dynamical Systems - S, 2017, 10 (3) : 543-556. doi: 10.3934/dcdss.2017027
[20]

Hong Yi, Chunlai Mu, Guangyu Xu, Pan Dai. A blow-up result for the chemotaxis system with nonlinear signal production and logistic source. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2537-2559. doi: 10.3934/dcdsb.2020194

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences