American Institute of Mathematical Sciences

Advanced
January  2019, 4: 8 doi: 10.1186/s41546-019-0042-6

Nonlinear regression without i.i.d. assumption

Qing Xu and  Xiaohua (Michael) Xuan ,

UniDT, Shanghai, China

In this paper, we consider a class of nonlinear regression problems without the assumption of being independent and identically distributed. We propose a correspondent mini-max problem for nonlinear regression and give a numerical algorithm. Such an algorithm can be applied in regression and machine learning problems, and yields better results than traditional least squares and machine learning methods.
Keywords: Nonlinear regression, Minimax, Independent, Identically distributed, Least squares, Machine learning, Quadratic programming.
Citation: Qing Xu, Xiaohua (Michael) Xuan. Nonlinear regression without i.i.d. assumption. Probability, Uncertainty and Quantitative Risk, 2019, 4 (0) : 8-. doi: 10.1186/s41546-019-0042-6
References:
[1]

Ben-Israel, A. and T.N.E. Greville. (2003). Generalized inverses:Theory and applications (2nd ed.), Springer, New York.,

[2]

Boyd, S., N. Parikh, E. Chu, B. Peleato, and J. Eckstein. (2010). Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Found. Trends Mach. Learn. 3, 1-122.,

[3]

Boyd, S. and L. Vandenberghe. (2004). Convex Optimization, Cambridge University Press. https://doi.org/10.1017/cbo9780511804441.005.,

[4]

Demyanov, V.F. and V.N. Malozemov. (1977). Introduction to Minimax, Wiley, New York.,

[5]

Jin, H. and S. Peng. (2016). Optimal Unbiased Estimation for Maximal Distribution. https://arxiv.org/abs/1611.07994.,

[6]

Kellogg, R.B. (1969). Nonlinear alternating direction algorithm, Math. Comp. 23, 23-38.,

[7]

Kendall, M.G. and A. Stuart. (1968). The Advanced Theory of Statistics, Volume 3:Design and Analysis, and Time-Series (2nd ed.), Griffin, London.,

[8]

Kiwiel, K.C. (1987). A Direct Method of Linearization for Continuous Minimax Problems, J. Optim. Theory Appl. 55, 271-287.,

[9]

Klessig, R. and E. Polak. (1973). An Adaptive Precision Gradient Method for Optimal Control, SIAM J. Control 11, 80-93.,

[10]

Legendre, A.-M. (1805). Nouvelles methodes pour la determination des orbites des cometes, F. Didot, Paris.,

[11]

Lin, L., Y. Shi, X. Wang, and S. Yang. (2016). k-sample upper expectation linear regression-Modeling, identifiability, estimation and prediction, J. Stat. Plan. Infer. 170, 15-26.,

[12]

Lin, L., P. Dong, Y. Song, and L. Zhu. (2017a). Upper Expectation Parametric Regression, Stat. Sin. 27, 1265-1280.,

[13]

Lin, L., Y.X. Liu, and C. Lin. (2017b). Mini-max-risk and mini-mean-risk inferences for a partially piecewise regression, Statistics 51, 745-765.,

[14]

Nocedal, J. and S.J. Wright. (2006). Numerical Optimization, Second Edition, Springer, New York.,

[15]

Panin, V.M. (1981). Linearization Method for Continuous Min-max Problems, Kibernetika 2, 75-78.,

[16]

Peng, S. (2005). Nonlinear expectations and nonlinear Markov chains, Chin. Ann. Math. 26B, no. 2, 159-184.,

[17]

Seber, G.A.F. and C.J. Wild. (1989). Nonlinear Regression, Wiley, New York.,

show all references

References:
[1]

Ben-Israel, A. and T.N.E. Greville. (2003). Generalized inverses:Theory and applications (2nd ed.), Springer, New York.,

[2]

Boyd, S., N. Parikh, E. Chu, B. Peleato, and J. Eckstein. (2010). Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Found. Trends Mach. Learn. 3, 1-122.,

[3]

Boyd, S. and L. Vandenberghe. (2004). Convex Optimization, Cambridge University Press. https://doi.org/10.1017/cbo9780511804441.005.,

[4]

Demyanov, V.F. and V.N. Malozemov. (1977). Introduction to Minimax, Wiley, New York.,

[5]

Jin, H. and S. Peng. (2016). Optimal Unbiased Estimation for Maximal Distribution. https://arxiv.org/abs/1611.07994.,

[6]

Kellogg, R.B. (1969). Nonlinear alternating direction algorithm, Math. Comp. 23, 23-38.,

[7]

Kendall, M.G. and A. Stuart. (1968). The Advanced Theory of Statistics, Volume 3:Design and Analysis, and Time-Series (2nd ed.), Griffin, London.,

[8]

Kiwiel, K.C. (1987). A Direct Method of Linearization for Continuous Minimax Problems, J. Optim. Theory Appl. 55, 271-287.,

[9]

Klessig, R. and E. Polak. (1973). An Adaptive Precision Gradient Method for Optimal Control, SIAM J. Control 11, 80-93.,

[10]

Legendre, A.-M. (1805). Nouvelles methodes pour la determination des orbites des cometes, F. Didot, Paris.,

[11]

Lin, L., Y. Shi, X. Wang, and S. Yang. (2016). k-sample upper expectation linear regression-Modeling, identifiability, estimation and prediction, J. Stat. Plan. Infer. 170, 15-26.,

[12]

Lin, L., P. Dong, Y. Song, and L. Zhu. (2017a). Upper Expectation Parametric Regression, Stat. Sin. 27, 1265-1280.,

[13]

Lin, L., Y.X. Liu, and C. Lin. (2017b). Mini-max-risk and mini-mean-risk inferences for a partially piecewise regression, Statistics 51, 745-765.,

[14]

Nocedal, J. and S.J. Wright. (2006). Numerical Optimization, Second Edition, Springer, New York.,

[15]

Panin, V.M. (1981). Linearization Method for Continuous Min-max Problems, Kibernetika 2, 75-78.,

[16]

Peng, S. (2005). Nonlinear expectations and nonlinear Markov chains, Chin. Ann. Math. 26B, no. 2, 159-184.,

[17]

Seber, G.A.F. and C.J. Wild. (1989). Nonlinear Regression, Wiley, New York.,

[1]

Peter Frolkovič, Karol Mikula, Jooyoung Hahn, Dirk Martin, Branislav Basara. Flux balanced approximation with least-squares gradient for diffusion equation on polyhedral mesh. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 865-879. doi: 10.3934/dcdss.2020350
[2]

Kengo Nakai, Yoshitaka Saiki. Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1079-1092. doi: 10.3934/dcdss.2020352
[3]

Ke Su, Yumeng Lin, Chun Xu. A new adaptive method to nonlinear semi-infinite programming. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2021012
[4]

Ke Yang, Wencheng Zou, Zhengrong Xiang, Ronghao Wang. Fully distributed consensus for higher-order nonlinear multi-agent systems with unmatched disturbances. Discrete & Continuous Dynamical Systems - S, 2021, 14 (4) : 1535-1551. doi: 10.3934/dcdss.2020396
[5]

Gernot Holler, Karl Kunisch. Learning nonlocal regularization operators. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021003
[6]

Divine Wanduku. Finite- and multi-dimensional state representations and some fundamental asymptotic properties of a family of nonlinear multi-population models for HIV/AIDS with ART treatment and distributed delays. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021005
[7]

Riccarda Rossi, Ulisse Stefanelli, Marita Thomas. Rate-independent evolution of sets. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 89-119. doi: 10.3934/dcdss.2020304
[8]

Min Ji, Xinna Ye, Fangyao Qian, T.C.E. Cheng, Yiwei Jiang. Parallel-machine scheduling in shared manufacturing. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020174
[9]

Peng Luo. Comparison theorem for diagonally quadratic BSDEs. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020374
[10]

Ying Lin, Qi Ye. Support vector machine classifiers by non-Euclidean margins. Mathematical Foundations of Computing, 2020, 3 (4) : 279-300. doi: 10.3934/mfc.2020018
[11]

Yuyuan Ouyang, Trevor Squires. Some worst-case datasets of deterministic first-order methods for solving binary logistic regression. Inverse Problems & Imaging, 2021, 15 (1) : 63-77. doi: 10.3934/ipi.2020047
[12]

Huu-Quang Nguyen, Ya-Chi Chu, Ruey-Lin Sheu. On the convexity for the range set of two quadratic functions. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020169
[13]

Wei-Chieh Chen, Bogdan Kazmierczak. Traveling waves in quadratic autocatalytic systems with complexing agent. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020364
[14]

Håkon Hoel, Gaukhar Shaimerdenova, Raúl Tempone. Multilevel Ensemble Kalman Filtering based on a sample average of independent EnKF estimators. Foundations of Data Science, 2020, 2 (4) : 351-390. doi: 10.3934/fods.2020017
[15]

Nicholas Geneva, Nicholas Zabaras. Multi-fidelity generative deep learning turbulent flows. Foundations of Data Science, 2020, 2 (4) : 391-428. doi: 10.3934/fods.2020019
[16]

Sihem Guerarra. Maximum and minimum ranks and inertias of the Hermitian parts of the least rank solution of the matrix equation AXB = C. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 75-86. doi: 10.3934/naco.2020016
[17]

Yahia Zare Mehrjerdi. A new methodology for solving bi-criterion fractional stochastic programming. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020054
[18]

Touria Karite, Ali Boutoulout. Global and regional constrained controllability for distributed parabolic linear systems: RHUM approach. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020055
[19]

Feifei Cheng, Ji Li. Geometric singular perturbation analysis of Degasperis-Procesi equation with distributed delay. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 967-985. doi: 10.3934/dcds.2020305
[20]

Yasmine Cherfaoui, Mustapha Moulaï. Biobjective optimization over the efficient set of multiobjective integer programming problem. Journal of Industrial & Management Optimization, 2021, 17 (1) : 117-131. doi: 10.3934/jimo.2019102

 Impact Factor: 

Tools

Metrics

  • PDF downloads (6)
  • HTML views (98)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences