Learning rates for partially linear functional models with high dimensional scalar covariates

Yifan Xia; Yongchao Hou; Xin He; Shaogao Lv

doi:10.3934/cpaa.2020172

Article Contents

2020, Volume 19, Issue 8: 3917-3932. Doi: 10.3934/cpaa.2020172

This issue Previous Article Preface to the special issue on analysis in machine learning and data science Next Article Sparse generalized canonical correlation analysis via linearized Bregman method

Learning rates for partially linear functional models with high dimensional scalar covariates

1.
School of Statistics, Southwestern University of Finance and Economics, Chengdu, 611130, China
2.
School of Statistics and Management, Shanghai University of Finance and Economics, Shanghai, China
3.
College of Statistics and Mathematics, Nanjing Audit University, Nanjing, China

^* Corresponding author
^* Corresponding author

Received: December 31, 2018

Revised: March 31, 2019

Published: May 2020

This work is partly supported by the National Social Science Fund of China(NSSFC-16BTJ013, NSSFC-16ZDA010), Sichuan Social Science Fund (SC14B091) and Sichuan Project of Science and Technology(2017JY0273). Shaogao Lv is the corresponding author and his research is partially supported by NSFC-11871277

Abstract Full Text(HTML) Figure(0) / Table(1) Related Papers Cited by

Abstract

This paper is concerned with learning rates for partial linear functional models (PLFM) within reproducing kernel Hilbert spaces (RKHS), where all the covariates consist of two parts: functional-type covariates and scalar ones. As opposed to frequently used functional principal component analysis for functional models, the finite number of basis functions in the proposed approach can be generated automatically by taking advantage of reproducing property of RKHS. This avoids additional computational costs on PCA decomposition and the choice of the number of principal components. Moreover, the coefficient estimators with bounded covariates converge to the true coefficients with linear rates, as if the functional term in PLFM has no effect on the linear part. In contrast, the prediction error for the functional estimator is significantly affected by the ambient dimension of the scalar covariates. Finally, we develop the proposed numerical algorithm for the proposed penalized approach, and some simulated experiments are implemented to support our theoretical results.

Keywords:

Mathematics Subject Classification: Primary: 58F15, 58F17; Secondary: 53C35.

Citation:

Full Text(HTML)

Table 1. The averaged performance measures of the proposed method in simulated example

(n, v)	$ \\|\widehat{{\bf {\gamma} }}-{\bf {\gamma} }^0\\|_2$	$ {E}\\|\widehat{f}-f^0\\| $	$ \\|\widehat{{\mathop{\bf y}}}-{\mathop{\bf y}}\\|_2 $
(100, 1.1)	0.3401 (0.0166)	4.0851 (0.0085)	4.3526 (0.0151)
(100, 2)	0.3338 (0.0166)	4.0578 (0.0085)	4.3226 (0.0151)
(100, 4)	0.3232 (0.0166)	4.0329 (0.0085)	4.2887 (0.0151)
(200, 1.1)	0.2235 (0.0136)	4.0797 (0.0088)	4.2708 (0.0137)
(200, 2)	0.2230 (0.0134)	4.0540 (0.0089)	4.2460 (0.0134)
(200, 4)	0.2166 (0.0119)	4.0313 (0.0087)	4.2185 (0.0126)

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	N. Aronszajn, Theory of reporudcing kernels, Trans. Amer. Math. Soc., 68 (1950), 337-404. doi: 10.2307/1990404.
[2]	P. J. Bickel, Y. Ritov and A. B. Tsybakov, Simultaneous analysis of Lasso and Dantzig selector, Ann. Statist., 37 (2009), 1705-1732. doi: 10.1214/08-AOS620.
[3]	O. Bousquet, A Bennet concentration inequality and its application to suprema of empirical processes, C. R. Math. Acad. Sci. Paris, 334 (2002), 495-550. doi: 10.1016/S1631-073X(02)02292-6.
[4]	S. Boyd and L. Vandenberghe, Convex optimization, Cambridge University Press, Cambridge, 2004. doi: 10.1017/CBO9780511804441.
[5]	P. Buhlmann and S. van de Geer, Statistics for High-Dimensional Data: Methods, Theory and Applications, Springer-Verlag Berlin Heidelberg, 2011. doi: 10.1007/978-3-642-20192-9.
[6]	T. T. Cai and P. Hall, Prediction in functional linear regression, Ann. Statist., 34 (2006), 2159-2179. doi: 10.1214/009053606000000830.
[7]	T. T. Cai and M. Yuan, Minimax and adaptive prediction for functional linear regression, J. Amer. Statist. Assoc., 107 (2012), 1201-1216. doi: 10.1080/01621459.2012.716337.
[8]	A. Cuevas, M. Febrero and R. Fraiman, Linear functional regression: The case of fixed design and functional response, Canadian J. Statist., 30 (2002), 285-300. doi: 10.2307/3315952.
[9]	D. Donoho, Compressed sensing, IEEE. Trans. Inform. Theory, 52 (2006), 1289-1306. doi: 10.1109/TIT.2006.871582.
[10]	F. Ferraty and P. Vieu, Nonparametric Functional Data Analysis: Theory and Practice, Springer, New York, 2006.
[11]	X. Ji, J. W. Han, X. Jiang, X. T. Hu, L. Guo, J. G. Han, L. Shao and T. M. Liu, Analysis of music/speech via integration of audio content and functional brain response, Inform. Sci., 297 (2015), 271-282. doi: 10.1016/j.ins.2014.11.020.
[12]	D. H. Kong, J. G. Ibrahim, E. Lee and H. Zhu, FLCRM: Functional linear cox regression model, Biometrics, 74 (2018), 109-117. doi: 10.1111/biom.12748.
[13]	D. H. Kong, K. J. Xue, F. Yao and H. H. Zhang, Partially functional linear regression in high dimensions, Biometrika, 103 (2016), 147-159. doi: 10.1093/biomet/asv062.
[14]	M. Ledoux, The Concentration of Measure Phenomenon, Mathematical Surveys and Monographs, American Mathematical Society, Providence, RI, 2001.
[15]	P. Muller and S. Van de Geer, The partial linear model in high dimensions, Scand. J. Statist., 42 (2015), 580-608. doi: 10.1111/sjos.12124.
[16]	C. Preda and G. Saporta, PLS regression on a stochastic process, Comput. Statist. Data Anal., 48 (2005), 149-158. doi: 10.1016/j.csda.2003.10.003.
[17]	J. O. Ramsay and B. W. Silverman, Functional Data Analysis, 2$^nd$ edition, Springer, New York, 2005.
[18]	E. Sanchez-Lozano, G. Tzimiropoulos, B. Martinez, F. De la Torre and M. Valstar, A functional regression approach to facial landmark tracking, IEEE Trans. Pattern Anal. Mach. Intell., 40 (2018), 2037-2050. doi: 10.1109/TPAMI.2017.2745568.
[19]	H. Shin and M. H. Lee, On prediction rate in partial functional linear regression, J. Multi. Anal., 103 (2012), 93-106. doi: 10.1016/j.jmva.2011.06.011.
[20]	M. Talagrand, New concentration inequalities in product spaces, Invent. Math., 126 (1996), 505-563. doi: 10.1007/s002220050108.
[21]	Q. G. Tang and L. S. Cheng, Partial functional linear quantile regression, Sci. China Math., 57 (2014), 2589-2608. doi: 10.1007/s11425-014-4819-x.
[22]	J. P. Thivierge, Functional data analysis of cognitive events in EEG, IEEE Int. Confer. Syst. Man Cyber., (2007), 2473–2478. doi: 10.1109/ICSMC.2007.4413811.
[23]	R. J. Tibshirani, Regression shrinkage and selection via the lasso, J. R. Statist. Soc. B, 58 (1996), 267-288.
[24]	S. Van. de. Geer, Emprical Processes in M-Estimation, Cambridge University Press, New York, 2000.
[25]	M. J. Wainwright, Sharp thresholds for high-dimensional and noisy sparsity recovery using $\ell_1$-constrainted quadratic programming, IEEE Trans. Inf. Theory, 55 (2009), 2183-2202. doi: 10.1109/TIT.2009.2016018.
[26]	F. Yao, H. G. Muller and J. L. Wang, Functional data analysis for sparse longitudinal data, J. Amer. Statist. Assoc., 100 (2005), 577-590. doi: 10.1198/016214504000001745.
[27]	Y. Yuan and T. T. Cai, A reproducing kernel Hilbert space approach to functional linear regression, Ann. Statist., 38 (2006), 3412-3444. doi: 10.1214/09-AOS772.
[28]	D. D. Yu, L. L. Kong and I. Mizera, Partial functional linear quantile regression for neuroimaging data analysis, Neurocomputing, 195 (2016), 74-87. doi: 10.1016/j.neucom.2015.08.116.