A bilevel optimization approach to obtain optimal cost functions for human arm movements

Sebastian Albrecht; Marion  Leibold; Michael Ulbrich

doi:10.3934/naco.2012.2.105

Article Contents

2012, Volume 2, Issue 1: 105-127. Doi: 10.3934/naco.2012.2.105

This issue Previous Article Solvability of a class of thermal dynamical contact problems with subdifferential conditions Next Article An efficient algorithm for convex quadratic semi-definite optimization

A bilevel optimization approach to obtain optimal cost functions for human arm movements

1.
Chair for Mathematical Optimization, Department of Mathematics, Technische Universität München (TUM), Boltzmannstr. 3, 85747 Garching b. München
2.
Institute of Automatic Control Engineering, Technische Universität München (TUM), Arcisstr. 21, 80290 München
3.
Chair for Mathematical Optimization, Department of Mathematics, Technische Universität München (TUM), Boltzmannstr. 3, 85747 Garching b. München

Received: April 2011

Revised: August 2011

Published: March 2012

Abstract / Introduction Related Papers Cited by

Abstract

Using a bilevel optimization approach, we investigate the question how humans plan and execute their arm motions. It is known that human motions are (approximately) optimal for suitable and unknown cost functions subject to the dynamics. We investigate the following inverse problem: Which cost function out of a parameterized family (e.g., convex combinations of functions suggested in the literature) reproduces recorded human arm movements best? The lower level problem is an optimal control problem governed by a nonlinear model of the human arm dynamics. The approach is analyzed for a dynamical 3D model of the human arm. Furthermore, results for a two-dimensional experiment with human probands are presented.

Keywords:

Mathematics Subject Classification: 90C90, 49N45, 49M25.

Citation:

Related Papers

Cited by

References

[1]	P. Abbeel and A. Y. Ng, Apprenticeship learning via inverse reinforcement learning, in "Proc. Int. Conf. on Machine Learning," (2004), 1-8.
[2]	W. Abend, E. Bizzi and P. Morasso, Human arm trajectory formation, Brain, 105 (1982), 331-348.doi: 10.1093/brain/105.2.331.
[3]	S. Albrecht, C. Weber, M. Sobotka, A. Peer, M. Buss and M. Ulbrich, Optimization criteria for human trajectory formation in dynamic virtual environments, in "Haptics: Generating and Perceiving Tangible Sensations," Lecture Notes in Computer Science, 6192 (2010), 257-262.doi: 10.1007/978-3-642-14075-4_37.
[4]	C. G. Atkeson and J. M. Hollerbach, Kinematic features of unrestrained vertical arm movements, J. Neurosci., 5 (1985), 2318-2330.
[5]	J. F. Bard, An algorithm for solving the general bilevel programming problem, Math. Oper. Res., 8 (1983), 260-272.doi: 10.1287/moor.8.2.260.
[6]	J. F. Bard, Convex two-level optimization, Math. Program., 40 (1988), 15-27.doi: 10.1007/BF01580720.
[7]	J. F. Bard, "Practical Bilevel Optimization: Algorithms and Applications," Kluwer Academic Publishers, Dordrecht, 1998.
[8]	A. Billard and M. J. Mataric, Learning human arm movements by imitation: evaluation of a biologically inspired connectionist architecture, Robot. Auton. Syst., 37 (2001), 145-160.doi: 10.1016/S0921-8890(01)00155-5.
[9]	J. Bracken and J.T. McGill, Mathematical programs with optimization problems in the constraints, Oper. Res., 21 (1973), 37-44.doi: 10.1287/opre.21.1.37.
[10]	W. Candler and R. Norton, Multilevel Programming, World Bank Development Research Center, Departml Working Paper, 20 (1977).
[11]	R. F. Chandler, C. E. Claser, J. T. McConville, H. M. Reynolds and J. W. Young, Investigation of inertial properties of the human body, AMLR Technical Report, Wright-Patterson Air Force Base, OH, (1975).
[12]	B. Colson, P. Marcotte and G. Savard, An overview of bilevel optimization, Ann. Oper. Res., 153 (2007), 235-256.doi: 10.1007/s10479-007-0176-2.
[13]	B. Corteville, E. Aertbelien, H. Bruyninckx, J. De Schutter and H. Van Brussel, Human-inspired robot assistant for fast point-to-point movements, in "Proc. IEEE ICRA," (2007), 3639-3644.
[14]	S. Dempe, A necessary and a sufficient optimality condition for bilevel programming problems, Optimization, 25 (1992), 341-354.doi: 10.1080/02331939208843831.
[15]	S. Dempe, "Foundations of Bilevel Programming," Kluwer Academic Publishers, Dordrecht, 2002.
[16]	S. Dempe, Annotated bibliography on bilevel programming and mathematical programs with equilibrium constraints, Optimization, 52 (2003), 333-359.doi: 10.1080/0233193031000149894.
[17]	P. Deuflhard and F. Bornemann, "Scientific Computing with Ordinary Differential Equations," Springer, Berlin, 2002.
[18]	M. Diehl, and D.B. Leineweber and A.A.S. Schäfer, "MUSCOD-II User's Manual," IWR-Preprint, University of Heidelberg, Germany, 2001.
[19]	T. A. Edmunds and J. F. Bard, Algorithms for nonlinear bilevel mathematical programs, IEEE Trans. Syst. Man Cyb., 21 (1991), 83-89.doi: 10.1109/21.101139.
[20]	R. Featherstone, "Robot Dynamics Algorithms," Kluwer Academic Publishers, Dordrecht, 1987.
[21]	T. Flash and N. Hogan, The coordination of arm movements: an experimentally confirmed mathematical model, J. Neurosci., 5 (1985), 1688-1703.
[22]	M. Gleicher, Retargetting motion to new characters, in "Proc. Int. Conf. on Computer Graphics and Interactive Techniques," (1998), 33-42.
[23]	E. Hairer, S. P. N{\o}rsett and G. Wanner, "Solving Ordinary Differential Equations," Springer, New York, 1993.
[24]	C. M. Harris and D. M. Wolpert, Signal-dependend noise determines motor planning, Nature, 394 (1998), 780-784.doi: 10.1038/29528.
[25]	H. Hatze, Neuromusculoskeletal control systems modeling - a critical survey of recent developments, IEEE Trans. Autom. Control, 25 (1980), 375-385.doi: 10.1109/TAC.1980.1102380.
[26]	A. V. Hill, The heat of shortening and the dynamic constants of muscle, Proc. Roy. Soc. B, 126 (1938), 136-195.doi: 10.1098/rspb.1938.0050.
[27]	K. W. Lilly, "Efficient Dynamic Simulation of Robotic Mechanisms," Kluwer Academic Publishers, Dordrecht, 1993.
[28]	C. K. Liu, A. Hertzmann and Z. Popovi\'c, Learning physics-based motion style with nonlinear inverse optimization, ACM Trans. Graph., 24 (2005), 1071-1081.doi: 10.1145/1073204.1073314.
[29]	Y. Maeda, T. Hara and T. Arai, Human-robot cooperative manipulation with motion estimation, in "Proc. IEEE IROS", 4 (2001), 2240-2245.
[30]	P. Marcotte and G. Savard, Bilevel programming: a combinatorial perspective, in "Graph Theory and Combinatorial Optimization" (eds. D. Avis, A. Hertz, and O. Marcotte), Springer, New York (2005), 191-217.doi: 10.1007/0-387-25592-3_7.
[31]	K. Mombaur, A. Truong and J. P. Laumond, From human to humanoid locomotion - an inverse optimal control approach, Auton. Robot., 28 (2010), 369-383.doi: 10.1007/s10514-009-9170-7.
[32]	P. Morasso, Spatial control of arm movements, Exp. Brain Res., 42 (1981), 223-227.doi: 10.1007/BF00236911.
[33]	P. Morasso, Three dimensional arm trajectories, Biol. Cybern., 48 (1983), 187-194.doi: 10.1007/BF00318086.
[34]	E. Nakano, H. Imamizu, R. Osu, Y. Uno, H. Gomi, T. Yoshioka and M. Kawato, Quanitative examinations of internal representations for arm trajectory planning: Minimum commanded torque change model, J. Neurophysiol., 81 (1999), 2140-2155.
[35]	W. L. Nelson, Physical principles for economies of skilled movements, Biol. Cybern., 46 (1983), 135-147.doi: 10.1007/BF00339982.
[36]	A. Y. Ng and S. Russell, Algorithms for inverse reinforcement learning, in "Proc. Int. Conf. on Machine Learning," (2000), 663-670.
[37]	J. Nocedal and S. J. Wright, "Numerical Optimization," Second Edition, Springer, New York, 2006.
[38]	J. V. Outrata, On the numerical solution of a class of Stackelberg problems, Z. Oper. Res., 34 (1990), 255-277.
[39]	J. V. Outrata, M. Kocvara and J. Zowe, "Nonsmooth Approach to Optimization Problems with Equilibrium Constraints," Kluwer Academic Publishers, Dordrecht, 1998.
[40]	M. J. D. Powell, The BOBYQA algorithm for bound constrained optimization without derivatives, Cambridge DAMTP Report NA2009/06, University of Cambridge, Cambridge, 2009.
[41]	D. Ramachandran and E. Amir, Bayesian inverse reinforcement learning, in "Proc. Int. Conf. on Artificial Intelligence," (2007), 2586-2591.
[42]	N. D. Ratliff, J. A. Bagnell and M. A. Zinkevich, Maximum margin planning, in "Proc. Int. Conf. on Machine Learning," (2006), 729-736.
[43]	S. Russell, Learning agents for uncertain environments (extended abstract), in "Proc. Int. Conf. on Computational Learning Theory," (1998), 101-103.
[44]	S. Schaal, Is imitation learning the route to humanoid robots?, Trends in Cognitive Sciences, 3 (1999), 233-242.doi: 10.1016/S1364-6613(99)01327-3.
[45]	K. Shimizu and M. Lu, A global optimization method for the Stackelberg problem with convex functions via problem transformation and concave programming, IEEE Trans. Syst. Man Cyb., 25 (1995), 1635-1640.doi: 10.1109/21.478449.
[46]	C. Smith and H. I. Christensen, A minimum jerk predictor for teleoperation with variable time delay, in "Proc. IEEE IROS," (2009), 5621-5627.
[47]	S. Stroeve, Impedance characteristics of neuromusculoskeletal model of the human arm, Biol. Cybern., 81 (1999), 475-494.doi: 10.1007/s004220050577.
[48]	W. Suleiman, E. Yoshida, F. Kanehiro, J. P. Laumond and A. Monin, On human motion imitation by humanoid robot, in "Proc. IEEE ICRA," (2008), 2697-2704.
[49]	E. Todorov, Stochastic optimal control and estimation methods adapted to the noise characteristics of the sensorimotor system, Neural. Comput., 17 (2005), 1084-1108.doi: 10.1162/0899766053491887.
[50]	E. Todorov, Optimality principles in sensorimotor control, Nature Neurosci., 7 (2004), 907-915.doi: 10.1038/nn1309.
[51]	Y. Uno, M. Kawato and R. Suzuki, Formation and control of optimal trajectory in human multijoint arm movement, Biol. Cybern., 61 (1989), 89-101.doi: 10.1007/BF00204593.
[52]	Y. Uno, R. Suzuki and M. Kawato, Minimum muscle-tension-change model which reproduces human arm movement, in "Proc. 4th Symp. Biol. Physiol. Eng.," (1989), 299-302.
[53]	L. N. Vicente and P. H. Calamai, Bilevel and multilevel programming: a bibliography review, J Global. Optim., 5 (1994), 291-306.doi: 10.1007/BF01096458.
[54]	A. Wächter and L.T. Biegler, Line search filter methods for nonlinear programming: motivation and global convergence, SIAM J. Optim., 16 (2005), 1-31.
[55]	A. Wächter and L. T. Biegler, On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming, Math. Program., 106 (2006), 25-57.
[56]	D. A. Winter, "Biomechanics and Motor Control of Human Movement," Third Edition, Wiley, New York, 2005.
[57]	J. M. Winters and L. Stark, Analysis of fundamental human movement patterns through the use of in-depth antagonistic muscle models, IEEE Trans. Biomedical Eng., 10 (1985), 826-839.doi: 10.1109/TBME.1985.325498.
[58]	J. J. Ye and D. L. Zhu, Optimality conditions for bilevel programming problems, Optimization, 33 (1995), 9-27.doi: 10.1080/02331939508844060.
[59]	T. Yoshikawa, "Foundations of Robotics: Analysis and Control," MIT Press, Cambridge, 1990.