Over the past two decades, rapid advances in computer performance have not been matched with similar advances in the development of intelligent robots and systems. Although humans are quite adept at mastering complex and dynamic skills, we are far less impressive in formalizing our behavior into algorithmic, machinecodable strategies. Therefore, it has been difficult to duplicate the types of intelligent skills and actions, we witness every day as humans, in robots and other machines. This not only limits the capabilities of individual robots, but also the extent to which humans and robots can safely interact and cooperate with one another. Nevertheless, human actions are currently our only examples of truly “intelligent” behavior. As such, there exists a profound need to abstract human skill into computational models, capable of realistic emulation of dynamic human behavior.
Models of human skill can transfer intelligent control behaviors to robots. This is especially critical for robots which have to operate in remote or inhospitable environments, which humans cannot reach. For example, sending humans to operate in space is often expensive, dangerous, or outright impossible, while sending robots instead is comparatively cheap and involves no risk to human life. Replacing astronauts with robots is only feasible, however, if the robots are equipped with sufficient autonomous skill and intelligence; we suggest that a robot can acquire those necessary capabilities through abstracted models of human skill.
Contents
Abstract
Acknowledgments
1 Introduction
1.1 Motivation
1.2 Related work
- 1.2.1 Skill learning through exploration
1.2.2 Skill modeling from human data
1.2.3 Neural network learning
1.2.4 Locally weighted learning
1.3 Overview
2 Experimental design
2.1 Motivation
2.2 Simulation environment
- 2.2.1 Dynamic driving simulator
2.2.2 Road descriptions
2.3 Model class
3 Cascade Neural Networks with Kalman Filtering
3.1 Motivation
3.2 Cascade neural networks
3.3 Node-decoupled extended Kalman filtering
- 3.3.1 Learning architecture
3.3.2 Computational complexity
3.4 Comparison experiments
- 3.4.1 Problem descriptions
3.4.2 Learning results
3.4.3 Noisy learning results
3.4.4 Discussion
4 HCS Models: Continuous Learning
4.1 Cascade with quickprop learning
- 4.1.1 Experimental data
4.1.2 Model inputs and outputs
4.1.3 Cq training
4.1.4 HCS models
4.2 Cascade with NDEKF learning
- 4.2.1 Model inputs and outputs
4.2.2 Ck training
4.2.3 HCS models
4.3 Analysis
- 4.3.1 Model stability
4.3.2 Learning convergence
4.3.3 Discussion
5 HCS Models: Discontinuous Learning
5.1 Hybrid continuous/discontinuous control
- 5.1.1 General statistical framework
5.1.2 Action definitions
5.1.3 Statistical model choice
5.1.4 Prior probabilities
5.1.5 Task-based modifications
5.2 Experimental results
- 5.2.1 Model training
5.2.2 HCS models
5.3 Analysis
- 5.3.1 Sample curve control
5.3.2 Probability profile
5.3.3 Modeling extension
6 Model validation
6.1 Need for model validation
6.2 Stochastic similarity measure
- 6.2.1 Hidden Markov Models
6.2.2 Similarity measure
6.2.3 Properties
6.2.4 Distance measure
6.2.5 Data preprocessing
6.2.6 Vector quantization
6.2.7 Discretization compensation
6.2.8 HMM training
7 Human-to-human similarity
7.1 Comparing human control strategies
- 7.1.1 Experimental data
7.1.2 Classification experiments
7.1.3 Bayes classification
7.1.4 Spectral classification
7.1.5 Task-based classification
7.1.6 Classification with performance drift
7.2 Comparing Navlab driving data
- 7.2.1 Experimental data
7.2.2 Classification experiments
7.3 Analysis
- 7.3.1 One-sided similarity measure
7.3.2 Bayes classifier limitations
7.3.3 Similarity measure variations
8 Human-to-model validation
8.1 Human-to-model comparisons
8.2 Classification experiment
8.3 Model inputs revisited
9 Conclusion
9.1 Contributions
9.2 Future work
Appendix A Human control data
A.1 Larry
A.2 Curly
A.3 Moe
A.4 Groucho
A.5 Harpo
A.6 Zeppo
Appendix B HMM Training
B.1 Forward-backward algorithm
B.2 Baum-Welch algorithm (with scaling)
Appendix C Author’s Publications
Bibliography
Download
Learning and Validation of Human Control Strategies
