Dean F. Hougen, Learning and Robotics

Dean F. Hougen

Research in Learning and Robotics

While taking a neural networks course taught by James Slagle, I became interested in the combination of connectionist networks and reinforcement learning, thanks to the seminal paper "Neuronlike Adaptive Elements that can Solve Difficult Learning Control Problems" by Andrew G. Barto, Richard S. Sutton, and Charles W. Anderson (IEEE Transactions on Systems, Man, and Cybernetics, SMC-13, pages 834-846, 1983). In this paper, the authors apply their Associative Search Element/Adaptive Critic System to the pole-balancing problem. So, for a class project, I applied their system to a different problem -- learning to track a "bug."

That experience led me to try to invent my own system which could also solve the pole-balancing problem. I did this for an independent-study course with Dr. Slagle and the result (tested in simulation) was so positive that I decided to publish it and this became my first professional publication.

Dean Hougen. "Use of an Eligibility Trace of Self-Organize Output." In Science of Artificial Neural Networks II, Proceedings of SPIE 1966, pages 436-447, 1993.

Presenting that paper to a research group here in the Artificial Intelligence, Robotics, and Vision Laboratory (AIRVL), an undergraduate by the name of John Fischer became excited about the possibility of implementing my system on a real robot. Because of John's background in electrical engineering, he was just the person to build such a robot.

I quickly learned how such an objective can be inspirational to other students and was joined in the effort by additional undergraduates and a high-school student in an advanced program. Our collaboration resulted in a publication for myself and the two other most active contributors to the project.

Dean Hougen, John Fischer, and Deva Johnam. "A Neural Network Pole-Balancer that Learns and Operates on a Real Robot in Real Time." In Proceedings of the Machine-Learning Conference-Conference on Learning Theory, Workshop on Robot Learning, pages 73-80, 1994.

John and I traveled to New Jersey to present this paper and participate in an exhibition of our robot there and, with money I was fortunate enough to get from AAAI, John, Deva, and I traveled to Seattle to present our robot in the AAAI Robot Competition and Exhibition. This was the first time that the University of Minnesota had participated in the AAAI Robot Competition and Exhibition but, thanks to the excitement we generated, it wasn't the last!

Unfortunately, the problem with pole-balancing in a real robot is that the repeated reversals of direction are hard on the gearbox. So, pole-balancing was set aside and we decided to work on a similar problem, trailer-backing. John again worked on the building the robot itself, while I worked on the learning system. By this time I had decided this would be my thesis work with Jim Slagle as one advisor and Maria Gini, our robotics expert, as my other. Together we produced the following paper.

Dean F. Hougen, John Fischer, Maria Gini, and James Slagle. "Fast Connectionist Learning for Trailer Backing using a Real Robot." In Proceedings of the IEEE International Conference on Robotics and Automation, pages 1917-1922, April 1996. [ps; pdf]

Unfortunately, John's custom hardware didn't last and, because it hadn't been properly documented, no one could repair it after John graduated and left the project. So, it was back to simulation for the time-being.

First, I extended the learning system to handle a truck with two trailers.

Dean F. Hougen, Maria Gini, and James Slagle. "Rapid, Unsupervised Connectionist Learning for Backing a Robot with Two Trailers." In Proceedings of the IEEE International Conference on Robotics and Automation, pages 2950-2955, April 1997. [ps; pdf]

Then I considered the trade-off between a learned and an assigned partitioning of the input space.

Dean F. Hougen, Maria Gini, and James Slagle. "Partitioning Input Space for Reinforcement Learning for Control." In Proceedings of the IEEE International Conference on Neural Networks, pages 755-760, June 1997. [ps; pdf]

Fortunately, I was soon able to team up with Paul Rybski, a graduate student. Paul had experience building robots from LEGO with HandyBoard controllers. With Paul's new trailer-backing robot we were quickly back in the business of learning on real robots.

One of the first things we noticed was that, while the overall success rate of the robotic system and the simulation were very similar, the systems just didn't "look" the same when viewed in action. We were able to quantify this difference as well as make some suggestions that other roboticists might find of use.

	Dean F. Hougen, Paul E. Rybski, and Maria Gini. "Repeatability of Real World Training Experiments: a Case Study." In AAAI Spring Symposium on Integrating Robotics Research: Taking the Next Leap, March 1998. [ps; pdf]
	Dean F. Hougen, Paul E. Rybski, and Maria Gini. Repeatability of Real World Training Experiments: A Case Study. Autonomous Robots, 6(3): 281-292, 1999. [ps; pdf]

Naturally, I put all of this together with much more and completed my degree.

Dean F. Hougen. Connectionist Reinforcement Learning for Control of Robotic Systems. Doctoral Thesis, University of Minnesota, October 1998. [ps; pdf]

Of course, more new material continues to come out of this line of research. Last year I published a comparison of two input partitioning methods and there is more to come.

Dean F. Hougen, Maria Gini, and James Slagle. "An Integrated Connectionist Approach to Reinforcement Learning for Robotic Control: The Advantages of Indexed Partitioning." In The Seventeenth International Conference on Machine Learning, pages 383-390, June 2000. [ps; pdf]

The views and opinions expressed in this page are strictly those of the page author. The contents of this page have not been reviewed or approved by the University of Oklahoma.

Last update: 21 August 2001