HULC Exoskeleton by Lockheed Martin

Lockheed Martin has taken over the development of the HULC Exoskeleton originally from Berkeley. It looks pretty slick (see the video below), though it is mainly for the lower body and doesn't provide augmented strength but rather just makes lots of tasks much easier while being lightweight and not adding much bulk. This is the direction I think exoskeletons should go, at least at first.



Did he say "Mantonomous"?

[via Singularity Hub]

Humanoid - The Album

I suppose I am required to listen to this album, huh?

Simulation!

Oh simulation. I have avoided it for so long because a) it's hard to simulate a humanoid robot and b) it doesn't match the real thing anyways. But I got a simulation working this week and even got a little video of the robot getting pushed sideways and taking a step:

Big Robot Pictures

Found a bunch of nice robot photos on the Boston Globe website. They're almost all humanoid robots that perform various entertainment and service purposes. The rest seem to be camouflaged and carrying guns.

Mendeley

I supposed this isn't directly balance-related, but I just got this reference-managing program, Mendeley, which has solved all of my problems. For the longest time, I had been just writing my own bibtex files. Then I was using this program, Jab-ref, for a while. But Mendeley is just way cool. It easily organizes and syncs your references. You can attach PDFs AND highlight and add notes to them! There's even some social-networking functionality built into the website, which is pretty cool. Check out my profile.

Iterative Local Dynamic Programming

I like a good paper. This is a good paper:

• Todorov E and Tassa Y, "Iterative local dynamic programming," in proceedings of the 2nd IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, pp 90 - 95, 2009 (pdf)

Actually, all of the papers on Emanuel Todorov's page are really worth looking over.

They have a very simple algorithm for locally optimizing the trajectory of a nonlinear stochastic system. I have used differential dynamic programming for some time, but I believe this method is simpler and faster.

The key insights are the use of linear basis functions for fast evaluation of the value function partial derivatives and the use of a fixed sample cloud of states around the nominal trajectory that is used to fit the value function and feedback controller.

I've been working on my own matlab implementation, but haven't gotten it to work just yet. It still doesn't solve one of my major criticisms of these non-parametric trajectory optimization approaches: you have to give it a good initial trajectory, which can be difficult in many cases.

Robust/Adaptive/Dual Control

I have been frustrated by our robot model for some time and have spent some effort trying to improve it, but it is a difficult problem. Recently, I've been interested in how we can improve the effectiveness of our controls even if the model is wrong. I would call that robust control. I say that because I don't think there is a clear definition and a lot of people struggle with it, even wikipedia seems to lack an explanation.

When you put the problem in an optimality context, it becomes a little clearer. Basically, you have the problem of performing some task (like balancing) while you are also trying to estimate parameters (like mass distributions or push magnitudes). If you think about it, there should be some optimal motion that allows you to perform that task while at the same time doing your best to improve the estimates. That is dual control (also a lame wikipedia entry - maybe I will improve it!).

There are major drawbacks to dual control, however. First, in it's general form, it is an extremely difficult problem to solve. That is because the solution is basically found using stochastic dynamic programming on both the system state and parameter belief state, leading to an explosion in computation (curse of dimensionality). But there are simplifications. It is almost always performed as a receding horizon control, only considering so many future steps. Some people only look one step ahead, they call this "cautious control."

Even if you are able to solve the problem, it's possible that it's application could lead to bizarre behavior in a real system. This is often due to the fact that the best way to get parameter estimates is to "shake" the system around, exciting the dynamics. This could be undesirable or unstable.

So it's definitely something I'm thinking about and that others should as well. Certainly as our systems become more complex and difficult to model, these techniques will become more important. I will leave you with a few noteworthy citations:

• Wittenmark, Björn, "Adaptive Dual Control Methods: An Overview," In 5th IFAC symposium on Adaptive Systems in Control and Signal Processing, 1995 (pdf)
  
  
• Pieter Abbeel, Morgan Quigley, Andrew Y. Ng, "Using inaccurate models in reinforcement learning," ICML, 2006 (pdf)
  
  
• Simpkins A, de Callafon R and Todorov E, "Optimal trade-off between exploration and exploitation," In proceedings of the American Control Conference, pp 33-38, 2008 (pdf)