I actually came here to make a post on an Idea that I had, but though I'll give an update on my work first:
I used webots and imported maps from yahoo into a large flat piece of land with the provided blimp. I moved the blimp around and captured images with camera. Then I started testing my system with the provided images, trying to localize the blimp on a map taken of google. Note that the two different map sources means using images from different times.
So I started testing my localization scheme and that's when start it started to change. First the particle filter was too slow and I replaced it with an EKF (from opencv). But then the EKF would easily accumulate errors with wrong matches. Now I have switched to Multi hypothesis EKF(this I had to write myself) and its working ok, but the whole thing is underestimating the height of the vehicle.
I am guessing that the source of the underestimation is the input of the EKF update. What I am doing is literally I am feeding an average of several estimates for the input of the EKF update step. As for the height, I keep an estimate of the logarithm of the height. Since the height estimates are very inaccurate, as I am currently using the size of the features to obtain the estimates, averaging the logarithm gives a lot more pull to lower estimates than to higher estimates.
So what I will be doing now is changing the MHEKF to keep my height estimates in their original units...