It has to do with pressure altitude. The way your aircraft tracks its height above ground is by the atmospheric pressure. So your altimeter is actually tracking the height of the pressure surface you are flying along.
When you are flying from an area of High surface pressure to an area of Low surface pressure, the height above the ground of that constant pressure surface (let''s for the sake of argument say 700 mb) goes down. Your altimeter will show the same height, but you will be closer to the ground (therefore, look out below or you might crash!)
When you are flying from an area of low surface pressure to an area of high surface pressure, it works the other way. Your altimeter will read a constant height, but you will actually be climbing relative to the ground. (i.e. "sky high")