I have been practicing for a little while picking a lock on my desk drawer, which has been very successful

However, while I can pick it very quickly and very reliably one way, it is very difficult to go the other way. To make this clearer, when the pins are at the top of the lock, I can pick it and rotate the cylinder until the pins are facing down, the pins then reset and the desk is locked. However, when I try and pick the lock while the pins are at the bottom, no matter what I do the thing just won't open.
I think the reason for this is because when the pins are on the bottom, the spring tension that keeps the pins up is working against gravity, so perhaps pins are being overset because the friction with the barrel is too much for the spring tension on the pins to overcome. However it doesn't seem to matter how gently I apply tension, it still won't work.
If this is the case, does anyone have any tips for working with a lock where the cylinder is upside-down? Or, if I'm wrong, can anyone tell me WHY the locks are harder to pick when the cylinder is upside-down?