Strange scope probe problem?

Maybe there is some oscillation going on that is suppressed by the capacitance of the scope probe. Try using a small capacitor instead of the scope and probe to see if it behaves the same way.