Only with friction. Without it you'll build exactly enough speed on your fall to the center of gravity that it carries you out to the same distance on the other side.

Think a swing on the playground. If you're high up and swing down you build up speed and that speed carries you up again, just that due to friction you have to input energy to keep going up to the same height.

Fun story: my dad built a swing for my niece recently and the friction was very low. When i tested it, i was swinging standing up and because almost no friction slowed me down I was unable to stop and got quite scared. I had to sit down while swinging and put my feet on the ground to break.