The prime directive is a great example of how even a good rule taken to the extreme can end up causing more harm than good.

But beyond that, it's just an easy aid for the writers to add a point of conflict for their stories. The prime directive as a value within the federation seems secondary to me.