The Pennine Way is generally walked in a northbound direction but no one is going to stop you going the other way either. Both are well served by public transport at any rate; Edale by train and Kirk Yetholm by bus. Travelling south does mean that you get the long section from Kirk Yetholm to Byrness out of the way early and there is something to be said for that. Equally well, the proximity to civilisation supports a start from Edale and you do get to build up your hill experience as you go north.
In Edale, the Pennine Way starts at the Old Nag's Head pub but we take our view of Edale from the trail as it leaves the village to set off towards Kinder Scout via Upper Booth, Jacob's Ladder and Edale Cross; this was an old packhorse route between Cheshire and Derbyshire that still acts as a walking route to Hayfield today. This replaces an older route that took everyone up Grindsbrook Clough before crossing Kinder Scout. That got a bad name and the its replacement does better by going around the edge of Kinder Scout instead.