As you come into Lyme Regis from the A35 you will need to head for the bottom of the town. 
You will come to some traffic lights with Rainbow on your left and the lovely Guildhall in front of you.
You go through the traffic lights but before you reach the oncoming set you need to take a right just before the next set of lights up a narrow passageway.