Catch a Kayakoy dolmus from the side of the big mosque, the Yeni Hamidiye Cami, in Fethiye. They say clearly on the front screen that they're going to Kaya and some will also be going further on to Gemiler Beach.
There are plenty of very good restaurants in the centre of village, so you'll have plenty of choice from the recommendations that you'll get from other members. Some have pools that you can use for unlimited time providing of course that you buy some food and / or drink while there.
Avoid Tuesday afternoons as the tours that have done the Tuesday Market, generally come up there for lunch. Sundays can also get very busy because of the number of Turkish visitors.
If you're fit and decide to walk across the hills to Oludeniz, don't set off after 3pm as the sun drops very quickly and, in the event that you miss one of the waymarkings, you don't want to be finding your way in the dark. Take at least a litre of water per person. Wear sensible shoes or trainers. Make sure you have a mobile phone and let someone know what your intentions are.