During the season the dolmuses run half hourly, on the hour and the half hour to Hisaronu, Ovacik and Fethiye. You have to change at Hisaronu to get to Oludeniz. The first one out of the village in the morning leaves at 8am and the last one out of Hisaronu back to Kaya is about 20.30. As Tinks says, just wave them down and they'll stop.
If you don't want to use the dolmuses, and there are 3 or 4 of you, it's quite economical to use taxis. It's 15 TL by taxi each way between Hisaronu and Kaya Mosque and about 20 - 25 TL after midnight.
There are plenty of very good restaurants in the village to suit all tastes.
The good news is that you won't have to worry about booking an early morning call if you're near the mosque !