Think boat only goes from Dalyan on a monday as this ties in with Koyceyiz market. Can just take boatfrom the town, go to just in front of the mosque in the middle of town and ask, or go on organised tour with one of the local agencies, which would include visit to market, Sultaniye springs and lunch, you could look at Brilliant tour website.
Or you could rent a boat privately from Dalyan, cost I think, around 80 - 100 lira return, plus entrance to the hot springs, around 5 lira last year, so perhaps more now.As far as I know, there is no boat service from Koycegiz itself.
other alternative is to drive towards marmaris and take the Ekincik turn off, Sultaniye springs should also be on sign, but in small lettering so not so easy to see as Ekincik. It is a long winding road down to Ekincik bay, and you would have to rent a boat there to take you to Sultaniye, probably similar cost to going from Dalyan, so with the extra petrol to get to Ekincik, more expensive.
Sultaniye better then the 'touristic mud baths' on the opposite bank close to Dalyan.
