Actually we went there in October and stayed overnight and found it great. The tourists all go home once the sun goes down and it's very quiet. We stayed in a renovated greek house, which was amazing. We had an evening meal out and took a load of local wine back, to sit on our porch and drink, while enjoying the views. Well recommended!
One word of warning, there does not seem to be anything in the way of cheap overnight accomodation, not that we found anyway, so expect to pay top prices unless you book ahead.