We found that the important thing is insulation. Not necessarily the whole house but, if you've got open-plan, installing doors, curtains, anything really to keep the warmth in while still maintaining air-flow. We used air-con for short periods to get the temperature up together with oil-filled electric radiators to maintain the temperature. We also had an open fire, and for 4 or 5 winters a soba, because we had our own renewable firewood supply.
I don't know the cost of running oil-filled electric radiators, but if used in moderation, they're not hugely expensive.