From what I understand, if he doesn't rip you off from the start he gets you in the end. It is incredible that he gets any exchange customers, firstly because he is so well known as a con man and secondly with his location - surrounded by legit change shops and multinational banks. You could perhaps understand it if his shop was in the middle of nowhere and so were you.
It's amazing that people will use a clothes shop and expose themselves to the risk of getting ripped off, in pursuit of that extra 0.02 to the pound.