Who are your best customer service representatives - based on customer satisfaction scores, staff friendliness scores and other criteria? If you don't have the data, ask around. When you find those top representatives, ask them what they do and how they do it. Then shadow them and watch them do it.
Your representatives are closest to the customers, and many understand how to drive customer satisfaction. They have the secret sauce; you just need to capture the recipe. Plus, should you embed their best practices in training, the fact that the idea originated at the frontline adds instant credibility.