An iSixSigma Discussion Forum exchange involved the manager of a call center who was interested in eliminating her organization’s “online chat” channel of communication and have all contact flow through the phones instead.
However, she wanted to see if she could make an apples-to-apples comparison between the online chat and phone channels, given that they handle significantly different volumes of requests. Readers weigh in on how each metric is measured and suggest a deeper root-cause analysis of the call vs. chat before determining that one method is better than another.
MARCH 24
Pam: I am wondering if we want to move volume from one channel (chat) to another (phone) in a call center. We are comparing metrics in both channels to make a decision… We are moving volume because phone metrics are better than chat, [but] should we consider volume? Phone metrics are better, but the volume is 100,000 compared to chat, where metrics are almost the same but volume is 500,000. Is this comparison logical considering the fact that the volumes are so different? Is it a fair comparison??
nmudd: I’m not sure if the volume difference is the big factor in determining if the comparison is logical. Using the appropriate statistical test should account for the differing sample size. I would be more concerned with the stability of the volumes in recent history (i.e., “has one or both been increasing/decreasing recently?”). If both populations were stable and met the other assumptions of the statistical test, then I would think you could test volume before vs. after for a statistically significant difference, which would indicate the effectiveness of your solution… On a side note, I am curious why you would want to shift volume from chat to phone call as it seems live chat might be a more efficient, though sometimes more impersonal, way to handle support requests.
Pam: I did a [two-sample] t-test and it came out insignificant. The volumes have not been stable because slowly we are decreasing chat volume and increasing phone volume on purpose. So, I am not sure if its right to compare 20 percent of phones to 100 percent of chat and say that 20 percent of phone [contacts] performs better in terms of metrics, and therefore we should move all 100 percent of chat to phones.
MARCH 25
nmudd: I see. I think it might be difficult, but not impossible, to compare the metrics one for one between the two. You would have to take a careful look at each metric before making the statement that one performs better than the other. If you are comparing a metric such as the number of issues resolved, then you must normalize according to volume so it is a fair comparison due to the drastically different volumes.
There may be other reasons other than volume why a direct apple-to-apple comparison doesn’t work. For example, it might take two minutes to resolve a certain type of issue on the phone, but to resolve that same thing via chat it might take four minutes. [There’s] no difference in the level of service (or satisfaction) but it just takes longer to type instructions than it does to speak them … Another thing to consider might be the cost per incident.
I would also take a hard look at the metrics and make sure that they are customer focused. I am making no judgments about the relevancy of the metrics you are using but many times organizations get caught up in meeting/exceeding metrics that mean very little to their customers.
MARCH 27
BBinNC: I’m a tad confused. You declare that you are making the switch based on “better phone metrics”, but then say that statistical tests on the metrics considered relevant are inconclusive. What, then, does “better” mean? What evidence is there and how is it evaluated???I do believe it is valuable to consider volume (amongst satisfaction, [turnaround time], accuracy, close rate and any other relevant variables). It is hard to say without data how valuable it is in making this decision, but I fear ignoring it would be an error.
You state that there exists a vast discrepancy between the chat volume and phone volume, but have you investigated why the phone volume is 100,000 and the chat volume is 500,000? For example, does the discrepancy exist because chat is actually preferred by those using it (good) or because you were unable to handle the call volume and customers received faster service using chat (bad)?
Are there other metrics that, upon analysis, indicate that moving all volume to voice lines is a “good decision”?
Pam: Thanks. There is discrepancy between chat and phone volumes because it’s intentional. Chat had a higher handle time. So, cost was one factor. There are other customer metrics that we looked at and thought that it was a good decision since the customer metrics were a little higher. But I don’t think that we have looked at anything else other than cost and customer metrics.
jpmarth: BBinNC raises a valid point. You may want to dig deeper into the “why” to determine the root cause for the numbers between call and chat. Our company has several call centers, all handling calls from the same 800 number. We added a chat feature on our website, but found that customers prefer the call option as the subject matter requires an extensive dialog that can take longer through chat. We now know that in order to decrease calls (costly) and increase chat (cheaper) we need to adjust our business model of how we deliver information to the customer in order to make it possible for chat to be simple.
MBBinWI: You also need to take into account [that] the preferred communication method for different people is going to be different. Thus, you may be eliminating a preferred method, and thus eliminating a customer, by removing a channel. I think you would be better off looking at how to make all methods more effective and lower cost. Just my humble opinion.
Want to join in on the conversation? Click here.
