Dream to Learn is shutting down...

We are very sorry to say that Dream to Learn will be shutting down as of December 28th, 2019. If you have content that you wish to keep, you should make a copy of it before that date.


Call Center Sensemaking Systems 1

POSTED IN: Building Bridges from R to IBM Watson

Whiteboard Scribbles: Building Sensemaking Systems to Benefit Customers, Employees and Shareholders...


Thank you for calling! Please press #1 for English - or Press #2 if you’d prefer to stab a pencil into the back of your hand, rather than enter telephone tree hell.

When humans dial a Company’s customer support numbers – they are usually hoping for one of two things:

1)   HELP ME… (do something, change something)

2)   TELL ME… (information I want or need to know - so I can get back to important stuff)


Phone calls and SMS interactions with airlines, banks, technical support, utility companies, etc. usually fit into either or both of these categories. (Other media channels like web conferencing or augmented reality also can include a third group that includes “show me / teach me / guide me”)

Anyway - let's pull the pencil out of our hand and explore how "Sensemaking Systems" (crafted with, among other things, Cognitive Computing ingredients) can make everyone happy!




What is interesting here is that a Company’s team and Customer are usually in lock step on objectives: How can Company get Human’s problem taken care, as efficiently as possible? While, of course, operating with resource constraints that probably require some degree of automation by Company.

Everyone wants to get the Customer off the phone, and back to binge watching Netflix. Faster times to completion, or better use of automation also gives the Human Agents / Customer Service team members valuable time back - so they can actually take a normal lunch break, rather than scarfing a stale sandwich at their cubicle.



- Not getting yelled at by angry customers.

-  Not being overwhelmed with work (being able to actually take a normal coffee break or lunch)

-  Proper tools for the job (e.g. being equipped to answer questions and handle requests) and respect in their judgement.





- Talking to humans – especially smart ones who can “help me” and “tell me” fast

- Bots that don't suck. Sensible automation that delivers as promised - and solves problems efficiently (e.g. password resets)

- Respectful and Efficient use of time (Asking Customer for their name and address 3 times is neither)



- Money, of course! (profits, cash flow and dividends)

-  Automation that is leveraged to keep costs down (see above)

- Long-Term Shareholder Value (organization KPIs including loyalty/churn, tech innovation) and via risk reduction and wise strategy.


And this is where “Sensemaking Systems” can help. As the name implies – Sensemaking Systems make sense of the situation to better serve the customer.  A system that is listening for the right signals – can do a better job of getting the “help me” or “tell me” challenge solved.

 Sensemaking Signals - Developing a Deeper Understanding of the Customer

1)   TYPE - What kind of person are we talking to?  Aggressive or shy?  Young or old?   Risk taker or hand-sanitizer?   If we don't know for sure, given other signals, data and prior knowledge - can we make a best guess about what kind of person they are?

2)   MOOD - How are they right now? Did they just swear at the automated agent and are currently hot under the collar - or did they engage in playful banter?

3)   HISTORY – Do we have a history? Prior conversations, dialog, or knowledge about consumption patterns? Or is this the fourth call from their phone number today? If so maybe they need a little TLC.

Etc.. this is information that helps systems be smarter, and solve problems better

Now for Tools. Before we get too far along, full disclosure that I work for IBM Watson, so I’m going to look at it through the lens of IBM Watson Developer Cloud services - http://www.ibm.com/watson/developercloud/services-catalog.html plus analytics. There are, of course, a number of alternatives (other companies, open source, home built) for many tools below. Explore! And consider the best fit for you.

 Tools (API) to Pop the Signals out of Interactions

Conversation Service -> As the blurb says "Add a natural language interface to your application to automate interactions with your end users. Common applications include virtual agents and chat bots that can integrate and communicate on any channel or device."  - https://www.ibm.com/watson/developercloud/conversation.html

Speech to Text (STT) -> First line workhorse - audio transformed to transcripts which are then processed to extract signal and meaning out of the Customer utterances. Remember to keep the logs (if permitted) as you may develop future hypotheses or new signal extraction methods in the future, where historic data can be very informative.

Text to Speech (TTS) -> Talking Bots with personality. Don't like Michael and his American accent? Maybe Customer can ask to speak with Kate – because everything sounds smarter when said with an English accent. The bots can also be EXPRESSIVE – if Kate is “So sorry about the hard drive being corrupted” – the TTS services today can make it sound like she’s actually sorry (or excited, or uncertain) - https://dreamtolearn.com/ryan/r_journey_to_watson/34

Alchemy API - Cracking the Carbon of Unstructured data - Pulling the people, places, things, ideas, concepts, tone, taxonomy, knowledge graph locations from utterances is a great way to release the signal energy from utterances. Nice demo: https://alchemy-language-demo.mybluemix.net/

Tone Analyzer -> Demo: https://tone-analyzer-demo.mybluemix.net/  and for some color here are a couple of Tone Signals (anger and confidence) popping out of the utterance :

"Dam#it! I got a speeding ticket. Never again! I swear"

Tone/emotion signals:

>> anger  0.62 // disgust  0.14 // fear  0.03 // joy  0.10 // sadness  0.18

>> analytical  0.00 // confident  0.62 // tentative  0.00

>> openness 0.00 / conscientious 0.07 / extravert 0.04 / agreeable 0.25 / em_range  0.14

Here's another example from the demo...

Personality Insights -> Generates 52 Signals to help understand what makes someone tick – “Big 5” from the Five Factor Model / OCEAN, (informs Engagement Methods) twelve categories of NEEDS (Informs actions for product or service fulfillment) ; and VALUES (informs person’s selection or evaluation of actions) - here is the demo and summary: http://www.ibm.com/watson/developercloud/personality-insights.html

Natural Language Classifier –> I talk more about this in other blogs, but it’s a ‘roll your own’ signal extraction machine. Very cool - and very powerful when used in ensemble deployments. E.g. https://dreamtolearn.com/ryan/r_journey_to_watson and Tony's (NLC) Handbook: https://ibm.box.com/s/rdlog2sue79178816s0rabkbi7ifu5vg 


Ok, so how do we apply these tools to start putting a smile on the face of our three stakeholder groups – Customers: Agents and Shareholders? 

Let’s look at a Whiteboard example brainstorming diagram (not complete) – and unpack what we can do with a little bit of signal and a little bit of prior knowledge. We'll make our way around the diagram clockwise:



AGENT ASSIST & TOOLS – Making the Cubicle Suck a little bit less:

Best Fit Agent – Let’s face it – some personality types just click.  IF you know you have a Type A Red-Bull young feller on your hands – there are probably agents that are a better fit to chat. Predictive and learning systems can help gate Customers to ‘best fit’ Agents that are most likely to have good (measurable) outcomes.

Cognitive Wingman – Point and Shoot Tasks - When I chat with folks about augmented reality – I often refer to the Jarvis-like cognitive pal as a “Cognitive Wingman”. Imagine a tiny little helper on my shoulder, or yours. Now imagine if an agent has 6 steps to complete, but can ask their (personalized) Cognitive Wingman to take you through steps 3 and 4. With one ear open, but doing the other stuff concurrently. Better together. Faster resolution. Less repetition for repetitive tasks.

Point-and-Shoot Text to Speech (TTS) – this is a simpler version of above – but is a drag and drop (or verbal commanded), tool that Agent can customize – and also point and shoot to read-repeat phrases (e.g. legal disclaimers). Can make it a bit fun, while getting the job done.

Fleeting - is when the sensemaking system understands cases where GROUPS of people are in the system concurrently - and share the same needs - the same problem. We can "bundle" or fleet our callers. Provide them an option to wait 20 minutes on hold, or opt in to join a group of 5 other customers (muted or not) with 1 agent – and the gang works through the problem together. Another moment in the sharing economy ;) - you’ve killed 5 birds with one stone.


Monitoring Real Time of each channel can help teams better support each other. 50 Cubes and 50 channels can be monitored for Anger, Profanity, or other “signs of struggle” – and when a threshold exceeded – a Team Leader can receive an SMS, or see cube alert flash –so they can plug in to call and help their buddy out, and get Customer on their way https://www.youtube.com/watch?v=MaRkMesxd48 -

Predict things that Matter using signals above – over time models can start to detect signs of trouble before they get too bad. Personality Type Red Bull, plus problem C at 4am? That’s a recipe for Rage.  Let’s gate this one to the safe hands of Sally.

Aggregate and Analyze - Manage what teams measure – reports of help pathways now have added color. Heat-map what channels are warm and why.  Ability to inform team leaders and operations managers of the consequences of choke points, so they can make sensible decisions about tradeoffs in resource constrained world.

EVOLVE AND ADAPT – Organizations Learning and Growing

Data Catalysts and CRM: Surfacing signal from interaction data, and getting to know the Customer better, provides knowledge that can be useful outside of the “help me” moment – for example – to CRM systems in understanding best products or services or perks for Customer at renewal time.

Deeper Understanding Model: As Company starts to surface signal and work with signals to solve near-term problems – more models can be developed on the foundation. New methods – like Topological Data Analysis can leverage new flavors of data – to get insights into Churn, and also optimal call center models and methods. 

Learning Organizations with Healthy Culture: An enabled team, getting results will start to look to the future, rather than rigidly applying mental models of 25 years ago. They will feel empowered and energized. And curious. This will drive ongoing innovation, which translates to Shareholder value.


Tune Solution to Each Customer - using knowledge of prior touches, relationship, and current mood – ensure the process is informed and adapts to it.  Clearly in a hurry? That matters. Third time calling today from your number - and a big spender? Let’s just get you right to a level 1 agent. Call you a VIP and take the edge off.  Here’s a fun blog on Jungian Archetypes - https://dreamtolearn.com/ryan/data_analytics_viz/99

Remember remember.... Prior Knowledge - System should remember prior calls (that day or a month ago) – this is not a first date! Computer memory is cheap. Use it.

Bots Are Cool - if they are not Crappy Bots - Younger customers especially are quite happy to use automated stuff – provided it does not suck. It’s 2017 after all, not 1989! IF it works, and it’s fast – no humans are required - just “Help Me” or “Tell me”.

Press #3 For "Never Again" – DTMF Tones are fun but hey – why not just SAY (or SMS) what is desired? Humans have put a man on the moon and built computers that can win at Chess, Go and Jeopardy. The solution should just “Get it”. Speak Act and Resolve. Let’s Go!  Not sucking is a great retention strategy – makes the Shareholders happy.

In closing

This concludes this summary of the Whiteboard Scribbles “Building Sensemaking Systems to Benefit Customers, Employees and Shareholders

Hopefully some snippets in here that stimulate a little thought on how you can put three big smiles on the faces of your Customers, your Agents, and your Shareholders.

The key take away is to experiment and explore ways your organization can use all available information (dark data) and new approaches (including Cognitive APIs) to create flexible and adaptive systems - that result in smiling customers, faster times to resolution, and by extension happier employees and shareholders.


Care to go deeper? Check out these Patents...

8,478,594 Systems and methods for automatically determining culture-based behavior in customer service interactions

8,346,556 Systems and methods for automatically determining culture-based behavior in customer service interactions


In Pursuit of Kick Ass Customer Service:


Interested in more content by this author?

About the Author

Ryan Anderson

Ryan Anderson

Hi! I like to play with data, analytics and hack around with robots and gadgets in my garage. Lately I've been learning about machine learning.

About this blog

This is an informal blog that explores tools, code and tricks that group members have developed to engage IBM Watson cognitive computing services - from the R Programming Language. Packages include RCURL to access Watson APIs - for services that include Natural Language Classifier and Speech to Text. THIS IS MY PERSONAL BLOG - it does not represent the views of my employer. Code is presented as 'use at your own risk' (it has lots of bugs)

Created: September 13, 2015


Up Next