Natural language processing technology

Natural language processing technology has traditionally been used in applications such as defense and law enforcement.

One of the most successful use cases has been the ability to identify ‘named entities’ in textual documents.

Named entities are names of people, places, organizations and events, and are identified based on linguistic context rather than relying on predefined lists.

For example in the sentence ‘Heather Apples is my best friend,’ one can conclude that Heather Apples must be a person, even if this name does not appear in a list of names.

This in turn allows connections to be made across documents, across cases, and ultimately a social network to be automatically generated.

Much of the enterprise information ecosystem relies on structured data that typically resides in relational databases.

The holy grail of NLP has been to convert ‘unstructured data’ (text and multimedia) into structured data and we are inching closer towards that goal with advances in NLP and multimedia indexing technology.

NLP and sentiment analysis

For marketers, sentiment analysis of social media is the most widely used NLP capability.

However recently, customers are questioning the utility of analytics showing positive versus negative sentiment of a brand. “This is interesting, but how do I use it?”

Sentiment and volume of mentions can be converted to actionable analytics if the data is:

  1. Calibrated based on expected reach.
  2. Accompanied by demographical information.

This leads to solutions such as social segmentation and more targeted marketing campaigns.

For marketers the ability to identify ‘propensity signals’ from social media posts or customer satisfaction surveys is extremely useful and leads to social prospecting solutions.

Someone who is tweeting that they “saw their friend’s new BMW 7 series and am seriously thinking about one too” is expressing more than mere positive sentiment, they are expressing a propensity to purchase.

Social prospecting solutions require NLP capabilities that can sift out passing mentions of a brand, and focus on those where this is an intent to purchase.

Similarly, one can look for signals indicating intent to abandon, book a holiday etc.  Some of these signals are specific to a brand, while others are more general interest in acquiring a type of product (e.g. camping equipment) or service.

State of the art NLP systems can mine social media for such expressions of interest and return social handles of people matching the customer’s criteria. True NLP technology goes beyond keyword matching and takes into account the context of a mention. 

NLP and machine learning can also be used to analyze customer content. As an example, news content can be automatically tagged with topic labels, key people, places and events mentioned as well as readability scores.

A longer article using more sophisticated vocabulary would return a higher readability score. This in turn, can be used by recommendation engines to personalize reading suggestions based on what the user has previously looked at.

NLP technology and ecommerce

In the ecommerce world, NLP technology can be used to automatically annotate product catalogues with html tags reflecting additional product attributes such as “lightweight, pockets” etc. – attributes that may not necessarily be included in the metadata.

This in turn can assist with search engine optimization as well as better recommendation. NLP analysis of email campaign messages can reveal what “features” of an email message users were responsive to, leading to a greater degree of personalization.

Companies wishing to use NLP technology should consider multiple factors when making a decision, including scalability, flexibility of integration as well as native support for multiple languages.

NLP as a service is emerging as a new offering, and is useful for analyzing content that is of low update velocity. Ideally, a comprehensive NLP offering would include the ability to process both enterprise content and external, high velocity data such as social media.

The interaction of these two leads to compelling solutions, for example in leveraging socially trending topics (or brands) to promote customer content matching those topics or brands.

The Wall Street Journal recently renamed its Marketplace section as Business and Technology along with the following explanation.

Every Business is a Technology Business. Whether it’s taxi cabs or taco delivery, today’s enterprises are urgently figuring out ways to manage the growth of information technology and to turn its disruptive potential to their advantage.

NLP and Machine Learning are technologies that have the potential to disrupt the landscape in business intelligence, marketing, ecommerce and enterprise information systems overall.

In order to fully realize the advantages of using NLP, there needs to be interaction between these systems and other components of the enterprise. For example, a social prospecting solution is useful if it is connected to a CRM database, allowing the company to augment information on existing customers, and prospective customers.

Similarly, NLP should be a key component in digital marketing platforms offering capabilities such as personalized email, recommendation and mobile apps.

New advances in NLP such as the incorporation of deep learning, are increasing both the accuracy and breadth of capabilities of NLP.

We are moving to an era where critical business decisions, and marketing will rely increasingly on unstructured data.  By leveraging this till now largely unexploited treasure trove of data, organizations will be better poised to react in real-time and more importantly, be proactive about their strategy.