How we beat Google in sentiment analysis
Review sites often give a generalized rating, such as 3 out of 5. It means something was good, something was not. If there are hundreds of such reviews on at least a few sites, it is not easy to make an objective picture.
Or an emotional client can give 1 out of 5. But it's not at all clear what the reason is. Definitely they did not like something and it must be corrected, but what exactly? Is it an accident or a systematic problem?
Similarly, customers can highlight something as particularly nice, which can be shown as a competitive advantage. But what is it about good reviews that all customers highlight?
This article focuses on the problems of accurately identifying sentiment analysis in complex reviews and their solutions
Determining sentiment in the text using neural network models has become almost a school task for beginner data scientists. However, despite the high quality of models and the ease of working with them, it is not so easy to characterize real data from reviews of restaurants, hotels or museums. Even Google’s algorithms can’t do it properly.
What are the reasons?
In some reviews, the text contains a lot of neutral information, from which it is difficult to automatically highlight something specific:
Sometimes, even if the review is unequivocally positive, it is difficult to understand what exactly attracted users if there are many different topics raised in one review.
Searching only by keywords does not always help, as clients call the same things differently, for example, “intimate place” and “relaxed atmosphere” are about the same thing.
Analyzing each sentence individually does not often help if the punctuation is not observed or a sentence contains several sentiments.
Examples, please.
Let’s take a closer look at this review.
We understand very clearly that a customer is happy about coffee and disappointed by the service, despite the smiling emoji in the end. But for artificial neural networks everything is not so clear.
Here is the result of Google’s sentiment analysis from Natural Language API (Google Service).
Sentiment 0 (no answer) about “service”, and very low positive sentiment about “coffee”. Not enough, right?
For example our sentiment analysis for this review is
"The coffee is delicious, but the service is a tragedy :)" — 0.994435399093432 [coffee]
"The coffee is delicious, but the service is a tragedy :)" — 0.012208339626454045 [service]
Where 1 - is excellent, 0 - is terrible
Much better, agree?
Our developed method is based on the autoencoder architecture of the neural networks, it allows ‘extracting’ all the mentioned objects and the sentiment related specifically to them from the review.
For example:
Our algorithm ‘understood’ that a customer is angry about both barista and service. It has never been possible to automatically reach such a conclusion.
It shows our model copes with rather complicated cases when one word can have different meanings depending on the context.
In these reviews
Our model rated coffee low (0.03).
But in these cases
“cold coffee” has got high rating (0.99).
In this example the model determined the problem was “ cold coffee” not just coffee.
Mark for cold coffee - is 0.02
As you can see the model copes with complex cases.
Here is a small demo that shows the rating of the cafe in the context of different criteria. We get data about all the chain cafe in Vilnius.
You can choose 4 topics you are interested in and they will be shown on the map. When you choose an object, all reviews and the ability to filter by topic will be shown
Of course there are much more ways how to visualize data and show more parameters, for example here is the spider diagram for UK chain of italian restaurants.
It is possible to track new reviews in real time, identify problem areas in the chain and respond to them in a timely manner.
Do you want to see your customers feedback? Leave a call back request and our AI model will analyze them.
Earlier we wrote about automation of the contact center, where we managed to reduce costs and staff burnout and improve the quality of customer service.
We also developed a tool to measure creativity using NLP methods - check how creative you are.
