Unsupervised Aspect Based Sentiment Analysis at Scale

Editor's Notes

• #10: For this, the simplest solution would be to use Text analytics, which is Azure Cognitive service that enables you to enter your text and get automatically metadata and features. Tailwind Traders can use it to have insight into how people are interacting with their restaurants. Let’s have a look at how it works.
• #11: Why choose these APIs ? They work, and it’s easy. Easy:  The APIs are easy to implement because of the simple REST calls.  Being REST APIs, there’s a common way to implement and you can get started with all of them for free simply by going to one place, one website, www.microsoft.com/cognitive.  (You don’t have to hunt around to different places.)  Flexible:  We’ve got a breadth of intelligence and knowledge APIs so developers will be able to find what intelligence feature they need; and importantly, they all work on whatever language, framework, or platform developers choose. So, devs can integrated into their apps—iOS, Android, Windows—using their own tools they know and love (such as python or node.js, etc.). Tested: Tap into an ever-growing collection of powerful AI algorithms developed by experts. Developers can trust the quality and expertise build into each by experts in their field from Microsoft’s Research organization, Bing, and Azure machine learning and these capabilities are used across many Microsoft first party products such as Cortana, Bing and Skype. 
• #12: The pre-built AI services, whether it's in text or computer vision or language understanding usually will fill 80% of your business scenario needs. However, you're going to find that there are 20% of your business needs that might not be filled by a prebuilt model, and in those cases you're going to need something more custom. What tends to be really interesting, in my own personal experience, and as you're going to see this applies to Tailwind Traders as well, is that sometimes these 20% end up being the most valuable. In many cases, I might want to start with the cognitive services because it really makes it easy to get started, and that enables me to focus more deeply on the 20% of scenarios that I can't reach with the cognitive services. So let me give you a quick example in demo. We take the Text Analytics API, and we see here. I had a wonderful trip to Seattle last week. I even visited the Space Needle 2 times and we get really nice 84% positive sentiment. But here's something that's Interesting. If I changed one letter in “wonderful”, all of a sudden this typo causes the sentiment to drop to 50%. Now, this specific example is not a crazy example because you know it's something that I can change with maybe putting in a model for spelling detection. But there might be other things as I'm building and trying to build more robust and more custom versions of sentiment analysis that might be domain specific to my restaurant, and less applicable in the general model.
• #13: In that case, I might want to build my own so the way that I do this is I can use the Azure Machine Learning Service. What the Azure Machine Learning Services is -- it's a set of cloud services and APIs, with a Python SDK, that enable you to prepare your data, build models, train the models, and manage them, as well as the different experiments that you're using to build these models, and finaly deploy them at scale.
• #14: So the Azure ML Services are broken up to a few different artifacts. We have the models that we're going to build, the experiments of how we come up and develop them, the pipelines that allow us to take our experiments and build the pipeline for their deployment. This is especially useful for linking our data to model development and redeployment. We have compute management, which allows us to manage compute at scale, creating clusters of VMs as needed. Image management allows us to manage different environments for our models and deployments, allowing us to deploy them on edge devices, on the cloud, or even on prem. And finally the data storage is also integrated into one workspace.
• #15: So now that we have our baseline and I think that's great. Let’s discuss some of the more advanced Azura ML Services. They provide a foundation, on top of which we can train any models, including some open source models, and then put them into production and manage.
• #16: We're going to be taking an open source model developed by Intel AI Labs as part of their NLP Architect Library. What it does is it provides us with something called aspect based sentiment analysis. Before when we were looking at our restaurant reviews, we got just a very simple sentiment average for each restaurant. However, for Tailwind Traders that doesn't help to actually go in and fix any of the problems, because they still have one more step that they have to do - they have to send in an inspector, who can actually quantify and see why sentiment is so good at one branch versus another. Aspect based sentiment analysis allows us to address this challenge. What it does is is it takes our text an it extracts what's called a lexicon from it, of both aspect-related and opinion-related words. Each of the aspects represents things we're interested in, like the price of food, cleanness of the place, etc. And when we analyze actual review - it would give us a sentiment for each one of these individual aspects, which then in turn allows us to provide much more actionable responses from our models. These are the ABSA key messages: Producing knowledge regarding specific aspects hence enables to gain targeted business insight. Unsupervised - does not require costly manually tagged data for train Domain Adaptive – produces domain specific aspect and opinion terms   These are the additional NLP architect key messages: Develop low-resource and NLP models that require small amounts of training data. Increase adoption of NLP by developing deep-learning based tools that can be used by non-machine-learning experts. Produce optimized NLP models that are ready for deployment on Intel AI hardware.
• #17: So, in order to put this Intel model into production, we’re going to use the Azure Machine Learning Service. I think it's important to understand how the end to end process works, from taking a model from open source up to deploying it into production. The first thing that we have to do is to prepare a data set, then we have to choose our experiment environment, in this case we're going to be using Jupyter notebooks. We have to go and train this model, and iterate a few of different versions. Once we do have a model that is ready for deployment, we need to work on putting the model into production.
• #18: In this session, we're going to be focusing on the experimentation phase, though I strongly recommend that you check out the AIML 50 session, which talks about how you can take your models that you've developed and put them into production.
• #19: So for Tailwind Traders when they're thinking of how to take an open source state of the art model and put it into production, they've noticed that they have a bunch of critical challenges. Things like being able to manage the environmental dependency across the different packages and their different versions. Training a model with a globally distributed team in a globally distributed company is also a challenge. Managing large data sets and understanding how to optimize costs for storing their data and compute, as well as scaling their compute. Also in terms of compliance and making sure that they are ensured data protection, that only certain people have access to data and only the correct parts of the data. When they build the models, making sure that the models are interpretable in a way that they can understand how they were trained how they can be reproduced and the last real big challenge that they have is managing distributed training. Each of these things is somehow provided by the Azure ML Service, but it would take a very long time to go through each and every single one of them individually So what will do here is we're going to drill down into one of them and you will see the complexity of it, and then as we go and we solve and put this aspect based sentiment analysis model for Tailwind Traders in production.
• #20: 19
• #21: 20
• #22: I am providing links to relevant docs here for your convenience.
• #23: When I need to train an Azure ML model, it works as follows. There are just a few steps that we need to get started. The first thing that I need to do is I need to take the code that I have for model training and I need to send it to the Azure Machine Learning Service. The service will then create the configuration for the experiment that I’ve just sent, and it will create a Docker image. It will deploy it to the compute target that I've configured which will be linked to the data that I’ve uploaded in my dataset. It will then launch the training script that I've configured and then in real time it will send me back the stream logs and metrics so that I can see how the process goes, as if I were developing on my own computer!
• #24: To take this Aspect-Based Sentiment Analysis model into production, there would by 8 simple steps! 1. Configure Azure Machine Learning Workspace that we saw above, it would act like a central place for all our activities – from storing data to training models. 2. Define a compute target – on which our training code is going to run. 3. Upload our data set. 4. Create the experiment with the train code, encompassing all the dependencies that we have and the training script. 5. Training script can be taken from the existing Intel’s code. Sometimes we might need to adjust the train script from the original library for distributed training, in other cases changes to the code would be very minimal. 6. Run the estimator to perform the training 7. Once we're done with the results of training, we get the model which we then need to register. 8. Registered model can be deployed and tested, and becomes ready for production.
• #41: Let’s have a look at how such a model can be trained using a compute cluster. Typically, as a data scientist, I will do most of my development from a Jupyter Notebook. So I can select any Jupyter environment (local, Azure Notebooks or a Notebook inside Azure ML Workspace), and start my work from there using Azure ML Python SDK.
• #42: When we're dealing with model optimization, we also have a challenge of distributed hyperparameter tuning. What is distributed hyperparameter tuning?
• #43: What are hyperparameters typically? When you see a new state of the art model that's been open sourced, whether that’s BERT, or in this case our aspect based sentiment analysis model, typically you see these models trained on certain data. The performance of the model is Dependent on the distribution of the data that it’s been trained on, and there are set of parameters that are configured as part of the models that are optimized for the given data set, so that the researcher who is publishing the model can get the best possible performance. Now, when you take these models and you run them on your own data or train them for your own scenario, you're going to have a different distribution of data, so these hyperparameters might need to be adjusted, otherwise you will see a big gap in performance of your model when you apply it to Tailwind Traders data, comparing to what’s written in the leaderboards or in the papers.
• #44: This it can be a real challenge going through a manual hyperparameter tuning. Typically, what we do is we take a data set and we come up with an algorithm, and then we configure these parameters to build our first model, and then we change the parameters again and we build a second model and we tried to see whether it's better than the first one, and we do this a couple of times until we max out the potential of this algorithm. And then we might change our model and then we have to do this process all over again, and you can imagine it’s time-consuming tedious repetitive and really expensive process.
• #45: Well, the Azure ML Service helps you manage this by running distributed versions of your model and allowing you to choose which are the primary metrics. You want to optimize based on the hyperparameters that you provide, and it provides a bunch of different policies, such as bandit policies, medium stopping policy and truncation policies that enable early termination, so if it sees that a certain set of hyperparameters isn't providing the best possible results, it will terminate those run jobs and release and then run and prioritize the compute for models that are much more promising -- until you find the optimal set of hyperparameters
• #46: If you want to get started with that here's the documentation. We will be diving into it in a second so you can see how this service works now.
• #47: So let's tie this all together what we've seen here in literally less than an hour.
• #48: There are also a few takeways that I want you to get away with. First of all, AI is instrumental to companies success in the modern world, and most of the AI can be incorporated into production using Cognitve Services. However, for more complex tasks, Microsoft provides both simple “code-less” ways to train a baseline model, and very sophisticated Azure ML Service that can simplify life of a serious data scientist quite a bit. Whether you are experiences ML professional or a “traditional” developer – we have you covered!
• #49: Here is the summary of all docs pages that we have touched upon – they all present a great material for you to study after the session. I want especially to draw your attention to the last link, which is an overview article explaining how the thights we have learnt today can be useful in developing production models.
• #52: So thank you for your time and I hope you enjoyed.