![STEP 2: PREPARINGTENSORFLOW INPUT
• [X: [0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, ...N],Y: [0, 0, 1, 0, 0, 0, ...M]]
• [X: [0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, ...N],Y: [0, 0, 0, 1, 0, 0, ...M]]
• Array representing pattern with 0/1. N = vocabulary size. 1 when
word position in vocabulary is matching word from pattern
• Array representing intent with 0/1. M = number of intents. 1 when
intent position in list of intents/classes is matching current intent](https://image.slidesharecdn.com/oraclecodeone2018dev4293abaranovskis-181030083505/85/Machine-Learning-Applied-Contextual-Chatbots-Coding-Oracle-JET-and-TensorFlow-22-320.jpg)
Machine Learning Applied - Contextual Chatbots Coding, Oracle JET and TensorFlow
- 1. Machine Learning Applied - Contextual Chatbots Coding, Oracle JET andTensorFlow Andrejus Baranovskis, CEO andTechnical Expert, Red Samurai Consulting Oracle ACE Director and Oracle Groundbreaker Ambassador
- 2. Oracle ExpertsTeam ADF, JET, ORACLE FUSION, ORACLE CLOUD, MACHINE LEARNING Oracle PaaS Partner Community Award for Outstanding Java Cloud Service Contribution 2017
- 3. Session Goal HowTo BuildYour Own Machine Learning Chatbot
- 4. AGENDA • Technical Architecture • Solution WalkThrough • Machine Learning Introduction • Implementation Points
- 6. Machine Learning Chatbot Context Communication Chatbot UI Classification Chatbot messaging Oracle JET
- 7. Chatbot Custom application logic Generic listener Oracle JET
- 8. CHATBOT CONTEXT • Chatbot framework needs a structure in which conversational intents are defined (this can be JSON file) • Conversational intent contains: • tag (unique name) • patterns (sentence patterns for neural network text classifier) • responses (one will be used as a response)
- 12. LEARNING AND INFERENCE Training data Feature vector Learning algorithm Model Test data Feature vector Model Prediction
- 13. KEY PARAMETERS • Cost Function - score for each candidate parameter, shows sum of errors in predicting.The higher the cost, the worse the model parameters will be • Epoch - each step of looping through all data to update the model parameters • Learning rate - the size of the learning step
- 15. REGRESSION EXAMPLE w - parameter to be found usingTensorFlow
- 17. CLASSIFICATION EXAMPLE Linear boundary line learned from the training data - equal probability for both groups
- 18. WHYTENSORFLOW? • TensorFlow has become the tool of choice to implement machine learning solutions • Developed by Google and supported by its flourishing community • Gives a way to easily implement industry-standard code
- 20. STEP 1: PREPARING DATA • Tokenise patterns into array of words • Lower case and stem all words. Example: Pharmacy => pharm. Attempt to represent related words • Create list of classes - intents • Create list of documents - combination between list of patterns and list of intents
- 21. STEP 1: PREPARING DATA
- 22. STEP 2: PREPARINGTENSORFLOW INPUT • [X: [0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, ...N],Y: [0, 0, 1, 0, 0, 0, ...M]] • [X: [0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, ...N],Y: [0, 0, 0, 1, 0, 0, ...M]] • Array representing pattern with 0/1. N = vocabulary size. 1 when word position in vocabulary is matching word from pattern • Array representing intent with 0/1. M = number of intents. 1 when intent position in list of intents/classes is matching current intent
- 23. STEP 2: PREPARINGTENSORFLOW INPUT
- 24. STEP 3:TRAINING NEURAL NETWORK • Use tflearn - deep learning library featuring a higher-level API forTensorFlow • Define X input shape - equal to word vocabulary size • Define two layers with 8 hidden neurones - optimal for text classification task (based on experiments) • DefineY input shape - equal to number of intents • Apply regression to find the best equation parameters
- 25. STEP 3:TRAINING NEURAL NETWORK • Define Deep Neural Network model (DNN) • Run model.fit to construct classification model. Provide X/Y inputs, number of epochs and batch size • Per each epoch, multiple operations are executed to find optimal model parameters to classify future input converted to array of 0/1
- 26. STEP 3:TRAINING NEURAL NETWORK • Batch size: • Smaller batch size requires less memory. Especially important for datasets with large vocabulary • Typically networks train faster with smaller batches.Weights and network parameters are updated after each propagation • The smaller the batch the less accurate estimate of the gradient (function which describes the data) could be
- 27. STEP 3:TRAINING NEURAL NETWORK
- 28. STEP 4: INITIAL MODELTESTING • Tokenise input sentence - split it into array of words • Create bag of words (array with 0/1) for the input sentence - array equal to the size of vocabulary, with 1 for each word found in input sentence • Run model.predict with given bag of words array, this will return probability for each intent
- 29. STEP 4: INITIAL MODELTESTING
- 30. STEP 5: REUSETRAINED MODEL • For better reusability, it is recommended to create separate TensorFlow notebook, to handle classification requests • We can reuse previously created DNN model, by loading it with TensorFlow pickle
- 31. STEP 5: REUSETRAINED MODEL
- 32. STEP 6:TEXT CLASSIFICATION • Define REST interface, so that function will be accessible outside TensorFlow • Convert incoming sentence into bag of words array and run model.predict • Consider results with probability higher than 0.25 to filter noise • Return multiple identified intents (if any), together with assigned probability
- 44. QUESTIONS
- 45. CONTACTS • Andrejus Baranovskis (https://andrejusb.blogspot.com) • Email: [email protected] • Twitter: @andrejusb • LinkedIn: https://www.linkedin.com/in/andrejus-baranovskis-251b392 • Web: http://redsamuraiconsulting.com
- 46. REFERENCES • Source Code - https://github.com/abaranovskis-redsamurai/shenzhen • Contextual Chatbot inTensorFlow - https://bit.ly/2pFbTw4 • TensorFlow Book - http://tensorflowbook.com/