Knowledge Mining with Azure Search Technical Deck
Download as PPTX, PDF0 likes168 views
This document describes Microsoft's cognitive search capabilities for enriching and annotating content through natural language processing and computer vision. It discusses how unstructured data like text, images and videos can be ingested from various sources and stores, enriched with built-in and custom cognitive skills, and indexed for exploration and search. The enriched and annotated documents can then be used to train and deploy custom machine learning models.
![…,
{
"@odata.type": "#Microsoft.Skills.Custom.WebApiSkill",
"uri" "https://myskill.azurewebsites.net/api/OrgId"
"httpHeaders": {"Api-Key": "mySecret" },
"context": "/document/organizations/*" ,
"inputs":
[
{ "name": “organizationName", "source": "/document/organizations/*" },
],
"outputs":
[
{ "name": "organizationId", "targetName": "organizationId" }
]
},](https://image.slidesharecdn.com/knowledgeminingwithazuresearch-technicaldeck-200415170536/85/Knowledge-Mining-with-Azure-Search-Technical-Deck-16-320.jpg)
![{
"values": [
{
"recordId": "7cad2",
"data":
{
"myOuput1": “animals"
}
},
{
"recordId": "7cad3",
"data":
{
"myOutput1": “colors"
}
},
…
]
}
{
"values": [
{
"recordId": "7cad2",
"data":
{
"myInput1": "fox",
"myInput2": "cat",
}
},
{
"recordId": "7cad3",
"data":
{
"myInput1": "blue",
"myInput2": "red",
}
},
…
]
}](https://image.slidesharecdn.com/knowledgeminingwithazuresearch-technicaldeck-200415170536/85/Knowledge-Mining-with-Azure-Search-Technical-Deck-17-320.jpg)
!["skills": [
{
"@odata.type": "#Microsoft.Skills.Text.LanguageDetectionSkill",
"inputs":
[
{ "name": "text", "source": "/document/content" }
],
"outputs":
[
{ "name": "languageCode", "targetName": "myLanguageCode" },
{ "name": "languageName", "targetName": "myLanguageName" }
]
},](https://image.slidesharecdn.com/knowledgeminingwithazuresearch-technicaldeck-200415170536/85/Knowledge-Mining-with-Azure-Search-Technical-Deck-21-320.jpg)
![…,
{
"@odata.type": "#Microsoft.Skills.Text.NamedEntityRecognitionSkill",
"categories": [ "Organization" ],
"defaultLanguageCode": "en",
"inputs":
[
{ "name": "text", "source": "/document/content" },
"name" "languageCode" "source" "/document/myLanguageCode"
],
"outputs":
[
{ "name": "organizations", "targetName": "organizations" }
]
},](https://image.slidesharecdn.com/knowledgeminingwithazuresearch-technicaldeck-200415170536/85/Knowledge-Mining-with-Azure-Search-Technical-Deck-22-320.jpg)
![…,
{
"@odata.type": "#Microsoft.Skills.Text.NamedEntityRecognitionSkill",
"categories": [ "Organization" ],
"defaultLanguageCode": "en",
"inputs":
[
{ "name": "text", "source": "/document/content" },
"name" "languageCode" "source" "/document/myLanguagecode"
],
"outputs":
[
{ "name": "organizations", "targetName": "organizations" }
]
},](https://image.slidesharecdn.com/knowledgeminingwithazuresearch-technicaldeck-200415170536/85/Knowledge-Mining-with-Azure-Search-Technical-Deck-24-320.jpg)
![…,
{
"@odata.type": "#Microsoft.Skills.Custom.WebApiSkill",
"uri" "https://myskill.azurewebsites.net/api/OrgId"
"context": "/document/organizations/*" ,
"httpHeaders": {"Api-Key": "mySecret" },
"inputs":
[
{ "name": “organizationName", "source": "/document/organizations/*" },
],
"outputs":
[
{ "name": "organizationId", "targetName": "organizationId" }
]
},](https://image.slidesharecdn.com/knowledgeminingwithazuresearch-technicaldeck-200415170536/85/Knowledge-Mining-with-Azure-Search-Technical-Deck-25-320.jpg)
![/document
/keyPhrases
/0
/1
/…
/n
/organizations
/0
/1
/…
/n
organizationId
organizationId
organizationId
organizationId
/images
/0
/1
/…
/n
tags
tags
tags
tags
New Indexer Property
{
…
"outputFieldMappings":
[
{
"sourceFieldName":
"/document/organizations/*/organizationId",
"targetFieldName":
"orgIds"
} ,
…
]
}](https://image.slidesharecdn.com/knowledgeminingwithazuresearch-technicaldeck-200415170536/85/Knowledge-Mining-with-Azure-Search-Technical-Deck-29-320.jpg)
Knowledge Mining with Azure Search Technical Deck
- 2. Understanding the latent value in all content
- 3. Text (1) Validate enrichment pipeline
- 4. Tags “throwing”, “ball”, “girl”, “grass”, “basketball” Caption “A girl throwing a ball”
- 5. Entities Persons “Anita Christiansen”, “Conrad Nuber”, Locations “Bothell”, “Woodinville” Organization “Litware Insurance Corp.”
- 7. Computer Vision Face Emotion Content Moderator Video Indexer Custom Vision Service Custom Decision Q-n-A Maker Language Understanding (LUIS) Text Analytics Bing Spell Check Translator Text Speaker Recognition Bing Speech Custom Speech Translator Speech Unified Speech Service Bing Autosuggest Bing Search Bing Entity Search Bing Statistics add-in Bing Visual Search Bing Custom Search
- 8. Management Free Keyword search Faceting Geospatial support Multi-Language Support Suggestions/auto-complete Customizable scoring models Proximity Search Synonyms etc.
- 9. INGEST Data in any format, any Azure store ENRICH EXPLORE Annotations Cognitive skills Search
- 10. Annotated Documents Customer Data Built-in Cognitive Skills OCR, Key Phrase Extraction, People Names, Company Names, Sentiment Analyzer, Computer Vision, etc. Search Index .pdf .doc .jpeg … Third Party Enrichers Custom classification models, Custom entity extraction, etc. Azure Machine Learning
- 11. Annotated Documents Built-in Cognitive Skills OCR, Key Phrase Extraction, People Names, Company Names, Sentiment Analyzer, Computer Vision, etc. Search Index Third Party Enrichers Custom classification models, Custom entity extraction, etc. Azure Machine Learning Customer Data .pdf .doc .jpeg …
- 13. Annotated Documents Search Index Built-in Cognitive Skills OCR, Key Phrase Extraction, People Names, Company Names, Sentiment Analyzer, Computer Vision, etc. Third Party Enrichers Custom classification models, Custom entity extraction, etc. Customer Data .pdf .doc .jpeg …
- 14. Key Phrase Extraction Sentiment Analysis Organization Entity Extraction Location Entity Extraction Persons Entity Extraction Language Detection Face Detection Tag Extraction Celebrity Recognition Landmark Detection Handwriting Recognition (Preview) Printed Text Recognition
- 16. …, { "@odata.type": "#Microsoft.Skills.Custom.WebApiSkill", "uri" "https://myskill.azurewebsites.net/api/OrgId" "httpHeaders": {"Api-Key": "mySecret" }, "context": "/document/organizations/*" , "inputs": [ { "name": “organizationName", "source": "/document/organizations/*" }, ], "outputs": [ { "name": "organizationId", "targetName": "organizationId" } ] },
- 17. { "values": [ { "recordId": "7cad2", "data": { "myOuput1": “animals" } }, { "recordId": "7cad3", "data": { "myOutput1": “colors" } }, … ] } { "values": [ { "recordId": "7cad2", "data": { "myInput1": "fox", "myInput2": "cat", } }, { "recordId": "7cad3", "data": { "myInput1": "blue", "myInput2": "red", } }, … ] }
- 21. "skills": [ { "@odata.type": "#Microsoft.Skills.Text.LanguageDetectionSkill", "inputs": [ { "name": "text", "source": "/document/content" } ], "outputs": [ { "name": "languageCode", "targetName": "myLanguageCode" }, { "name": "languageName", "targetName": "myLanguageName" } ] },
- 22. …, { "@odata.type": "#Microsoft.Skills.Text.NamedEntityRecognitionSkill", "categories": [ "Organization" ], "defaultLanguageCode": "en", "inputs": [ { "name": "text", "source": "/document/content" }, "name" "languageCode" "source" "/document/myLanguageCode" ], "outputs": [ { "name": "organizations", "targetName": "organizations" } ] },
- 24. …, { "@odata.type": "#Microsoft.Skills.Text.NamedEntityRecognitionSkill", "categories": [ "Organization" ], "defaultLanguageCode": "en", "inputs": [ { "name": "text", "source": "/document/content" }, "name" "languageCode" "source" "/document/myLanguagecode" ], "outputs": [ { "name": "organizations", "targetName": "organizations" } ] },
- 25. …, { "@odata.type": "#Microsoft.Skills.Custom.WebApiSkill", "uri" "https://myskill.azurewebsites.net/api/OrgId" "context": "/document/organizations/*" , "httpHeaders": {"Api-Key": "mySecret" }, "inputs": [ { "name": “organizationName", "source": "/document/organizations/*" }, ], "outputs": [ { "name": "organizationId", "targetName": "organizationId" } ] },
- 26. Search Index Built-in Cognitive Skills OCR, Key Phrase Extraction, People Names, Company Names, Sentiment Analyzer, Computer Vision, etc. Third Party Enrichers Custom classification models, Custom entity extraction, etc. Customer Data .pdf .doc .jpeg … Annotated Documents
- 27. /document /languageCode /keyPhrases /organizations /images /1 /2 /… /n /1 /2 /… /n organizationId organizationId organizationId organizationId /1 /2 /… /n tags tags tags tags document.pdf
- 28. Annotated Documents Built-in Cognitive Skills OCR, Key Phrase Extraction, People Names, Company Names, Sentiment Analyzer, Computer Vision, etc. Third Party Enrichers Custom classification models, Custom entity extraction, etc. Customer Data .pdf .doc .jpeg … Search Index
- 30. Annotated Documents Customer Data Built-in Cognitive Skills OCR, Key Phrase Extraction, People Names, Company Names, Sentiment Analyzer, Computer Vision, etc. Search Index .pdf .doc .jpeg … Third Party Enrichers Custom classification models, Custom entity extraction, etc. Azure Machine Learning
- 32. “Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi…” Class A Class B Class C
- 33. “Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi…” “Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi…” Entity type A Entity type B
- 38. Cognitive Search Documentation | Sign up for Azure Search Azure Machine Learning Package for Text Analytics Documentation | Create a Data Science Virtual Machine Cognitive Services Documentation | Sign up
Editor's Notes
- #3: Understanding latent value in all content
- #8: I verified accuracy of this slide with Giampaolo. Notes : Voice Font is part of Unified Speech Service. Custom Decision is not out for //build.
- #10: INGEST (Understanding documents in a variety of format) AUGMENT (Extract “information”, Create structure out of the unstructured.) EXPLORE (Search)
- #13: MongoDB?
- #18: TODO: Change properties (foo bar)
- #20: We use the term skillset to refer to all the skills that should be run as part of the enrichment process. In a basic example…
- #21: Sometimes you need to do something more complex. For instance, you may want to use the language you detected to improve the accuracy of the key-phrase extractor. Or you may want to get metadata of metadata.
- #24: Sometimes you need to do something more complex. For instance, you may want to use the language you detected to improve the accuracy of the key-phrase extractor. Or you may want to get metadata of metadata.
- #28: At each step of enrichment more structure is added to the document. Before-a-skill and after-a-skill diagram. (SHOW RESTFUL CALL)
- #30: At each step of enrichment more structure is added to the document. Before-a-skill and after-a-skill diagram. (SHOW RESTFUL CALL)
- #35: http://medicalentitydetector.azurewebsites.net/
- #39: TODO: Add link to Cognitive Services.