Salesforce Research creates Swiss Army Knife for natural language processing
Salesforce Research has created a natural language processing architecture that can handle multiple models and tasks. Typically, natural language processing (NLP) has a model for each function such as translation, sentiment analysis and question and answer.
The research, led by Salesforce Chief Scientist Richard Socher, revolves around a challenge dubbed Natural Language Decathalon (decaNLP). The challenge spans 10 tasks--question answering, machine translation, summarization, natural language inference, sentiment analysis, semantic role labeling, relation extraction, goal-oriented dialogue, database query generation, and pronoun resolution--and feeds into a system that jointly learns.
Think of decaNLP as a Swiss Army Knife for natural language processing. If NLP is customized repeatedly it won't scale. Salesforce was looking for a general purpose NLP approach where every task is transformed into a question answering format and trained jointly.
Socher said the approach melds deep learning and NLP and moves the discussion to one that revolves around an meta architecture. He added that an architecture approach can also be used to prevent model sprawl as NLP functions are layered together.
"This is a project that can have immediate useful applications because it's a model that is a single deploy and easier to maintain," said Socher. "We're bringing a bunch of tools together."
Salesforce is likely to use the decaNLP approach in its product roadmap for Einstein and its various clouds.
The decaNLP is combined with a multitask question answering network that jointly learns all tasks without any specific model. The network also allows for adaptation by completing new tasks through related descriptions.
Here's a diagram of the multitask question answering network.
And finally, Salesforce Research came up with code for processing datasets, training and evaluating models and ultimately coming up with a score called the decaScore.
NLP trained on the decaNLP system will in theory be better equipped to provide a framework for chat bots as well as any information in a customer service exchange.