The major componenets of our dialogue system

NLUU

Responsible for all natural-language text input, analysis, and output.

Dialogue Manager

Handles flow of conversation, creating appropriate responses given an input, and requests data in a logical way.

Task Manager

An intelligent blackbox that connects to the database and can intelligently ask for data and rate courses based on relevancy to the conversation.

Tools used

NLTK

We use nltk’s part of speech tagger to extract parts of sentences that we believe are likely keywords and the VADER sentiment analyzer to identify the sentiment of a client’s utterance, stripping away semantics. Additionally, we use nltk's word stemmer and the operating system's list of words to create a dictionary whose keys are word stems and values are lists of words whose stem is the key. This allows us to effectively expand keywords into a number of related words when querying the database to hopefully provide better results. nltk can be found here.

Microsoft LUIS

We use Microsoft's LUIS to extract intent from a client's response. We have trained the LUIS program on a number of utterances and flagged each as a particular intent. For instance, "Register me for math of cs" yields a ScheduleClass intent, meaning that the client wants to add a class to their schedule. We then use this knowledge of what the client wants to do to determine our next actions. Microsoft LUIS is part of the Azure program, and you can make an Azure account here, and LUIS can be found here. We have included our LUIS JSON file so that anyone wishing to extend or use our project can train on top of our existing intents and entities instead of starting over. For our project to run, you will likely need to change the luisurl in the config.json file to match the url of the LUIS project that you create.

System Architecture

When user provides raw text input, it first passes through the Natural Lanugage Understanding Unit (NLUU) that uses NLTK and LUIS to analyze intentionality and extract entities from the sentence. This information is passed on to the Dialogue Manager where the intention is taken into account to determine where the conversation is, where it will go, and what information is needed from the database.' From there, the Dialogue Manager makes a request to our intelligent database, known as the Task Manager. The Task Manager has functions that provide simple APIs for accessing course data on different criteria. Once that information is received, the response format is specified by the DM and the NLUU creates a raw string text response to be presented to the user.

Overview

Each component of our architecture is responsible for its own set of duties, from analyzing text to database queries.

Future Work

This project has a lot of potential for growth, both in terms of expanded functionality and overall polish/consistency. While the project managed to integrate most of our initially proposed features, there were a few features which ended up scrapped or only partially implemented due to time constraints. One feature which was considered early in the project’s conception was course time blocks and conflict management. While our final version displays meeting dates/times when available, the system does not account for scheduling conflicts when registering a user for courses. This was a result of the formatting of the meeting times in our data source, and with some additional development time, the conversation could check for time conflicts when registering.

Another feature which was partially integrated in the dialogue system is course professors. While our conversation displays the professor of a course when displaying, the system could also consider user preference of instructor, and even query on a user’s favorite professors. Additionally, the quality of results returned from queries in our dialogue system could be improved by the use of Term Frequency/Inverse Document Frequency, or TF/IDF, instead of the simpler occurrence matrix which is currently in use. Currently, our occurrence matrix doesn’t account for the lack of utility that some words have, e.g. “explore” is a word that is not particular to any single department. In the case of “explore”, the departments selected based on this keyword would skew towards departments with more classes. TF/IDF would allow us to account for this, because the Inverse Document Frequency component would factor in the presence of the word in all the departments.

While the conversation system is stable and relatively consistent in its current state, there are a number of ways the overall consistency and versatility. Namely, the dialogue system could benefit from an expanded decision tree which would result in a more natural flow of conversation, with additional branches and userQueries to reflect the wide variety of intents which could potentially appear in a conversation of this domain. Lastly, the LUIS AI component of the system could also benefit from additional intents, more training and improved entity extraction. Perhaps the system could even be built on an alternative to LUIS which is specialized to identify and classify intents/entities unique to the course registration domain of conversation.

Overview

Each component of our architecture could be improved in a number of ways. They are described here.