Writing Smarter Bots: adding AI-level natural language processing of text entities

Yesterday, I was doing some work writing a new Teams bot (details out soon). I needed to parse a fuzzy, natural language date time string, for example “in 3 days time, next Wednesday, tomorrow at 7 in the evening.”

This is the sort of thing which LUIS does really well. However, creating a stub LUIS instance purely to use the datetime-parsing features seemed like overkill and the wrong solution: it would introduce latency as the request would have to go to LUIS and back. Also, it would attract an extra cost at scale. I’m not sure how well-used my new bot will be; but it’s worth planning on being frugal in the cloud wherever possible.

I started looking around to see if anyone had packaged a .NET library to encompass this and found Microsoft.Recognisers.Text. As the name suggests, this is a GitHub project that’s written and maintained by Microsoft. This is actually the same code that goes into LUIS, open-sourced and made available for everyone to use. I love this new Microsoft approach of working in the open and making building blocks available to everyone. This means I can embed the same functionality, but processed locally, at a fraction of the speed and cost.

There are actually several sub-projects here, focused on different entities: numbers, numbers with units, date/times, sequences (like phone numbers or IP addresses), choices, and TIMEX expressions.  Each one has its own NuGet package (or NPM package if you’re using Node) and the GitHub page describes nicely how to use them.

For me, the process of parsing a possible date-time value came down to passing in the string, and then looking at the returned List<ModelResult> to see what sort of object (date, time, datetime, range) was returned and what the value was, and then processing from there.

var recognizer = new DateTimeRecognizer(Culture.English);
var model = recognizer.GetDateTimeModel();
var result = model.Parse("Tomorrow at 3pm");

or

var result = NumberRecognizer.RecognizeDateTime("next tuesday 9am", Culture.English);

Thanks to these packages, I’m now doing the same type of AI magic that LUIS does, using the same tooling that powers LUIS, offline and in my own application. Which ultimately means I don’t have to try and write my own date-time natural language parser, which is probably in everyone’s interest. Abstracting on the shoulders of giants!