Build a Doctor Appointment Bot With Azure Bot Service, Language Understanding, and Twilio SMS

Build a Doctor Appointment Bot With Azure Bot Service, Language Understanding, and Twilio SMS

Sunny Sun Lv4

A step-by-step tutorial to build a booking system

Telehealth services can improve access to healthcare for people living in remote areas. One common issue in remote areas is unreliable internet service. It’s a challenge for patients in remote areas to make appointments via web applications when internet service is poor or unavailable.

SMS is a reliable and low-cost alternative to reach people living in remote areas. In this article, you will learn how to build a doctor appointment booking system that allows users to book via SMS.

System Overview

The architecture of the SMS booking system is illustrated below:

A diagram to explain how the SMS booking system works. Users send text messages using their phones to a Twilio Phone Number, Twilio forwards the message to the webhook URL which is pointing to the Azure Bot, the Azure Bot forwards the message to the Language Understanding service for AI analysis. The Azure Bot responds to Twilio and Twilio forwards the response back to the users” phone.

Twilio is the communication channel between Microsoft Azure Bot Service and the end user. The core of the system is an Azure Bot built on the Microsoft Bot Framework . When the bot receives a message, it asks Language Understanding (LUIS) to analyze the message. LUIS responds with the intent of the message, which the bot uses to respond with a helpful answer.

Prerequisites

You’ll need these technologies to follow along:

Create the Language Understanding Service App

Azure configuration

You need to create a resource group to store the Azure resources you will create later.

Open a PowerShell and sign in to Azure using the Azure CLI:

az login

Run the following command to create a new resource group:

az group create –name rg-bot –location [AZURE_LOCATION]

Replace [AZURE_LOCATION] with the name of your preferred Azure location.

“Every resource in Azure is stored in a resource group, and every resource and resource group in Azure is stored in a specific Azure region or location. There are different reasons for picking different locations , but most commonly you want to pick the location that’s closest to you and your end users. You can find all regions by running az account list-locations -o table. Find your preferred location and use the value from the Name column to specify the location when creating Azure resources. Keep in mind that not all resources are available in all Azure locations.” — Microsoft Docs

To interact with specific types of Azure resources, the resource provider for those resource types needs to be enabled. The most common resource providers are enabled by default , but you’ll be using Language Understanding (LUIS) which is part of the Cognitive Services resource provider that is not enabled by default.

You can register the Cognitive Services resource provider in Azure Portal , or use the Azure CLI shown below.

First, check the status of the Cognitive Services resource provider using this command:

1
az provider show --namespace Microsoft.CognitiveServices -o table

If the RegistrationState is UnRegistered, then run the following command to register it:

1
az provider register --namespace Microsoft.CognitiveServices --wait

This command may take a minute to complete.

Create the LUIS App and Train the Model

Language Understanding (LUIS) is a cloud-based conversational AI service, part of Azure’s Cognitive Services . It can process natural language text to predict overall meaning and pull out relevant, detailed information.

To learn how to use LUIS, you need to know a few core concepts:

  • Intent: The purpose or goal expressed in a user’s utterance .
  • Entity: An item or an element that is relevant to the user’s intent.
  • Utterance: Unit of speech. Every sentence from a user is an utterance.

intent — what a user wants to do. book a flight. This element goes to utterances — different ways to express the intent, i.e. “I want to book a flight from New York to LA” or “Can I book a ticket from New York to LA” or “Book a flight from New York tomorrow. Entity — details of the intent, which looks like “New York, “LA,” or “tomorrow”

LUIS will act as the brain of the bot by helping the bot understand the incoming messages.

Create the LUIS app

Log in to the LUIS portal with your Azure account.

If you are a new user of LUIS, you will be prompted to Choose an authoring resource.

Click Create a new authoring resource if you haven’t created one before.

Another modal will appear to Create a new authoring resource.

  • Choose “rg-bot” in the Azure resource group dropdown
  • Enter “appointment-booking” in the Azure resource name field.
  • Pick your preferred region in the Location dropdown
  • Choose “F0” in Pricing Tier
  • Click Done

You will be redirected back to the previous Choose an authoring resource modal. Now, you can click on the Done button, then you will be redirected to the dashboard page.

On the dashboard page, click Create new app. The Create new app modal will appear.

Enter “Appointment-booking” in the Name field and click Done.

After the app is created, a tutorial modal will be shown. Click outside the modal to go to the newly created app details page.

You’ll need to collect some key information for later use.

  • Click on the MANAGE tab and then on the Settings link.
  • Take note of the App ID.
  • Click on the Azure Resources link on the left menu
  • Click on the Authoring Resource tab

Take note of the Primary key (LUIS API key) and the Endpoint URL.

Now, the appointment-booking LUIS app is created successfully. The next step is to create the heart of the app: the LUIS model.

Import the model

There are two ways to create the model. You can navigate to the Build tab of the app, and then create entities and intents manually. Or you can import a predefined model file into the app. You can download and import this LUIS model JSON file to save time.

  • After downloading the JSON model file, navigate to the Versions page via MANAGE > Versions.
  • Click on the Import button
  • Choose “ Import as JSON “ to open the import popup

  • Click on the Choose file button
  • Select your JSON file
  • Click on the Done button

The new model will be imported into the app.

Navigate to the Build > App Assets page and pick “ vAppointmentBookingBot.LUISModel.json “ from the versions dropdown. You can find it in the breadcrumb navigation at the top-left of the page.

Now, you will see the newly created intents and entities.

Train, Test, and Publish the LUIS Model

After the intents and entities are imported, the Train button in the top navigation bar is enabled.

Click the Train button to start the training process.

The Train button will be disabled and the Test button enabled after training is completed.

To test the new model, click on the Test button. A Test flyout panel will appear on the right. You can type an utterance into the test panel to try it out.

In the screenshot below, “i want to see doctor kathy” is given a score of 0.973 out of 1 for the BookAppointment intent. The Inspect window also identifies the Doctor entity as “kathy” correctly.

Since the test result looks pretty good, you can publish the LUIS app now.

  • Click on the Publish button on the navigation bar
  • Select the Production Slot
  • Click Done

After publishing is completed, a successful message notification is shown as shown above. That means the LUIS app is ready to be used!

Build the Bot

Create the bot using a Bot Framework template

In this tutorial, you’re going to use Bot Framework v4 SDK Templates to create the bot project. Open a shell and install the Bot Framework templates using the .NET CLI with these commands:

1
2
3
dotnet new -i Microsoft.Bot.Framework.CSharp.EchoBot  
dotnet new -i Microsoft.Bot.Framework.CSharp.CoreBot
dotnet new -i Microsoft.Bot.Framework.CSharp.EmptyBot

You’ll only be using the CoreBot template in this tutorial, but feel free to explore the EchoBot and EmptyBot template.

Now, you can use the newly installed template to generate a new bot project. Run the following command to create the bot project:

dotnet new corebot -n AppointmentBot

You can also create projects using the Bot Framework templates from the Visual Studio or VS Code.

After creating the project with the previous command, the project is created into the AppointmentBot/CoreBot folder and the root namespace is set to “CoreBot”.

This is inconsistent with how .NET templates usually work, but it can easily be rectified. The following PowerShell script will move the contents into the AppointmentBot folder, rename the project, and change all the namespaces to “AppointmentBot”.

Run the following script using PowerShell:

1
2
3
4
5
6
7
$CorrectProjectName = "AppointmentBot"  
Push-Location "./$CorrectProjectName"
Move-Item ./CoreBot/* ./
Remove-Item ./CoreBot
Move-Item ./CoreBot.csproj "./$CorrectProjectName.csproj"
Get-ChildItem * -Recurse -File | ForEach-Object { (Get-Content $_) -replace 'CoreBot', $CorrectProjectName | Set-Content $_ }
Pop-Location

Open the project using your preferred .NET editor. The project structure will look like the following:

The generated project comes with a flight booking bot sample. Remove those related models and dialog files as listed below:

[projectRoot]\CognitiveModels\FlightBooking.cs[projectRoot]\CognitiveModels\FlightBooking.json[projectRoot]\CognitiveModels\FlightBookingEx.cs[projectRoot]\Dialogs\BookingDialog.cs[projectRoot]\Dialogs\MainDialog.cs[projectRoot]\BookingDetails.cs[projectRoot]\FlightBookingRecognizer.cs

To save time, you can run the script below to remove the above files. Run the script from the project root folder by doing the following:

1
2
3
4
5
6
7
rm CognitiveModels/FlightBooking.cs  
rm CognitiveModels/FlightBooking.json
rm CognitiveModels/FlightBookingEx.cs
rm Dialogs/BookingDialog.cs
rm Dialogs/MainDialog.cs
rm BookingDetails.cs
rm FlightBookingRecognizer.cs

You will also need to remove lines 41 to 51 in the Startup.cs file. Those are the references to the deleted files.

After the cleanup, the Startup class will look like the following:

After cleanup

Appointment Booking Cognitive Model

Now that the project has been cleaned up, you can start implementing your own logic. Next, you’ll be creating the model, which LUIS will return to us with analysis data.

Next, you’ll create these files under the CognitiveModels folder:

  • DoctorBooking.cs: This file will contain the DoctorBooking class, which represents the data returned by LUIS.
  • DoctorBookingEx.cs: This file will extend DoctorBooking using a partial class to simplify accessing the entities of the LUIS results

Create the CognitiveModels/Doctorbooking.cs and add the following code:

I generated this class using the Bot Framework CLI and provided it for your convenience, but you can also generate this yourself.

You’ll need to install node.js and the BF CLI first, if you want to generate the _Doctorbooking.cs_ yourself.

You can install the BF CLI using the following command: _​​​​npm i -g @microsoft/botframework-cli_

Download the LUIS model JSON file to your project directory, and then run the following command from the project root directory:

_​​bf luis:generate:cs --in=AppointmentBookingBot.LUISModel.json --out=CognitiveModels/DoctorBooking.cs --className=AppointmentBot.CognitiveModels.DoctorBookin_

Create the CognitiveModels/DoctorBookingEx.cs file and add the following code:

Connect the bot to the LUIS app

To integrate the bot service with the LUIS app, you need to add the LUIS App ID, API key, and API Endpoint URL into the project configuration.

Replace the contents of appsettings.json with the JSON below:

1
2
3
4
5
6
7
8
9
{  
"MicrosoftAppType": "",
"MicrosoftAppId": "",
"MicrosoftAppPassword": "",
"MicrosoftAppTenantId": "",
"LuisAppId": "[YOUR_LUIS_APP_ID]",
"LuisApiKey": "<SET_USING_USER_SECRETS>",
"LuisApiEndpointUrl": "[LUIS_ENDPOINT_URL]"
}

Replace [YOUR_LUIS_APP_ID] with your LUIS App ID, and [LUIS_ENDPOINT_URL] with the LUIS Endpoint URL you took note of earlier.

Please note that you should not store sensitive information including API keys or tokens in your source code. That’s why you’ll configure the LuisApiKey using the Secret Manager tool .

Enable the Secret Manager tool for your project by running the following command at the project root directory:

1
dotnet user-secrets init

Run the following command to configure the LuisApiKey using the Secret Manager:

1
dotnet user-secrets set "LuisApiKey" "[YOUR_LUIS_API_KEY]"

Replace [YOUR_LUIS_API_KEY] with the LUIS App Primary Key you took note of earlier.

The bot application will retrieve the settings you just configured to establish the connection to your LUIS app in the AppointmentBookingRecognizer class below. Create a new file AppointmentBookingRecognizer.cs and add the following contents:

A recognizer is used to recognize user input and return intents and entities within a DialogContext. In the AppointmentBookingRecognizer class, a connection is established to the LUIS API endpoint. It also implements the RecognizeAsync method, which is called by dialogs to extract intents and entities from a user’s utterance.

Control the conversation flow using dialogs

You need to use Dialogs to manage conversations between the user and the bot.

“Dialogs are a central concept in the SDK, providing ways to manage a long-running conversation with the user. A Dialog can be composed with other dialogs.” — Microsoft Docs

Bot framework provides a rich set of dialogs to make it easier to create a conversation flow. In this example, you will create an AppointmentBookingDialog class to manage the main conversation. It is composed of a few dialogs, including a waterfall dialog and prompt dialogs.

The waterfall dialog defines the sequence of steps. The diagram below illustrates that the bot interacts with the user via a linear process.

A flowchart plotting how the bot conversation should flow. Choose doctor question asked? If no, go back to start. If yes, is doctor valid? If yes, ask appointment date time question. Did the user respond with a valid date time? If no, go back to appointment date time question. If yes, ask user to confirm. If confirmed, run final step.

Create a new file AppointmentDetails.cs into project root and add the following code:

The AppointmentDetails class is the model class for the dialog. Next, create the AppointmentBookingDialog.cs file into the Dialogs folder. AppointmentBookingDialog class will implement the process above.

Add the following code to the file:

Main Dialog

The MainDialog class manages the main process flow. Create the MainDialog.cs file in the Dialogs folder (you can download the code from here ).

This diagram gives you an overview of what the MainDialog class does.

It’s quite a lot of code, but the important part of the MainDialog class is below:

In a nutshell, when a message activity is received, the bot runs the MainDialog. The MainDialog prompts the user using the IntroStepAsync method, then calls the ActStepAsync method.

In the ActStepAsync method, the bot calls the LUIS app to get the luisResult object that includes the user’s intent and entities. The user’s intent and entities determine the next step, either performing validation or invoking other dialogs.

At the end, the bot calls the FinalStepAsync method to complete or cancel the process.

Twilio adapter and controller

By default, the Azure Bot service will connect to the web chat channel handled by the default AdapterWithErrorHandler adapter. The default adapter is injected into the default BotController class, and the controller exposes an endpoint /api/messages.

To connect the bot to Twilio, you will create a new TwilioAdapterWithErrorHandlerclass extended from the TwilioAdapter class. Run the following command to install the Microsoft.Bot.Builder.Adapters.Twilio NuGet package:

dotnet add package Microsoft.Bot.Builder.Adapters.Twilio –version 4.15.0

Make sure the Microsoft.Bot.Builder.Adapters.Twilio NuGet package is using the same versions as the other Microsoft.Bot.Builder.* packages specified in the csproj-file.

After the NuGet installation is completed, create the TwilioAdapterWithErrorHandler.cs file in the project root directory and add the following code:

To handle the HTTP webhook requests from Twilio, you’ll need to add a TwilloController. Create a new file TwilioController.cs in the Controllers folder, and add the following code:

The endpoint for TwilioController is /api/twilio. After adding the new endpoint, the bot can handle messages via both web channel and Twilio SMS channel.

Finally, you must register the dialogs and LUIS recognizer in the Startup class. Insert the following lines at the end of the ConfigureServices method in the Startup.cs file:

Test Locally

You’ll need to install the Bot Framework Emulator to test the bot locally. To Install the Bot Framework Emulator:

  • Navigate to GitHub releases page of the Bot Framework Emulator project
  • Click on the setup file for your OS to download it.
  • After the download is completed, click the file to start the installation. Follow the installation wizard and use the default options to complete the installation.

Next, start the bot project using the .NET CLI:

1
dotnet run

Now, start the Bot Emulator, click on the Open Bot button, and enter the bot’s URL in your local environment, which by default, is [_http://localhost:3978/api/messages_.](http://localhost:3978/api/messages.) Then click on the Connect button.

A chat window will be opened. You can type the message and start testing.

After you are happy with the test results, the bot can be deployed to Azure.

Deploy the Bot to Azure

To deploy the .NET bot to Azure, you need to use Azure CLI to create the following resources:

To create a new App Service plan, run the following command:

az appservice plan create -g rg-bot -n asp-bot –location [AZURE_LOCATION] –sku F1

Here’s what the parameters do:

  • -g or — resource-group: The resource group the resource should be placed in, in this case into the “rg-bot” resource group you created earlier.
  • -n or — name: The name of the App Service plan, which is asp-bot. “asp” is short for App Service plan.
  • -l or — location: The Azure location the resource should reside in. Replace [AZURE_LOCATION] with the location closest to you or your users, like you did when creating the resource group earlier.
  • — sku: The size (CPU/RAM/etc.) of the App Service plan by SKU, which in this case is F1 (free).

To make sure the .NET project will be deployed correctly, you’ll need to generate a deployment file. The deployment file can be generated with the command below:

1
az bot prepare-deploy — lang Csharp — code-dir “.” — proj-file-path “AppointmentBot.csproj”

Please note that --code-dir and --proj-file-path need to match to resolve the path to the project file.

Create the managed identity using the following command:

1
az identity create --resource-group "rg-bot" --name "identity-appointment-bot"

After the command is finished, a new identity-appointment-bot managed identity has been added in Azure, which will be used to create the new App Service and Bot Service in the next step.

The App Service and Bot Service can be generated using the existing App Service plan and the Azure Resource Manager (ARM) template which is part of the “CoreBot” .NET template.

You are using the DeploymentTemplates/template-with-preexisting-rg.json ARM template, but it requires a lot of parameters, which is why you should use a parameter file. Create a new ParameterFiles folder in the project root and create a new file RegisterAppParams.json with the following contents:

Some parameters have already been configured with the names of the previously created resources, but you still need to update a few with your own specific settings:

  • **appId**: The value of clientId in the response of the create identity command. You can also query the clientId like this: az identity show -g rg-bot -n identity-appointment-bot --query clientId
  • **tenantId**: The value of tenantId in the response of the create identity command. You can also query the tenantId like this: az account show --query tenantId
  • **appServicePlanLocation**: The Azure region you used when creating your App Service plan.
  • **botId**: The name for your Bot Service. This name has to be globally unique. Replace [UNIQUE_SUFFIX] with anything that would make the name unique, like “firstname-lastname1234”. If it doesn’t accept the name, change it up and try again.
  • **newWebAppName**: The name for your App Service. This name has to be globally unique because it will be used as a subdomain to azurewebsites.net. Replace [UNIQUE_SUFFIX] with anything that would make the name unique, like “firstname-lastname1234”. If it doesn’t accept the name, change it up and try again.

After the parameter file is updated, run the following command to generate the App Service and Bot Service:

1
2
3
4
az deployment group create `  
--resource-group "rg-bot" `
--template-file "./DeploymentTemplates/template-with-preexisting-rg.json" `
--parameters "@ParameterFiles/RegisterAppParams.json"

Azure will take a minute to deploy this infrastructure. After the App Service is generated, run the following command below to get the App Service hostname:

1
az webapp show -g rg-bot -n appointment-bot-[UNIQUE_SUFFIX] --query 'hostNames[0]'

Replace [UNIQUE_SUFFIX] with the suffix you used in the parameters file. Take note of this hostname as you will need it later.

Now that the App Service infrastructure has been provisioned, you can deploy your local .NET bot project. Run the following command which will create a ZIP file and deploy the ZIP file to App Service:

az webapp up -g rg-bot -n appointment-bot-[UNIQUE_SUFFIX]

It can take about 30 seconds for the deployment to complete. When it is done, you will see the success response shown below:

Setting up Twilio for SMS communication

You’ve tested the bot locally using web chat, but the goal of this tutorial is to communicate using SMS. You’ll need a Twilio Phone Number to receive and send SMS messages.

  • Go and buy a new phone number from Twilio . The phone number cost will be applied to your free promotional credit if you’re using a trial account.
    Make sure to take note of your new Twilio phone number. You’ll need it later on!
  • If you are using a trial Twilio account, you can only send text messages to Verified Caller IDs. Verify your phone number or the phone number you want to SMS if it isn’t on the list of Verified Caller IDs.
  • Lastly, you’ll need to find your Twilio Account SID and Auth Token. Navigate to your Twilio account page and take note of your Twilio Account SID and Auth Token located at the bottom left of the page.

When your Twilio Phone Number receives a text message, Twilio should forward it to your .NET bot hosted on Azure App Service.

To configure that, navigate to Phone Numbers> Manage > Active numbers, and click on your Twilio Phone Number to access the Configure page.

Under the Messaging section, set the dropdown under CONFIGURE WITH OTHER HANDLERS to “Webhook”, and in the adjacent text field, enter “ _https://”, then paste in the App Service hostname you took note of earlier, and then enter “/api/twilio_”.

The URL should look like https://your-hostname.azurewebsites.net/api/twilio.

Click the Save button on the bottom left. Take note of this webhook URL, you will need it again soon.

Lastly, you need to add some configuration to your App Service. Run the following command to configure the app settings:

az webapp config appsettings set -g rg-bot -n appointment-bot-[UNIQUE_SUFFIX] –settings LuisApiKey=[YOUR_LUIS_API_KEY]
TwilioNumber=[YOUR_TWILIO_PHONE_NUMBER] TwilioAccountSid=[YOUR_TWILIO_ACCOUNT_SID]
TwilioAuthToken=[YOUR_TWILIO_AUTH_TOKEN] `
TwilioValidationUrl=[YOUR_BOT_TWILIO_ENDPOINT]

Before running the command, replace the placeholders.

  • Replace [YOUR_LUIS_API_KEY] with the LUIS Primary Key you took note of earlier.
  • Replace [YOUR_TWILIO_PHONE_NUMBER] with the Twilio Phone Number you bought earlier. Enter the phone number using the E.164 which looks like +11234567890.
  • Replace [YOUR_TWILIO_ACCOUNT_SID] with your Twilio Account SID which you took note of earlier.
  • Replace [YOUR_BOT_TWILIO_ENDPOINT] with the webhook URL you took note of earlier. It should look like https://your-hostname.azurewebsites.net/api/twilio.

You can restart the app service to make sure the bot loads the app settings. Run the following command to restart the App Service:

1
az webapp restart -g rg-bot -n appointment-bot-[UNIQUE_SUFFIX]

End-to-End Test

Finally, you have built and assembled all the moving parts. Let’s test it!

Send a text message to your Twilio Phone Number, and your bot should receive a response. Here, you can see I sent the SMS messages below to my Twilio phone number, and it works!

Next Steps

Some important bot features aren’t covered in this article. You may want to explore this further when developing a production-grade bot.

  • Authentication: When a bot needs to access resources on behalf of a user, you must authenticate the user’s identity. The user authentication can be handled by Identity providers such as Azure AD with OAuth 2.0. Your bot will use the token generated by Azure to access those resources. The details on how to add Azure AD authentication to a bot can be found in Microsoft’s documentation .
  • Bot Framework Composer : It is an open-source visual designer and authoring tool for creating a bot with Azure Bot Service. You can use it to build dialogs in the UI and visualize the flow to business users. It also allows you to train LUIS models within the tool, thus saving the need to switch between different environments. If your project requires involving non-technical people to bot development, it is definitely a good tool to consider.

Conclusion

In this article, you walked through the steps to build an SMS booking system using Twilio, Azure Bot Framework, and LUIS. You could extend this by adding more channels, expanding the LUIS model to support real-life scenarios, and incorporating features like image recognition or multiple language support.

You can find the completed source code in this GitHub repo .

  • Title: Build a Doctor Appointment Bot With Azure Bot Service, Language Understanding, and Twilio SMS
  • Author: Sunny Sun
  • Created at : 2022-04-05 11:14:56
  • Updated at : 2024-08-16 19:46:17
  • Link: http://coffeethinkcode.com/2022/04/05/build-a-doctor-appointment-bot-with-azure-bot-and-twilio-sms/
  • License: This work is licensed under CC BY-NC-SA 4.0.