A Word on OAuth 2.0

This isn’t a book about security, but understanding basic authentication and authorization mechanisms is essential to be a developer. OAuth 2.0 is a standard authorization protocol. The three-legged OAuth 2.0 flow allows third-party applications to access services on behalf of another entity. In our case, we will be accessing a user’s Google Calendar data on behalf of that user. At the end of the three-legged OAuth flow, we end up with two tokens: an access token and a refresh token. The access token is included in requests to an API in the authorization HTTP header and provides data to the API declaring which user we are requesting data. Access tokens are typically short-lived to reduce the window during which a compromised access token can be utilized. When an access token expires, we can use the refresh token to receive a new access token.

To initiate the flow, we first redirect the user to a service that they can authenticate against, say, Google. Google presents an OAuth 2.0 login page where it authenticates the user and asks the user for their consent so that the bot can access the user’s data from Google on their behalf. When authentication and consent are successful, Google sends an authorization code back into the bot’s API, via what’s known as the redirect URI. Finally, our bot requests the access and refresh tokens by presenting the authorization code to Google’s token endpoint. Google’s OAuth libraries will help us implement the three-legged flow in our calendar bot.

Setting Up Google APIs

Before we jump into it, we should set ourselves up to be able to use the Google APIs. Luckily, Google makes this quite easy via the Google Cloud Platform API console. Google Cloud Platform is Google’s Azure or AWS; it is Google’s one-stop shop for provisioning and managing different cloud services. To get started, we navigate to https://console.cloud.google.com . If this is our first time visiting the site, we will be asked to accept the terms of service. After that, we will be placed in the dashboard (Figure 7-1).

Our next steps are as follows. We will create a new project. Within that project, we will ask for access to the Calendar API. We will also give our project the ability to log in on behalf of users using OAuth2. Once done, we will receive a client ID and secret. Those two pieces of data, plus our redirect URI, are sufficient for us to use the Google API libraries within our bot.

Click the Select a Project drop-down. You’ll be met with a pop-up that, if you have not used this console before, should be empty (Figure 7-2).

Click the + button to add a new project. Give the project a name. Once the project is created, we will be able to navigate to it through the Select a Project functionality (Figure 7-3). The project is also assigned an ID, prefixed by the project name.

When the open the project, we see the project dashboard, which initially looks intimidating (Figure 7-4). There are many things we can do here.

Let’s begin by getting access to the Google Calendar API. We first click APIs & Services. We can find this link in the first few items on the left navigation pane. The page already has quite a few things populated. These are the default Google Cloud Platform services. Since we’re not using them, we can disable each one. Once ready, we can click the Enable APIs and Services button. We search for Calendar and click Google Calendar API. Finally, we click the Enable button to add it to our project (Figure 7-5). We will receive a warning indicating that we may need credentials to use the API. No problem, we will do this next.

To set up authorization, we click the Credentials link on the left pane. We will be met with a prompt to create credentials. For our use case, in which we will be accessing the user’s calendar, we need an OAuth Client ID¹ (Figure 7-6).

We will first be asked to set up the consent screen (Figure 7-7). This is the screen that the user will be shown when authenticating against Google. Most of us have probably encountered these types of screens across different web applications. For example, whenever we log into an app via Facebook, we will be presented with a page telling us that the app needs permission to read all your contact information and photos and even deepest secrets. This is Google’s way of setting up a similar page. It asks for data such as the product name, logos, terms of service, privacy policy URLs, and so on. To test the functionality we minimally need a product name.

At this point, we will be taken back to the Create Client ID function. As the Application Type setting, we should select Web Application and give our client a name and a redirect URI (Figure 7-8). We utilize our ngrok proxy URI (see Chapter 5 for more on ngrok). For local testing, we are free to enter a localhost address. For example, you can enter http://localhost:3978.

Once we click the Create button , we will receive a pop-up with the client ID and client secret (Figure 7-9). Copy them because we will need the values in our bot. If we lose the client ID and secret, we can always access them by navigating to the Credentials page for the project and selecting the entry we created in the OAuth 2.0 Client IDs.

At this point we are ready to hook our bot up to the Google OAuth2 provider.

Integrating Authentication with Bot Builder

We will need to install the googleapis node package as well as crypto-js, a library that lets us encrypt data. When we send the user to the OAuth login page, we also include a state in the URL. A state is simply a payload that our application can use to identify a user and their conversation. When Google sends back an authorization code as part of the OAuth 2.0 three-legged flow, it will also send back the state. The state parameter should be something recognizable to our API but very hard for a malicious actor to guess, such as a session hash or some other information we are interested in. Once we receive it from Google’s auth page, we can continue the user’s conversation using the data in the state parameter.

To mask the data from bad actors, we will encode this object as a Base64 string. Base64 is an ASCII representation of binary data.² Since a malicious actor could easily compromise this information by simply decoding from Base64, we will use crypto-js to encrypt the state string.

First, let’s install the two packages.

Second, let’s add three variables to our .env file representing the client ID, secret, and redirect URI. We use the redirect URI we provided in Figure 7-8 and the client ID and secret we received in Figure 7-9.

Third, we need to generate the URL to the login page and send a button that can open this URL. The Google Auth APIs can do a lot of this for us. We will do a few things in our code. First, we import the crypto-js and googleapis packages. Next, we create an OAuth2 client instance including our client data. The state that we will send as part of the login URL contains the user’s address. As shown in the previous chapter, an address is sufficient to uniquely identify a user’s conversation, and Bot Builder contains the facilities to help us send messages to that user by simply presenting the conversation address. We use crypto-js to encrypt the state, using the ASE algorithm.³ AES is a symmetric-key algorithm, which means that the data is encrypted and decrypted using the same key or passphrase. We add the passphrase into our .env file with the name AES_PASSPHRASE.

Another thing to note is the scopes array. When requesting authorization to the Google APIs, we specify to Google which APIs we are looking for access to using scopes. We can think of each item in the scopes array as a piece of data we want to access about the user from Google’s APIs. Of course, this array needs to be a subset of the APIs our Google project may access to begin with. If we added a scope we did not enable for our project earlier, the authorization process would fail.

We also need to be able to send a button for the user to utilize to authorize the bot. We utilize the built-in SigninCard for this purpose.

The emulator renders the SigninCard as per Figure 7-10.

At this point we can click the Login button to log into Google and authorize our bot to access our data, but it would fail because we have not yet provided the code to handle the message from the return URI. We use the same approach to install a handler for the https://a4b5518e.ngrok.io/oauth2callback endpoint as we did to install the API messages endpoint. We also enable restify.queryParser, which will expose each parameter in the query string as a field in the req.query object. For example, a callback in the form redirectUri?state=state&code=code will result in a query object with two properties, state and code.

We read the authorization code from the callback and use the Google OAuth2 client to get the tokens from the token endpoint. The tokens JSON will look like the following data. Note that expiry_date is the datetime in milliseconds since the epoch.⁴

Once we receive the tokens, we call setCredentials on the OAuth2 object, and it can now be used to access the Google Calendar API!

In the code location where we have access to the Calendar API, we can write code that gets a list of calendars that we own and prints out their names. Note that calapi in the following code is a helper object that wraps the Google Calendar API in JavaScript promises. The code is available in the chapter’s code library.

This code results in the following console output, which is an unfortunate reminder of the rather lonely workout schedule that has not seen much action since I became a dad.

Fatherhood weight gain aside, this is great! We do have a few challenges. We need to store the users’ OAuth tokens so we can access them any time the users message us. Where do we store them? This one is easy: private conversation data. How do we get access to that data dictionary in this context? We do this by passing the user’s address to the bot.loadSession method.

Recall that we stored in the user’s address into the encrypted state variable. We can decrypt that object by using the same passphrase we used to encrypt the data.

After we receive the token, we can load the bot session from the address. At that point, we have a session object that has all the dialog methods such as beginDialog for us to use.

The processUserCalendars dialog could look something like this. It sets the tokens into the private conversation data, lets the user know they are logged in, and displays the names of all of the client’s calendars.

The interaction would look like Figure 7-11.

Seamless Login Flow

We have successfully logged in and stored the access token, but we have not yet demonstrated a seamless mechanism to redirect to a login flow when a dialog requires our users to be logged in. More specifically, if in the context of the calendar bot a user is not logged in and asks the bot to add a new calendar entry, the bot should show the login button and then continue with the Add Entry dialog once login is successful.

We will naturally implement a Login dialog and a Logout dialog. Logout simply checks the existence of tokens in the conversation state. If we do not have the tokens, we are already logged out. If we do, we use Google’s library to revoke the user’s credentials.⁵ The tokens are no longer valid.

Login is a waterfall dialog that begins an EnsureCredentials dialog before it gets to the next step. In the second step, it verifies whether it is logged in. See the following code. It does this by verifying that it receives the authenticated flag from the EnsureCredentials dialog. If yes, it simply lets the user know that she is logged in. Otherwise, an error is shown to the user.

Notice what we did here. We outsourced the logic of figuring out if we are logged in, logging in, and then sending the result back to a different dialog. As long as that dialog returns with an object with fields authenticated and, optionally, error, this just works. We will use the same technique to inject an authorization flow into any other dialog that requires it.

We illustrate the flow for the first case in Figure 7-12. A dialog requests that we have the user’s authorizations before continuing, like the Login dialog did in the previous code. The user is sent to the auth page. Once the auth page returns a successful authorization code, it sends a callback to our oauth2callback. Once we get the tokens, we call a StoreTokens dialog to store the tokens into the conversation data. That dialog will return a success message to EnsureCredentials . In turn, this returns a successful authentication message to the calling dialog.

If an error occurs, the flow is similar except that we replace the EnsureCredentials dialog with the Error dialog. The Error dialog will then return a failed authenticate message to the calling dialog, which can handle the error as it best sees fit (Figure 7-13). Recall, as we noted in Chapter 5, replaceDialog is a call that replaces the current dialog on top of the stack with an instance of another dialog. The calling dialog does not know, nor care, about this implementation detail.

In the case that the user clicks the login button when a dialog is not expecting a reply and EnsureCredentials is not on top of the stack, the flow is slightly different. We want to display a success or failure message to the user if the authorization succeeds or fails. To achieve this, we will put a confirmation dialog, AuthConfirmation, on the stack before invoking the StoreTokens dialog (Figure 7-14).

Likewise, in the case we receive an authorization error, we push the AuthConfirmation dialog on top of the stack, before pushing the Error dialog (Figure 7-15). This will ensure that the confirmation dialog displays the right type of message to the user.

Let’s see what the code for this looks like. The Login and Logout dialogs are done, but let’s look at EnsureCredentials, StoreTokens, and Error.

EnsureCredentials is composed of two steps. First, if the user has a set of tokens defined, the dialog finishes passing a result indicating that the user is good to go. Otherwise, we create the auth URL and send a SigninCard to the user, just like we did in the previous section. The second step also executes in case 1. It simply tells the calling dialog that the user is authorized.

StoreTokens and Error are similar. Both essentially return an authorization result to its parent dialog. In the case of StoreTokens , we also store the tokens into the conversation data.

Note that EnsureCredentials is going to consume the result of either of these two and simply pass it down to the calling dialog. It is up to the calling dialog to display a success or error message. There may not even be a success message; the calling dialog may just jump into its own steps.

That covers cases 1 and 2. To ensure cases 3 and 4 are covered, we need to implement this AuthConfirmation dialog. The role of this dialog is to display either a success or failure message. Recall that we place either an Error (case 3) or StoreTokens (case 4) dialog on top of AuthConfirmation. The idea is that AuthConfirmation will receive the name of the dialog to place on top of itself and then send the appropriate message to the user when it receives a result.

Lastly, how do we change our endpoint callback code? Before we get there, we write a few helpers to invoke the different dialogs. We expose a function called isInEnsure that verifies whether we are getting into this piece of code from the EnsureCredentials dialog. This will dictate whether we need the AuthConfirmation. beginErrorDialog and beginStoreTokensAndResume both utilize this approach. Finally, ensureLoggedIn is the function that each dialog that requires authorization must call to kick off the flow.

Finally, let’s look at the callback. The code looks similar to our callback in the previous section, except we need to add the logic to begin the right dialogs. If we encounter any errors while loading our session object or we get an OAuth error, such as the user declining access to our bot, we redirect the user to the Error dialog. Otherwise, we use the authorization code from Google to get the tokens, set the credentials in the OAuth client, and call into the StoreTokens or AuthConfirmation dialog. The following code covers the four cases highlighted at the beginning of this section:

Integrating with the Google Calendar API

We are now ready to integrate with the Google Calendar API. There are a few things we should address first. Google Calendar allows users to have access to multiple calendars and, further, to have a different permission level for each calendar. In our bot, we assume that at any point we are querying or adding events into only one calendar, as flawed as that may seem. We could extend the LUIS application and bot to include the ability to specify a calendar for each utterance.

To handle this, we create a PrimaryCalendar dialog that allows users to set, reset, and retrieve their primary calendar. Similar to the EnsureCredentials dialog being called at the beginning of each dialog that requires authentication, we create a similar mechanism to guarantee that a calendar is set as primary.

Before we get there, let’s talk about connecting to the Google Calendar API. The googleapis node package includes the Calendar APIs, among others. The API utilizes the following format:

A calendar call would look as follows:

First, we will adapt this to the JavaScript Promise⁶ pattern. Promises make it easy to work with asynchronous calls. A promise in JavaScript represents an eventual completion or failure of an operation, as well as its return value. It supports a then method that allows us to perform an action on the result and a catch method that allows us to perform an action on the error object. Promises can be chained: the result of a promise can be passed to another promise that produces a result that can get passed into another promise and so forth, resulting in code that look as follows:

Our modified Google Calendar Promise API will look as follows:

We wrap all the necessary functions in a module called calendar-api. Some of the code is presented here:

The PrimaryCalendar dialog is a waterfall dialog with three steps. Step 1 ensures that the user is logged in by calling EnsureCredentials. Step 2 expects to receive a command from a user. We can either get our current primary calendar, set a calendar, or reset our calendar; thus, the three commands are get, set, or reset. Set calendar takes an optional calendar ID. If a calendar ID is not passed, the set command is treated equivalently to reset. Reset simply sends the user a list of all available calendars to which a user has write access to (another simplifying assumption).

The get case is handled by this code:

The reset case sends the user a carousel of calendar cards. If the user enters a text input, the third step of the waterfall assumes that the input is a calendar name and sets the right calendar. If the input isn’t recognized, an error message is sent.

The createCalendarCard method simply sends a card with a title, subtitle, and a button that sends the set calendar command. The button posts back this value: Set primary calendar to {calendarId}.

This presents an interesting challenge. If a calendar card is sent in any context other than the PrimaryCalendar dialog, we need a full utterance like to resolve to a global action, which then invokes the PrimaryCalendar dialog. However, if we serve a card like this in the context of the primary calendar dialog, the button will still trigger the global action, therefore resetting our entire stack. We don’t want to set different text based on which dialog created the card because these buttons remain in the chat history and can be clicked any time.

In addition, if the PrimaryCalendar dialog is invoked, we would like to ensure that it does not get rid of the current dialog. For example, if I am in the middle of adding an appointment, I should be able to switch calendars and come back to the right step in the process afterward.

We override the triggerAction and selectAction methods to ensure the right behavior. If another instance of the PrimaryCalendar dialog is on the stack, we replace it. Otherwise, we push the PrimaryCalendar dialog to the top of the stack.

If a PrimaryCalendar dialog is invoked while the user is within another instance of the PrimaryCalendar dialog, we replace the top dialog with another instance of the PrimaryCalendar dialog. In reality, and bear with me here, this will occur only in the reset command, and it will actually replace the builder.Prompts.text dialog that we invoke in handleReset.

So, in essence we end up with a PrimaryCalendar dialog waiting for a response object that can now come from another PrimaryCalendar dialog. We can have the topmost instance return a flag when it is done so that the other instance simply exits when the third step resumes. Here is the final waterfall step to illustrate this logic:

The set action is less complex. If we receive a calendar ID along with the user message, we simply set that message and send back a card of the calendar. If we do not receive a calendar ID, we assume the same behavior as reset.

That was a lot to process, but it is a good illustration of some the dialog gymnastics that need to occur to ensure a consistent and comprehensive conversational experience. In the following section, we will integrate the authentication and primary calendar flows into the dialogs we developed in Chapter 6 and connect the logic to calls into the Google Calendar API.

Implementing the Bot Functionality

Let’s start with the first two items. We have created the EnsureCredentials and PrimaryCalendar dialogs for this very purpose. In the provided code, our authModule and primaryCalendarModule modules contain a couple of helpers to call the EnsureCredentials and PrimaryCalendar dialogs. Each of our functions can utilize the helpers to ensure that the credentials and the primary calendar are set.

This is too much responsibility for those dialogs. We would have to add two steps into every single dialog. Instead, let’s create a dialog that can evaluate all the prechecks in the correct order and simply pass one result to the calling dialog. Here’s how we would achieve this. We create a dialog called PreCheck . This dialog will make the necessary checks and return a response object with an error set if there is an error as well as a flag indicating which check failed.

Any dialog that needs auth and a primary calendar to be set only needs to invoke the PreCheck dialog and ensure there was no error. Here is an example from the ShowCalendarSummary dialog in the sample code. Note that the first step in the waterfall calls PreCheck, and the second step ensures all prechecks successfully passed.

So that’s it for the first two items. At this point, all that is left is the third one; we need to implement the actual integration with the Google Calendar API. The following is the example of what the third step of the ShowCalendarSummary dialog looks like. Notice that we gather the datetimeV2 entities to figure out what time period we need to retrieve events for, we optionally use the Subject entity to filter out calendar items, and we build a carousel of event cards, ordered by date. The createEventCard method creates a HeroCard object for every Google Calendar API event object.

The implementation of the remaining dialogs is available in the calendar-bot-buildup repository included with the book.

Conclusion

Building bots is both easy and challenging. It is easy to set up a basic bot with some simple commands. It is easy to get user utterances and execute code based on them. It is, however, quite challenging to get the user experience just right. As we have observed, the challenges in developing bots are twofold.

First, we need to make sense of the many permutations of natural language utterances. Our users can say the same things in numerous ways with nuanced variations. The LUIS application we’ve built for this book are a good start, but there are many other ways of expressing the same ideas. We’ll need to exercise judgment on when we say that a LUIS application is good enough. Bot testing is where a lot of this kind of evaluation occurs. Once we unleash a set of users on your bot, we will see how users end up using your bot and what type of inputs and behaviors they expect to be handled. This is the data we need to improve our natural language understanding and decide what features to build next. We will cover analytics tools that help with this task in Chapter 13.

Second, it is important to spend time on the overall conversations experience. Although this is not the focus of this book, a proper experience is key to our bots’ success. We did spend some time thinking around how to ensure that the user is logged in before we proceed into dialogs with any actions against the Calendar API. This is an example of the type of behaviors and flows that need to be thought through as we develop a bot. A more naïve bot may simply send the user an error saying they need to log in first, after which the user would have to repeat their input. A better implementation is the redirection through dialogs that we created in this chapter. Lucky for us, the Bot Builder SDK and its dialog model help us describe these complex flows in code.

We now have the skills and experience to develop complex and amazing bot experiences, with all types of API integrations. This is the real combined power of LUIS and the Microsoft Bot Framework!