What’s new for developers from Build 2024 – Copilot for Microsoft 365 Edition

ahhhhh, Build. A glorious smörgåsbord of announcements, sessions, content and news.

It’s a great time to learn new things, meet new people and get a feel for what’s important to Microsoft right now (hint: it’s Copilot).

Rather than just re-announcing everything that’s already been announced though, I’ve been trying to separate out all the different ways that developers can extend Copilot, and then categorising that information. Partly this hopes to provide a useful service to other developers, but a large part of it is to aid my own understanding and help clarify what is being said. Sometimes, the amount of information can be overwhelming.

I think what can help though, is to remember that Build is not just a place where Microsoft announces new stuff. It’s also a place where Microsoft recaps announcements made since the previous Build and affirms products and technologies they think are important. For many people, flagship events like Build and Ignite are how people stay current and they come to these events to “catch up”. For those of us that follow every announcement and blog post that happens though, these events can be confusing because there’s a real mixture of recap vs new. This post helps to explain all that.

This is the “all up” extension matrix that Microsoft presented at Build this year (this slide is from Developer’s Guide to Customizing Microsoft Copilot). We’re going to dig into parts of it to understand what was there before, what’s new and what’s still to come:

What did we already have going into Build?

Before Satya began his keynote last week, this was how developers could interact with Copilot (and AI more generally) in Microsoft 365:

Graph Connectors

An important but not terribly exciting part of the puzzle, Connectors are how you can bring additional data from outside Microsoft 365 into the data grounding model that Copilot uses to answer questions. This has been around for some time now, and I don’t think there were any big announcements concerning Connectors this year.

Message Extension based Copilot Plugins

Developers could take the existing investment in Message Extensions and build search-command-based plugins for Copilot for Microsoft 365, which would work in Microsoft Teams Copilot Chat. Watch: How to: build and deploy the EASIEST and yet MOST IMPACTFUL plugin for Copilot for Microsoft 365! (youtube.com)

Teams AI Library based bots

Developers could use the Teams AI Library (GA November 2023) to write bots that include AI. This is nothing to do with Copilot in Microsoft 365: it’s scaffolding that default to OpenAI but could use any LLM and re-imagines bot conversations from being selection-based to being more capability-based, with the LLM acting as a decision engine on what capabilities to invoke when. The bots built here work as Microsoft Teams bots, once packaged as Teams apps.

What new stuff was announced (and can I get it yet)?

New capabilities for plugins

The existing message extension based plugins will be able to support action commands as well as search commands. This will help them be more useful, as rather than just returning data they will also be able to perform actions. This is not available today but is “coming soon”.

In addition, developers will be able to create Copilot plugins using an OpenAPI specification and manifest files. This is not available today apart from in private preview. (Build session: Developer deep dive on building plugins for Microsoft Copilot)

Declarative Copilots

Declarative Copilots are a new concept, and a new way of using the built-in Copilot for Microsoft 365. Developers can define a “way of working” with Copilot – a set of instructional prompts, additional data or focused and scoped grounding data, and a set of plug-ins and connectors, packaged together into a single offering. This means that developers can produce a “copilot for xxx” business unit that has everything needed in one place with some surrounding prompt information. Users don’t have to pick the plugins they need, it’s done for them, and the business knows that this bot is grounded in a specific set of data with specific rules for how it will respond. It’s a neat way of packaging up and then exposing Copilot functionality. Is it development, or IT Pro configuration? Good question. 🙂

Declarative Copilots can appear as their own bots for users to approach and converse with directly, or in-context when users are chatting with Copilot and choose to invoke them as extensions.

An example of a Declarative Copilot might be a Policy Bot, which a developer has: a) scoped to only look at specific policy documents in a particular SharePoint site, b) included a number of useful plugins for referencing, signing or feeding back on policies, and c) provided an instructional prompt that Copilot should give clear, concise answers and end every message with a disclaimer.

I think this is the same underlying technology that has enabled the “build a copilot” from a SharePoint folder experience which was demo’d in the Build keynote.

This is not available today apart from in private preview, so hopefully we can expect it soon.

Custom Copilot

Custom Copilot is the new name being given to the Teams AI Library, with the added bonus that bots created here will be available for use in Copilot more generally and not just in the Teams interface. This is not available today: it’s “coming soon”. However, there’s nothing to stop you building out a bot using the Teams AI Library so that you’re ready to move it to Copilot when the time comes.

Copilot Handoff

You can imagine a world where a development team might produce a full-featured bot using Teams AI Library (or eventually maybe a Declarative Copilot), and ALSO a plugin to provide quicker and easier access to common information or actions. For users, it would be nice to be able to start work in a plugin and then jump into the “full experience” if needed.

To make that more seamless, we now have a concept of Copilot Handoff. Essentially this is a deep link button developers can add to their plugins, which will invoke the bot together with the conversation so far.

This is less of a “feature” than a “pattern”. Developers will need to create and store a continuation token which describes the conversation the user is having with the plugin, and pass that continuation token as a parameter in the deep link URL. Once clicked, the receiving bot code will use it to read the conversation from wherever it was stored and decide what to do with it. There’s no abstraction or helper layer here: developers need to implement this feature completely. It’s a nice idea, but it would be nice to see more abstraction and automation in this area. As it’s mostly something developers have to build, this is technically something you can do today, although it is in Developer Public Preview (likely due to the type of deep link that’s used). There’s a code sample example here: Microsoft-Teams-Samples/samples/msgext-copilot-handoff/ts at main · OfficeDev/Microsoft-Teams-Samples (github.com)

Conclusion

I’m sure there’s stuff I’ve missed here (so let me know).

I also haven’t covered Copilot Studio at all (I shall try to, but it’s not my area of expertise and others are doing a better job than me)

I also haven’t covered any non-Copilot Microsoft 365 Developer Extensibility news. That doesn’t mean there isn’t any, just that I haven’t had the chance to look for it yet.

Are you ready to leverage the power of Copilot for your team? Start experimenting with these new features today. Share your experiences and projects in the comments below or on social media. Don’t forget to subscribe to the blog for more updates and tutorials on Microsoft 365 development!

And, if you want to know, I’d suggest hitting the Build sessions, in particular:

Developer’s Guide to Customizing Microsoft Copilot
Azure AI Studio – Creating and scaling your custom copilots (microsoft.com)
Build Microsoft Copilot extensions with Copilot Studio
Developer deep dive on building plugins for Microsoft Copilot
Build your own copilot with Microsoft Copilot Studio
Reach 320M+ users in flow of work building your own copilot for Teams (microsoft.com)

Summary FAQs

1. What are Graph Connectors and how do they integrate with Copilot for Microsoft 365?

Graph Connectors are tools that allow developers to bring additional data from external sources into the data grounding model that Copilot uses. This integration helps Copilot provide more comprehensive answers by accessing a wider range of data. While no new announcements concerning Connectors were made at Build 2024, they remain a crucial component for enriching the data Copilot can access.

2. Can I start building plugins with action commands for Copilot today?

As of now, building plugins with action commands is not available. However, it is expected to be available soon. Keep an eye on updates from Microsoft to know when this capability becomes available.

3. What are Declarative Copilots and how can they benefit my business?

Declarative Copilots are a new way for developers to package Copilot functionalities into a single offering. By defining prompts, additional data, focused grounding data, and integrating plugins and connectors, Declarative Copilots can provide tailored “copilots” for specific business units or tasks. This consolidation ensures users interact with a bot grounded in pre-defined data and rules, making it easier for businesses to manage and deploy Copilot functionalities.

4. How is Custom Copilot different from existing bots built using the Teams AI Library?

Custom Copilot is essentially an evolution of the Teams AI Library. The primary difference is that bots created under Custom Copilot will be usable across various Microsoft 365 applications, not just within Teams. This broadens the scope and utility of the bots, making them more versatile across different platforms in the Microsoft 365 ecosystem.

5. What is Copilot Handoff and how can it improve user experience?

Copilot Handoff is a concept designed to allow seamless transitions between different bot functionalities. For example, a user can start interacting with a simple plugin and, if more complex information or actions are needed, jump into a full-featured bot while preserving the conversation context. This feature requires developers to create continuation tokens and handle the deep linking process, making the user experience more fluid and integrated.