Video: How Microsoft 365 Copilot chooses plugins … and how to make sure it chooses YOURS!

As Microsoft 365 Platform Developers building messaging extensions, I noticed something about how our audience is changing. We used to write messaging extensions for people, but things have changed. Now, our audience includes not just people, but also Copilot. This AI is now making crucial decisions about which messaging extensions are shown to users. It’s a shift that we all need to be prepared for.

So, how does Copilot actually make these decisions? Well, it’s a two-step process: shortlisting and reasoning. Initially, Copilot has to pick from a myriad of different apps. This includes the one you’ve developed, those from other developers, and many internally installed apps that it can call upon. To be considered, your app needs to pass through this shortlisting process.

Next is the reasoning stage. Here, Copilot takes the user’s original prompt and breaks it down into tasks that it aims to execute using skills. These skills could be built-in ones that Microsoft already has, or they might be in plugins. In this stage, Copilot matches the tasks it identifies in the user prompt to the skills that plugins have advertised.

Now comes the crucial question: how do you ensure your app gets chosen? Here’s the catch: you don’t want your app to be chosen all the time, just when you’re confident it can effectively answer the user’s query. If not, users will get frustrated and uninstall your plugin.

So, what should you do? You need to rewrite them, not for humans, but for Copilot. You need to be crystal clear about what your application does, the value it brings, and examples of how users might use it. By putting all this in the description, you can give Copilot a solid understanding of what your app does, thereby increasing its chances of being chosen in the shortlisting process.

You also need to look at skill descriptions. Copilot uses these to understand the skills your plugin has and how it can map those to user tasks. There are two areas where you can significantly influence this process: the description of your skills and the parameters.

Each command in your plugin and your manifest should have a detailed description. This is where you can really help Copilot understand your app’s capabilities. Be descriptive, provide examples, and emphasize the value that your app can offer. But also, pay special attention to the parameters – the inputs your plugin takes. Be clear with the descriptions you give to these parameters, as this is how Copilot matches user inputs to parameters in your skills.

If you get this wrong, your app is going to deliver disappointing results, and users will be frustrated. But if you get it right, your plugin becomes an integral part of Copilot, providing users with excellent results. They won’t even realise they’re dealing with your app; they’ll just appreciate the high-quality, useful outcomes.

Ref/more detail: https://learn.microsoft.com//microsoft-365-copilot/extensibility/orchestrator

How does the shortlisting process work in detail? What specific criteria does Copilot use to select an app? Other than the information given above and in the Microsoft Learn page, the actual inner workings of how Copilot works are proprietary and not publicly disclosed. However, it’s very likely that Copilot considers factors like the relevance of the application to the user query, the clarity and precision of the app’s descriptions, and the success rate of the app in previous interactions.

How can I measure or evaluate the effectiveness of my descriptions and parameters in attracting Copilot’s attention? Measuring the effectiveness of your descriptions and parameters may not be straightforward due to the opaque nature of Copilot’s selection process. However, you can gauge the effectiveness indirectly by tracking user interactions with your app, the frequency of its selection by Copilot, and user feedback. If your app is being selected more often and receiving positive feedback, it’s likely your descriptions and parameters are effective.

If my app isn’t being selected by Copilot as often as I’d like, what troubleshooting steps can I take to improve its visibility? If your app isn’t being selected as often as you’d like, it’s worth revisiting your descriptions, titles, and parameters. Ensure they are clear, concise, and accurately represent what your app does. Also, consider the user feedback. If users are uninstalling your app or leaving negative reviews, it might indicate areas for improvement.

What other factors, apart from descriptions and parameters, might influence Copilot’s selection of one app over another? Other factors that could influence Copilot’s selection may include the overall user experience of your app, the success rate in fulfilling similar tasks in the past, and possibly, user ratings. While the descriptions and parameters are crucial, they are part of a larger ecosystem that contributes to your app’s success.

How can I stay updated on any changes in the way Copilot selects and interacts with plugins, so that I can continually optimize my app? To stay updated on changes in the way Copilot selects and interacts with plugins, you should follow official Microsoft blogs, forums, and updates. Participating in developer communities can also keep you informed about the latest practices and trends. Microsoft often releases updates and guidelines for developers, which would be valuable resources for optimizing your app. I run a weekly update of news and items of interest for Microsoft 365 Developers which you can subscribe to in YouTube or podcast.