Microsoft Outlines Framework For Building AI Apps & Copilots

Microsoft has announced that it has expanded this ecosystem of Microsoft Copilots to include Copilot in Power BI and Copilot in Power Pages in preview, Copilot in Microsoft Fabric, available in preview soon, and Windows Copilot, which will start to become available for preview in June.

The company also introduced new features that will help developers build their own copilots and next-generation AI applications. This includes new tools called plugins that make copilots more useful by allowing them to interact with other software and services.

Speaking during the annual Microsoft Build developers conference, Kevin Scott, Microsoft’s chief technology officer said, “You may look at Bing Chat and think this is some super magical complicated thing, but Microsoft is giving developers everything they need to get started to go build a copilot of their own. I think over the coming years, this will become an expectation for how all software works.”

OpenAI introduced ChatGPT plugins in March. Microsoft announced plugins for Bing earlier this month. That technology allows ChatGPT and Bing Chat to help you find and book a restaurant reservation using an OpenTable plugin, for example. Today, Microsoft is adopting the same open plugin standard that OpenAI introduced for ChatGPT, enabling interoperability across ChatGPT and the breadth of Microsoft’s copilot offerings. That means developers can now use one platform to build plugins that work across both business and consumer surfaces, including ChatGPT, Bing, Dynamics 365 Copilot, Microsoft 365 Copilot and Windows Copilot. Microsoft also announced it is bringing Bing to ChatGPT as the default search experience.

As part of this shared plugin platform, Bing is adding to its support for plugins. In addition to the ones previously announced for OpenTable and Wolfram Alpha, it will also have Expedia, Instacart, Kayak, Klarna, Redfin, TripAdvisor and Zillow among many others in the Bing ecosystem. In addition, developers will now be able to extend Microsoft 365 Copilot with plugins. Plugins for Microsoft 365 include ChatGPT and Bing plugins, as well as Teams message extensions and Power Platform connectors – enabling developers to leverage their existing investments. And developers will be able to easily build new plugins with the Microsoft Teams Toolkit for Visual Studio Code and Visual Studio.

At Microsoft Build, the company showcased an AI development framework that helps developers build their own copilot. This stack includes support for plugins that augment the capabilities of AI systems by allowing them to retrieve real-time information, incorporate a company or other business data, perform new types of computations and safely take action on the user’s behalf.

At the root, think of plugins as a bridge. This could be a bridge between a large language model that was trained on public data from the internet and all the data that a company may keep privately about its benefits. The plugin is the bridge that gives the copilot access to those files when it answers a question from an employee at the company.

“A plugin is about how you, the copilot developer, give your copilot or an AI system the ability to have capabilities that it’s not manifesting right now and to connect it to data and connect it to systems that you’re building,” Scott said. “I think there’s going to eventually be an incredibly rich ecosystem of plugins.”

At Microsoft Build, the company also introduced new tools to simplify the development of copilots. For example, with the new Azure AI Studio, developers can now more easily ground conversational AI models on their private data. With a new capability called Azure OpenAI Service on your data, developers can uncover organization-specific insights from data, text and images using natural language-based application interfaces.

To further extend the capabilities of large language models, Microsoft also announced support for plugins with Azure OpenAI Service. Azure AI also now supports Azure Machine Learning prompt flow, a sophisticated prompt engineering tool, noted Montgomery.