Microsoft has taken a significant step forward in artificial intelligence technology with the introduction of a new feature for its Copilot Studio this week. This innovative 'computer use' feature allows AI agents to interact seamlessly with websites and desktop applications. In a manner reminiscent of similar functionalities offered by OpenAIs Operator and Claudes own 'computer use' feature, businesses will now have the capability to develop AI agents that utilize websites and desktop applications as tools to automate various tasks.

Charles Lamanna, the corporate vice president of Microsofts business and industry Copilot, elaborated on this exciting development. He noted, Computer use enables agents to interact with websites and desktop apps by clicking buttons, selecting menus, and typing into fields on the screen. This groundbreaking functionality means that agents can execute tasks even when there is no direct API available to connect to a given system. Essentially, if a human user can operate the application, the AI agent can replicate that interaction as well.

The potential applications for the new computer use tool in Copilot Studio are vast and varied. For instance, businesses can now leverage AI agents to automate tedious tasks such as data entry, conduct thorough market research, or even process invoices with remarkable efficiency. According to Microsoft, this new tool is designed to be remarkably resilient; it can detect changes in buttons and screens across applications and websites, ensuring that the agents continue to function effectively without breaking or failing.

Earlier this month, Microsoft also introduced a comparable feature called Actions in its consumer Copilot. Actions allows the AI to perform tasks in the background while users focus on other work. Now, Copilot can handle tasks such as booking restaurant reservations, securing event tickets, and purchasing items from online retailers. While the Actions feature is currently limited to select partners, it appears that Copilot Studio will have the advantage of being applicable to a broader range of websites and applications, thus enhancing its utility for businesses significantly.