Microsoft is currently testing a new feature called "Copilot Actions" on Windows 11, which has sparked attention due to its potential risks. Initially available only to users on Insider builds and turned off by default, this feature is not intended for widespread use yet. With the requirement of admin access to enable the feature, it indicates that it is still in an experimental phase.
One of the significant concerns related to Copilot Actions stems from Microsoft’s acknowledgment of "novel security risks." A specific threat mentioned is cross-prompt injection, which occurs when malicious content embedded in documents, websites, or UI elements tricks the AI into bypassing its original instructions. Such vulnerabilities could allow for serious repercussions, including the risk of data theft or even the installation of malware onto users' devices.
Despite its intended purpose to aid users, there are fears that the Copilot Actions feature could be manipulated, turning a useful tool into a potential threat. Microsoft has advised users to enable the feature only if they fully comprehend its security implications. Enabling this feature triggers a warning pop-up informing users that it is experimental and might negatively impact their PC’s performance or security.
A deeper concern lies in the level of access Copilot Actions receives upon activation. It gains read and write permissions to critical folders such as Desktop, Documents, Downloads, Pictures, Videos, and Music—which encapsulate a user’s entire digital personal life. While Microsoft asserts that there are safeguards, including requiring user approval and logging activities, skepticism remains concerning the degree of freedom given to an unpredictable AI system.
Moreover, Microsoft has reiterated that AI models can produce "hallucinated" outputs that are unexpected. This fact raises alarm when such systems can interface with sensitive applications and personal files.
In response to inquiries about safety measures users might follow if they choose to activate Copilot Actions, Microsoft did not provide detailed guidelines. Consequently, it appears that the current consensus advises caution; the feature promises power and innovation, but it is ultimately not suitable for all users. If there is any uncertainty about the associated risks, it is most prudent to keep Copilot Actions disabled until further reassurance about its safety and reliability is provided.