Windows and Office get smart: An overview of Microsoft’s AI services

In the last two years, Microsoft has invested a lot of money in the ChatGPT inventor Open AI and in its own AI developments.

Since then, Microsoft has been equipping more and more programs with artificial intelligence, above all the Edge browser and Office programs. But AI is also finding its way into Windows.

In this article, we present the new AI functions in Windows, the Windows apps, and Microsoft 365.

Artificial intelligence: Copilot

Microsoft’s AI engine Copilot has been part of the operating system since Windows 11 23H2 for U.S. users, and is available via the Copilot app in the Microsoft Store for others. It can be integrated into the taskbar or simply accessed via the Start menu.

The trick: The app simply links to Copilot in Edge, where the AI then runs as an independent program in its own window with a slightly different interface. However, Edge must be installed; the app will not work without the browser.

You can chat with Windows Copilot and ask questions, which the AI assistant answers with the help of Bing. It displays the sources it has used. You can choose between “Creative,” “Balanced,” and “Accurate” conversation styles.

However, the differences are hardly noticeable. The AI can also write letters, articles, and poems. With the help of Microsoft Designer and the support of the open AI tool Dall-e 3, Copilot can also generate images based on descriptions.

Mark Hachman / IDG

The Edge browser has been equipped with Copilot since 2023. You can open the AI function via the icon in the top right-hand corner.

Compared to the app, this has two advantages: Firstly, it allows you to ask questions directly about the website currently being accessed in the browser or request a summary of an online article. In the app, you can simply formulate a command such as “Summarize this article” and then add a link to the desired website.

You can also activate Copilot plugins in Edge, such as Kayak for searching for flights, accommodations, and car rental, or Opentable for searching for restaurants. This is not possible in the Windows app.

Cocreator & Imager Creator: Create images in Paint

The Windows drawing program Paint gets new features from time to time, most recently working on multiple layers and the AI function Image Creator were added — previously called Cocreator.

Following this renaming to Image Creator, Cocreator is now available again in parallel, albeit with more advanced functions.

However, most Windows users will not be able to use the new Cocreator, as it only runs on Copilot Plus notebooks (see box “AI processors in Copilot PCs”). The new Cocreator can now be found in the Paint toolbar alongside the Image Creator.

The Image Creator is a slimmed-down version of the Bing Image Creator available online and Microsoft Designer.

All three tools are based on Dall-e from Open AI and create images using descriptive text.

You simply type in what you want to see and Image Creator creates a corresponding image.

The Cocreator also works in the same way. However, while the Image Creator accesses Microsoft’s server resources on the internet, the Cocreator works locally on the computer and uses the NPU or the AI processor of the Copilot Plus notebooks.

Local processing ensures better data protection, but Cocreator still requires an internet connection.

Mark Hachman / IDG

The Image Creator in Paint accepts your descriptions and generates images from them. You can choose between photos and different painting styles.

Retouch photos with the Windows photo viewer

The Windows Photo Viewer, simply called “Photos” in earlier versions, has had a generative erase function for a few months now. This allows you to selectively remove individual objects from a photo.

The software fills the empty space amazingly well in the style of the image environment, the editing is often not recognizable. To do this, open the photo display by entering photos in the search field in the taskbar.

In the program, click on the “All photos” icon on the left or on “Folder;” here you can add a picture directory to the program via “Add folder.”

To open, double-click on an image in the preview and then click on the “Edit” button at the top left. Now select the “Delete” command and set the brush size on the right-hand side.

The smaller the object to be deleted, the smaller the brush should be. Use the mouse to select the object to be deleted or the distracting area. As soon as you release the mouse button, the software starts working and retouches the photo.

Microsoft

AI processors in Copilot Plus PCs

The use of AI functions is very computationally intensive. For this reason, AI operations are currently mostly outsourced to the cloud, i.e. to the global data centers of Amazon, Google, Microsoft, and others.

This causes potential problems with data protection, as the prompts, i.e. your queries, also end up with the IT companies and can therefore be assigned to you. At the same time, the AI operations in the data centers consume a lot of money and energy.

Samsung

This is why PCs have been developed that can also perform AI tasks locally. Microsoft certifies correspondingly powerful Windows computers with the “Copilot” logo. These new notebooks have been available since the summer, initially all models with a Snapdragon X processor from Qualcomm and Arm architecture.

AMD now also offers a CPU for Copilot Plus, Intel follows with its Core Ultra 200V mobile processor.

The CPUs in Copilot Plus PCs accelerate AI functions and calculations to such an extent that it is not necessary to move them to the cloud in most cases. These chips are known as Neural Processing Units (NPUs).

Designer: AI functions in the Microsoft 365 apps

Designer was initially only available in PowerPoint, but is now included in the Microsoft 365 word processor. You will therefore need a paid Personal or Family account to be able to use the AI functions.

Corporate customers are currently excluded. With a 365 account, you can use Designer in the online version of Word and in the desktop and online versions of PowerPoint. The program appears in the “Start” ribbon in each case.

In PowerPoint, give your first slide a title and call up the Designer. It suggests various design templates that match the theme of the presentation. However, this only works in English.

Advanced AI function Copilot Pro only with additional subscription

If you click on “Test Copilot Pro” in one of the 365 programs under File > Account > Manage account > Account, you can play with the extended AI functions for the Office programs free of charge for one month. After that, the subscription costs $22 per month.

After activation, the Copilot icon appears at the beginning of the line next to the cursor in Word. After clicking on it, you can send the AI a description of the type of text you need. For example, Copilot can write speeches on a given topic, create a template for a CV or write short stories.

In Excel, Copilot automatically creates formulas and sorts, analyzes and highlights data. However, this only works in the online version of the program and the files must be stored in a Onedrive or Sharepoint folder.

Although Microsoft has made improvements in the meantime, the AI in Excel is still not as good as we would like it to be.

In Outlook, Copilot is also limited to the online version. There, the AI summarizes the content of emails and translates foreign-language emails into your language of choice.

Copilot Pro can also compose messages itself based on a short description.

The video editor Clipchamp offers a little more AI. The tool displays the “Create video with AI” button on its start page.

Click on it to call up an AI wizard, which you can use to upload the images and videos for the Clipchamp film. After asking for a few pre-settings, it combines the media into a new film. Music can also be added if required.

In the online version, you will find additional functions supported by artificial intelligence. These include a text-to-speech module and a presentation coach that provides real-time feedback on camera recordings.

Finally, the Teams video conferencing software has received AI support: Users can ask Copilot questions during a conference or also use it as a writing assistant that automatically formulates chat messages after a few keywords have been entered.

You can also instruct the AI to summarize the content from chats and channels for you. The AI also translates foreign-language posts, which are displayed as subtitles in real time: However, “Live Captions” only runs on Copilot Plus notebooks.

Microsoft waits for Recall

When Microsoft announced the new AI function Recall in early summer, data protectionists expressed concerns.

Microsoft

Recall is supposed to take and save a screenshot every five seconds in the background. Using a timeline, users will then be able to see what they have been doing on their PC on a particular day and at a particular time.

With the help of artificial intelligence, the information in the images was to be analyzed and made available for questions such as “Where did I last read something about Copilot?”.

However, following protests, Microsoft has now put the project on hold for the time being. The function has now been implemented in the Insider pre-release versions.

However, the function is not expected to be integrated into normal Windows before spring 2025 — and if it is, then presumably as an additional app to be installed, as with Copilot.

The circle of potential users is also very limited, at least initially, because Recall requires a Copilot Plus PC.