If you missed the memo, AI agents are the next rung of the ladder to AGI and something all AI companies are betting on to finally bring an ROI on the billions they're pouring into the technology.

What are AI Agents, the next rung on the ladder to AGI
The next big thing in AI is closer than you think
All Things HowSakshi Garg

Desktop apps have a huge role to play in realizing this Agentic AI dream. Ultimately, users would want their AI companions to be able to perform and automate some of their tasks; desktop apps can handle this better than a browser tab. For OpenAI, the path to that future starts with their desktop app being able to read data from your desktop apps. The company says that they'll have more to say on ChatGPT performing tasks for you in apps in 2025.

The company introduced the 'Work with apps' feature in ChatGPT last month where it could access a very limited set of apps on macOS. Now, as part of their 12 Days of OpenAI, they have shipped an update to this feature that expands the roster of apps available.

Initially, ChatGPT only worked with coding apps, and that included only a handful of editors, IDEs, and terminals. But with the update now rolling to the ChatGPT macOS desktop app, not only has the list of IDEs expanded, but the chatbot can now also work with text-based apps.

Here's a full list of apps that ChatGPT can now access, including the original ones:

  • Text editors:
    • Apple Notes
    • Notion
    • TextEdit
    • Quip
  • Code editors:
    • Xcode
    • VS Code (including Code, Code Insiders, VSCodium, Cursor, Windsurf)
    • Jetbrains (including Android Studio, IntelliJ, PyCharm, WebStorm, PHPStorm, CLion, Rider, RubyMine, AppCode, GoLand, DataGrip)
    • TextEdit
    • BBEdit
  • Terminals:
    • Terminal
    • iTerm
    • Warp
    • Prompt

The feature is available for Plus, Pro, Teams, Enterprise, and Edu users. Enterprise and Edu users will need admin permission to use ChatGPT with apps.

Unfortunately, the update is only available for the macOS desktop app yet, with no word on when it will roll out to the Windows desktop app.

How does it work?

ChatGPT would never access an app without your permission. ChatGPT uses macOS's Accessibility API to make this feature work. So, you need to provide permission to ChatGPT to even be able to access the app in the first place.

Even when ChatGPT has access to the app, it can only view the contents of the app after you explicitly select it.

To use an app with ChatGPT, it should be actively running on your desktop.

Once an app is running, you can click the 'Work with Apps' button from the ChatGPT Prompt bar and select the app. Apps can also be selected using the Option + Shift + 1 keyboard shortcut; the topmost app on your desktop will be selected.

Source: OpenAI

Once you select an app, the context of the app is sent to ChatGPT along with your text prompt. You can also select anything in the app and ChatGPT will focus on the selected text/ code; full context will still be sent to ChatGPT. You can see the content being sent to ChatGPT from the prompt bar.

Source: OpenAI

Since ChatGPT can now automatically view the content of your screen, this means you don't have to copy-paste the code or text to send it to ChatGPT. For text editors and code editors, the full context of the foremost window (up to a truncation limit) is sent to ChatGPT. However, for terminals, only the last 200 lines of the open pane are sent.

However, ChatGPT does not have the ability to write the output in the selected app. You'll still need to manually copy-paste the output from ChatGPT into your text or code editors if you want to make any changes.

The OpenAI team demoed the feature with two apps – XCode and Notion. When the concerned app was selected, ChatGPT could see the contents of the app. Note that ChatGPT can see the entire document (or as much as the truncation limit allows) and not just what's visible on the screen presently.

Along with additional apps, the latest update also brings support for Advanced Voice mode when using apps with ChatGPT. With Advanced Voice mode, you can talk to ChatGPT about the contents on your screen.

With app integration on macOS, ChatGPT gets one step further to becoming an integral part of your workflow. OpenAI also establishes itself firmly as competitors like Anthropic and Google are already making huge strides in agentic computer use.