The Verge Writer Tests Local AI on a 256GB Mac Studio and Finds Promise, Friction

The writer said he had been hesitant to use AI regularly because he did not want to give personal data to cloud services. Running powerful models locally is increasingly possible, he wrote, and he wanted to see whether the technology is useful enough to justify spending large amounts of money on a computer or laptop with expensive RAM. He said he is not an AI expert and plans to post about the journey.

The account points to a broader hardware push. Apple is pitching its new Mac desktops partly on how well they handle local AI, and a line of RTX Spark Windows machines is due imminently with up to 128GB of RAM aimed at agentic AI. The writer said the idea of having an assistant that lives only on his desk and answers only to him appeals more than trusting cloud services from OpenAI, Google, Microsoft or Anthropic. He added that he still would not use it to write or edit for him or generate fliers.

As part of testing an M5 Ultra Mac Studio, and experimenting on other Mac and Windows systems, he installed local LLMs. He began with Hermes Agent, an open-source, self-hosted AI agent desktop app that runs on macOS, Windows and Linux and is free to use with local LLMs. The M5 Ultra Mac Studio has 256GB of unified memory, which allowed him to run almost any model.

He described the model landscape as immediately overwhelming. There are more models than he could count, some with specialized uses, and models with many billions of parameters generally can do more than smaller ones with only a few billion. Because local models do not incur token charges, he decided to start big. Using Hermes’ onboarding interface and model picker, he chose Qwen 3.8 Flash Next, a 125-billion-parameter model about 105GB in size. He also planned to spend time with smaller Qwen models on devices including an M6 Mac Mini, an M5 MacBook Air, an Asus TUF Gaming A14 with AMD Strix Halo processor and, eventually, RTX Spark.

Hermes and Qwen were not long in getting up and running, and he made them controllable from his phone with a Telegram bot. Then he hit what he called his usual nemesis with AI: the text box. “What do I do with this thing?” he wrote.

Following advice from YouTube, he first had Hermes create a daily morning briefing through a cron job. It scans his email and calendar, flags anything that needs immediate attention and gives a short weather report. He said the briefing is not all that useful and does not require a $12,000 computer, but it is something he can develop. The brief initially failed to generate, until he realized macOS could not be asleep when the job fired at 7:30 a.m. It now works reliably, though he needs to add more information for it to pull.

A more useful task was reorganizing his Steam library. He has more than 400 games and had recently started categorizing them manually because Steam does not do it automatically. He asked Hermes, “Can you reorganize my Steam library?” Hermes could see most of his games by viewing the installed Steam client and quickly offered organization options. He chose to organize games by genre while leaving his own specialized categories, such as favorites, co-op titles, party games and games to play with his wife.

The task required giving the local AI permissions over his machine. For the Steam library, he registered a Steam web API key and gave it to Hermes. With that access, Hermes sorted all the games into categories in a few minutes. He said the API key was easy to revoke afterward because he did not need it further.

The writer said he plans to continue testing local models across other devices, including smaller Macs and Windows machines, as part of his ongoing effort to see what local AI can usefully do.