GPU or unified memory?
By the end of this guide you will be able to decide whether a discrete GPU or a unified memory machine fits the models you actually want to run, and you will know which questions to answer before you spend anything. It is written for people buying their first machine for local AI, and for people deciding whether to add a card to a desktop they already own instead of replacing a laptop. It is one of the how-tos on the guides page.
What this assumes
The choice comes down to three things that pull in different directions: how much model you can hold, how fast you can move it, and how well the software you want to use is supported.
The background this leans on:
- Which workloads matter to you. Text generation, image generation, video and audio all behave differently, and local LLMs, image, video and audio are each worth a look to see which models you are drawn to.
- What "memory" means in each design. A discrete GPU has its own VRAM soldered to the board, fixed at purchase. A chip with unified memory gives the CPU and GPU one shared pool, so a large configuration can hold a large model even though it is not a graphics card.
- Why memory bandwidth is the other half of the story. Generation speed tracks how quickly weights can be read, not how clever the chip is.
- How quantization shrinks a model so that a given number of parameters fits in less memory. See picking a quantization.
- The general shape of the market, which is covered in the hardware section.
Steps
- Write down the two or three models you most want to run, and the largest one you would like to grow into. Hardware decisions age badly when they are made against today's model only.
- Turn each of those into a memory figure. Work through will this model fit in memory? so you have a number for weights plus the KV cache at the context window you plan to use.
- Set your capacity floor. Anything that cannot hold your target model in memory is out, because spilling to system RAM or disk collapses speed regardless of how fast the chip is. This is the single hardest constraint.
- Now rank the survivors by bandwidth. Among machines that all fit your model, the one that reads memory fastest will generate fastest. This is where discrete cards usually pull ahead of shared memory designs, and where a big unified memory machine can feel slower than its capacity suggests.
- Check the software you intend to use. Apps differ in how well they support each backend, so read LLM runtimes, image runtimes and audio models with your shortlist in hand. A machine that runs your chosen tool poorly is a bad machine for you even if the hardware is strong.
- Weigh the practical factors: noise, power draw, whether you need it to be portable, and whether you want to add a second card later. A desktop tower gives you upgrade room that a sealed laptop does not.
- Compare your final two directly. M3 Max vs RTX 4090 is the clearest example of the capacity against bandwidth trade, and the M3 Max page covers what shared memory means in daily use.
- Buy the machine that fits your largest realistic model, not the fastest machine that fits your smallest one.
Common mistakes
- Comparing total memory across designs as if the numbers mean the same thing. Shared memory is also holding the operating system and everything else you have open, so not all of it is available to a model.
- Optimising for generation speed while ignoring capacity. A model that does not fit runs at a speed no benchmark will rescue.
- Assuming more memory always means faster. Capacity and bandwidth are separate specifications and large configurations are not automatically high bandwidth.
- Forgetting the software question until after the box arrives. Tooling support varies by backend and it is the thing you cannot fix with money.
- Buying for image or video work based on text generation advice. Those workloads have different memory profiles, so check the image and video pages.