Key takeaways
- System requirements list the hardware and software a program needs; for AI, memory dominates.
- Check RAM, GPU and VRAM in Settings, Task Manager, About This Mac or nvidia-smi.
- VRAM is roughly weights plus KV cache plus overhead, and long contexts add up fast.
- 8 GB of VRAM runs 7 to 8 billion parameter models at 4 bits and small QLoRA jobs.
- Offload all layers that fit with n-gpu-layers, and move models and data to the GPU for training.
System requirements are the minimum and recommended hardware and software a program needs to run, and for AI work they mostly come down to GPU memory, system RAM and storage. Local language models, image generators and training jobs live or die by VRAM. I fine-tune models with QLoRA in FineTune Studio and run models locally, so this page covers system requirements, checking your laptop, SRS documents, local LLM hardware, the VRAM formula, 8 GB GPUs, GPU layer offloading, GPU training and laptops for ML.
What are system requirements?
System requirements list the hardware and software a program needs: processor, memory, graphics card and its VRAM, storage space, operating system version and sometimes drivers or internet access. Minimum requirements let it run; recommended requirements let it run well.
Also asked as: what is system requirements
For games and creative software, requirements focus on CPU and GPU performance. For AI tools, the critical numbers are GPU memory, system RAM and fast storage, since models must fit in memory to run at usable speed. Requirements also include software dependencies, such as CUDA versions for NVIDIA GPUs or specific Python versions. Always compare requirements with your actual hardware, and leave headroom, since background apps and other processes consume resources too.
How do you check system requirements on a laptop?
On Windows, open Settings, System, About for processor and RAM, and Task Manager's Performance tab for GPU model and dedicated GPU memory. On a Mac, choose About This Mac for chip, memory and macOS version.
Also asked as: how to check system requirements in laptop
For more detail on Windows, run dxdiag for graphics information, or use nvidia-smi in a terminal on NVIDIA systems to see VRAM and driver versions. Check free storage in File Explorer or Finder. Then compare with the software's published minimum and recommended requirements. For AI tools, note whether your GPU is NVIDIA, AMD, Intel or Apple silicon, since software support differs, with NVIDIA CUDA having the broadest compatibility.
What is a system requirements specification?
A system or software requirements specification, often called an SRS, is a document describing what a system must do and the constraints it must meet: functional requirements, non-functional requirements such as performance and security, interfaces and assumptions.
Also asked as: what is system requirements specification
Standards such as ISO/IEC/IEEE 29148 guide how to write them. A good SRS uses clear, testable statements, such as the system shall respond within two seconds for 95 percent of requests, and covers users, use cases, data, integrations and acceptance criteria. Students write SRS documents in software engineering courses and final-year projects. In AI products, include model accuracy targets, latency, data privacy rules and fallback behaviour when the model is unsure.
What hardware do you need to run a local LLM?
The key requirement is memory: enough GPU VRAM, or unified memory on Apple silicon, to hold the model weights plus the KV cache. A 7 to 8 billion parameter model at 4-bit quantization needs roughly 5 to 6 GB.
Also asked as: what is local llm hardware requirement
Rough tiers: 8 GB of VRAM runs 7 to 8 billion parameter models comfortably at 4 bits; 12 to 16 GB handles larger 12 to 14 billion parameter models; 24 GB fits around 30 billion parameters at 4 bits; and bigger models need multiple GPUs or high-memory Macs. CPU-only inference works with enough system RAM but is slower. Fast SSD storage speeds model loading, and 32 GB of system RAM gives comfortable headroom.
What is the formula for LLM VRAM usage?
VRAM is roughly weights plus KV cache plus overhead. Weights equal parameters times bytes per parameter: two bytes at 16-bit, one at 8-bit, about half a byte at 4-bit. Add 10 to 20 percent overhead for runtime buffers.
Also asked as: what is the complete formula to get llm vram usage
The KV cache equals two times layers times KV heads times head dimension times context length times bytes per value, times batch size. For a Llama 3 8B model, with 32 layers, 8 KV heads and a head dimension of 128 at 16-bit, that is about 128 KB per token, or about 1 GB for an 8,000-token context. Long contexts can therefore need as much memory as the weights, so quantized KV caches help.
What really fits in 8 GB of VRAM?
With 8 GB, you can comfortably run 7 to 8 billion parameter LLMs at 4-bit quantization with moderate context, Whisper large for speech recognition, and Stable Diffusion image generation with memory optimisations.
Also asked as: what really fits in 8gb vram
Larger models, around 12 to 14 billion parameters, may squeeze in at 3-bit quantization with short contexts, or run with partial CPU offloading at lower speed. QLoRA fine-tuning of small models, up to about 7 billion parameters with short sequences and small batches, is possible with memory-efficient tools such as Unsloth. Leave about one gigabyte free for the display and runtime, and watch context length, since the KV cache grows with every token.
Why does running a local LLM use so much VRAM?
Every model parameter must be held in memory during inference, and models have billions of them. On top of weights, the KV cache stores attention data for every token in your context, growing as conversations get longer.
Unquantized 16-bit models use two bytes per parameter, so an 8 billion parameter model needs about 16 GB for weights alone. Runtimes also allocate buffers for computation, and some reserve memory upfront for the maximum context. To reduce usage: choose quantized models, such as 4-bit, lower the context length setting, use quantized KV caches if supported, close other GPU applications, and offload some layers to the CPU when needed.
How do you tune n-gpu-layers for your VRAM?
In llama.cpp, --n-gpu-layers, or -ngl, sets how many transformer layers run on the GPU; the rest run on the CPU. Set it as high as fits in VRAM, ideally all layers, while leaving room for the KV cache.
Also asked as: how to tune --n-gpu-layers for your vram budget
Start with a high value such as 99 to offload everything; if loading fails with out-of-memory errors, reduce it step by step until it loads with some free memory. The startup log reports how many layers were offloaded and how much memory was used, and nvidia-smi shows actual usage. Each layer moved to the CPU slows generation noticeably, so a smaller quant fully on the GPU often beats a larger quant split across devices.
How do you use a GPU for training instead of a CPU?
In PyTorch, install a CUDA-enabled build, check that torch.cuda.is_available() returns true, then move the model and every batch of data to the GPU with the to method. On Apple silicon, use the mps device instead.
Also asked as: how to use gpu for training instead of cpu
A common pattern selects cuda when it is available and cpu otherwise, then calls model.to(device) and moves inputs to the same device inside the training loop. Errors about tensors on different devices mean something was not moved. In TensorFlow, GPU use is automatic when a compatible build and drivers are installed. Hugging Face Trainer and Accelerate handle device placement for you. If the GPU is not detected, check drivers, CUDA versions and that you installed the GPU build of the framework.
What is the best laptop for a machine learning engineer?
Choose a laptop with an NVIDIA RTX GPU with at least 8 GB, ideally 12 to 16 GB, of VRAM, or an Apple silicon MacBook with 32 GB or more of unified memory, plus 32 GB of RAM and a fast 1 TB SSD.
Also asked as: best laptop for machine learning engineer
NVIDIA laptops give the broadest compatibility with CUDA-based tools for training and fine-tuning; Macs offer large unified memory, long battery life and good local inference through MLX and llama.cpp, but weaker support for some training libraries. Laptops thermal-throttle under long training runs, so many engineers use the laptop for development and cloud GPUs or a desktop for heavy training. Prioritise memory over raw speed, since running out of memory blocks work entirely.
Sources
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
