속도와 메모리를 위한 LM Studio 최적화

LM Studio is the most user-friendly way to run local models — download a model, click Load, and you are chatting. Under the hood it wraps llama.cpp (and MLX on Apple Silicon), and exposes configuration options that control how it uses your hardware. The defaults work for casual use but leave significant performance on the table. This guide covers every knob that matters and how to set it for your workload.

Last verified: 2026-10-14 — LM Studio v0.3.26 / Bionic 1.1.8.

Apple Silicon uses MLX. On Apple Silicon Macs, LM Studio automatically uses Apple’s MLX framework rather than llama.cpp. MLX delivers substantially faster inference on M-series chips (see Apple Silicon and MLX optimization). Most configuration options in this guide apply to both backends; where behavior differs, it is called out explicitly.

Context window tuning

The context window controls how many tokens the model can process in a single request — your prompt plus the generated response combined. It is the single biggest driver of memory usage after the model weights themselves.

How to increase the context window in LM Studio

LM Studio defaults to a 2048-token context window for compatibility. Most modern models support 8K, 32K, or even 128K tokens, but LM Studio does not use the model’s maximum by default — it uses 2048 to keep memory usage predictable. Set it in 3 ways: In the UI (model load panel): When you load a model, the load panel shows a Context Length slider. Drag it to your desired value (e.g., 8192, 16384, 32768) before clicking Load Model. Per model (persistent default): Set a default context length for a specific model so it applies every time you load it. Go to the model in your library, click the gear icon, and set Context Length under Per-model Defaults. See Per-model Defaults for details. Via the API (per request):

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "Summarize this document..."}],
    "context_length": 32768
  }'

Via the CLI:

lms load llama-3.1-8b-instruct --context-length 32768

Context window memory cost

The KV cache grows linearly with context length. The formula:

KV cache memory = 2 × num_layers × hidden_dim × num_ctx × precision_bytes

For a 7B model (32 layers, 4096 hidden dimension) at f16 precision:

Context length	KV cache memory
2,048 (default)	~1 GB
8,192	~4 GB
32,768	~16 GB
128,000	~64 GB

Only set the context length as high as your workload actually needs. A coding agent that sends 2K prompts does not benefit from a 128K context window — it just wastes 64 GB of KV cache space that could serve other requests or fit a larger model.

Flash attention

Flash attention is the closest thing to a free performance improvement LM Studio offers. It reworks how the model computes attention between tokens — instead of materializing the full attention matrix in memory, it computes attention in tiled chunks that fit in fast SRAM. The result:

30-50% lower memory usage for the KV cache — the biggest win at long context lengths.
Faster inference — especially on GPU, where the tiled computation maps well to tensor cores.
Same output quality — flash attention is mathematically equivalent to standard attention, not an approximation.

How to enable flash attention in LM Studio

In the UI: When you load a model, the load panel shows a Flash Attention toggle. Enable it before clicking Load Model. Per model (persistent default): Set a default for a specific model so it applies every time you load it. Go to the model in your library, click the gear icon, and enable Flash Attention under Per-model Defaults. Via the API:

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "Hello"}],
    "flash_attention": true
  }'

Always enable flash attention. There is no reason to leave it off unless you hit a compatibility issue with a specific model or platform. It is the single highest-impact setting in this guide.

On Apple Silicon with the MLX backend, flash attention is enabled by default and the setting has no effect — MLX always uses optimized attention kernels.

GPU offload strategy

LM Studio automatically offloads model layers to your GPU. When the full model fits in VRAM, every layer runs on the GPU for maximum speed. When VRAM is tight, LM Studio splits layers between GPU and CPU — the GPU runs what it can, and the rest falls back to system RAM.

How LM Studio decides GPU layers

LM Studio’s GPU scheduling works in this order:

Full offload — if the model weights + KV cache fit in VRAM, all layers go to the GPU. This is the fastest configuration.
Partial offload — if the model does not fit entirely, LM Studio puts as many layers on the GPU as VRAM allows and runs the rest on CPU. Performance degrades because every forward pass must cross the PCIe bus for CPU-resident layers.
CPU-only — if no GPU is available or VRAM is too small for even a few layers, the entire model runs on CPU.

Control GPU offload

The GPU Offload setting controls how many layers to offload to the GPU. Set it in the UI, per-model defaults, or via the API: In the UI: When you load a model, the load panel shows a GPU Offload slider. You can set it to:

Auto (default) — LM Studio automatically detects available VRAM and offloads as many layers as fit.
Max — forces all layers to the GPU. Use this when you know the model fits.
Off — disables GPU offload entirely (CPU-only inference).
Manual — drag the slider to set a specific number of layers.

Per model (persistent default): Set a default for a specific model. Go to the model in your library, click the gear icon, and set GPU Offload under Per-model Defaults. Via the API:

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "Hello"}],
    "gpu": "max"
  }'

`gpu` value	Behavior
`"auto"` (default)	LM Studio auto-detects based on available VRAM
`"max"`	All layers on GPU — use when you know the model fits
`"off"`	No GPU offload — CPU only
`0.0` to `1.0`	Fraction of layers on GPU (e.g., `0.5` = 50% of layers)

Leave GPU offload on Auto and let LM Studio decide in most cases. Only set it manually when you need to force a specific split — for example, to reserve VRAM for a larger context window or to keep some layers on CPU for a model that barely fits.

VRAM budgeting

Each offloaded layer consumes VRAM for both weights and KV cache. A rough estimate:

VRAM per layer ≈ (parameters_per_layer × bytes_per_weight) + (KV_cache_per_layer × num_ctx)

For a 7B model at Q4_K_M (~4.5 bits per weight, ~32 layers):

Component	Per layer	32 layers total
Weights	~0.19 GB	~6 GB
KV cache (f16, 8K ctx)	~0.13 GB	~4 GB
Total		~10 GB

Leave 1-2 GB of VRAM headroom for CUDA context and overhead.

When you increase the context length, the KV cache grows and may push layers out of VRAM. If you set a 32K context window and LM Studio suddenly offloads half the layers to CPU, the KV cache ate your VRAM budget. Either reduce the context length, use a smaller quantization, or enable flash attention to shrink the KV cache.

KV cache precision

The KV cache stores key and value tensors for every token in every active sequence. By default, LM Studio uses f16 (16-bit floating point). You can switch to quantized precision to cut memory usage at a small quality cost. Set the cache type in the load panel or via the API: In the UI: When you load a model, expand Advanced Settings and look for KV Cache Quantization. You can set it to:

f16 (default) — full precision
q8_0 — 8-bit quantization
q4_0 — 4-bit quantization

Via the API:

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "Hello"}],
    "llama_k_cache_quantization_type": "q8_0",
    "llama_v_cache_quantization_type": "q8_0"
  }'

Cache type	Memory per element	Quality	When to use
`f16` (default)	2 bytes	Baseline	Maximum quality; you have plenty of memory
`q8_0`	1 byte	Near-identical to f16	Best balance — halves KV cache memory with negligible quality loss
`q4_0`	0.5 bytes	Slight degradation	Tight memory; long context or many concurrent sequences

Combine q8_0 KV cache with flash attention for the biggest memory win. Flash attention reduces the KV cache footprint by 30-50%, and q8_0 halves it again — together they can cut KV cache memory by 60-75% compared to the default.

q4_0 KV cache introduces measurable quality loss on tasks that rely on precise attention over long context — needle-in-a-haystack retrieval, multi-document summarization, and long conversations. Test your specific workload before committing to q4_0.

Batch size tuning

The Eval Batch Size parameter controls how many tokens LM Studio processes at once during the prefill phase (prompt ingestion). The default is 512. Set it in the load panel under Advanced Settings or via the API:

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "Hello"}],
    "eval_batch_size": 1024
  }'

Setting	Effect
`512` (default)	Balanced for interactive chat with short prompts
`1024`	Faster prefill for long prompts — roughly 50-100% faster time-to-first-token
`2048`	Maximum throughput for document ingestion — up to 150% faster prefill
`2048+`	Diminishing returns; higher memory usage without proportional speed gains

For interactive chat with short prompts (under 1K tokens), the default 512 is fine. Increase it only if you regularly ingest long documents and want faster time-to-first-token.

Larger batch sizes consume more VRAM during prompt ingestion. If you increase the batch size and hit out-of-memory errors, reduce it or lower the context length.

Model memory management

Keep models loaded with TTL

By default, LM Studio unloads a model from memory after a period of inactivity. The next request must reload the model from disk, which takes seconds to minutes depending on model size and disk speed. Set the TTL (Time To Live) to control how long models stay loaded: In the UI: When you load a model, the load panel shows a TTL dropdown. Set it to:

Until you close the app — model stays loaded indefinitely.
Until you unload it — model stays loaded until you manually unload it.
Custom duration — model unloads after the specified time (e.g., 5 minutes, 1 hour).

Via the API:

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "Hello"}],
    "ttl": -1
  }'

Via the CLI:

lms load llama-3.1-8b-instruct --ttl -1

Value	Behavior
Default (5 minutes)	Unload after 5 minutes of inactivity
`-1`	Keep loaded indefinitely — no reload penalty
`0`	Unload immediately after each request
Duration in seconds (`3600`, `86400`)	Keep loaded for that duration after the last request

For a dedicated inference server that runs the same model all day, set TTL to Until you close the app or -1. The model stays in memory and every request gets instant response with no cold-start penalty.

Multi-model serving

LM Studio can load multiple models simultaneously. When you load a second model, LM Studio checks if there is enough memory. If not, it unloads the least-recently-used model to make room. To manage loaded models: In the UI: The Models panel shows all loaded models. Click the unload button next to a model to free memory. Via the CLI:

# List loaded models
lms status

# Unload a specific model
lms unload llama-3.1-8b-instruct

# Unload all models
lms unload --all

Via the API:

# Unload a specific model
curl -X DELETE http://localhost:1234/v1/models/llama-3.1-8b-instruct

Local server configuration

LM Studio exposes an OpenAI-compatible API server. By default, it listens on http://localhost:1234.

Server settings

In the UI: Go to the Server tab and click Server Settings to configure:

Host — bind address (default: 127.0.0.1). Change to 0.0.0.0 to serve on the local network.
Port — listen port (default: 1234).
CORS — enable cross-origin requests for web clients.
Require API Key — require a bearer token for authentication.

Via the CLI:

lms server start --host 0.0.0.0 --port 8080

Setting the host to 0.0.0.0 exposes LM Studio to your local network. Only do this if you trust all devices on your network, or use a firewall to restrict access. For remote access over the internet, use Tokios instead — it provides authenticated remote access without opening ports.

API endpoints

LM Studio supports these OpenAI-compatible endpoints:

Endpoint	Purpose
`/v1/chat/completions`	Chat completions (most common)
`/v1/completions`	Text completions (legacy)
`/v1/embeddings`	Generate embeddings
`/v1/models`	List loaded models
`/v1/responses`	Responses API (newer format)
`/v1/decisions`	Decision models (classification, routing)

Example: list loaded models

curl http://localhost:1234/v1/models

Example: chat completion

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [
      {"role": "user", "content": "Hello!"}
    ]
  }'

Supported model formats

LM Studio supports 2 primary model formats:

GGUF (cross-platform)

GGUF is the most widely supported format. It works on Windows, Linux, and macOS — with or without a GPU. GGUF models are quantized to reduce file size (e.g., Q4_K_M, Q5_K_M, Q8_0). When to use GGUF:

You need cross-platform portability (Windows, Linux, Mac).
You want to run the same model on NVIDIA GPU and Apple Silicon.
You prefer the largest selection of quantization options.

MLX (Apple Silicon optimized)

MLX is Apple’s machine learning framework, optimized for M-series chips. On Apple Silicon, LM Studio automatically uses MLX for faster inference. When to use MLX:

You are on Apple Silicon (M1, M2, M3, M4).
You want maximum performance on Mac.
You do not need to run the same model on non-Apple hardware.

On Apple Silicon, LM Studio automatically selects the best runtime (MLX or llama.cpp) for your hardware. You do not need to configure this manually.

Apple Silicon and MLX optimization

On Apple Silicon Macs, LM Studio uses Apple’s MLX framework by default. MLX is optimized for Apple’s unified memory architecture and delivers substantially faster inference than llama.cpp on M-series chips.

Performance gains

Benchmarks from community testing:

Model	Quantization	Tokens/sec (MLX)
Llama 3.1 8B	Q4_K_M	~70-80 tok/s
Gemma 4 12B	Q4_K_M	~46 tok/s
Qwen 2.5 7B	Q4_K_M	~60-70 tok/s

MLX also supports advanced quantization on Apple Silicon, which delivers near-BF16 quality with 4-bit memory footprint — half the quality loss of Q4 quantization.

Apple Silicon optimization tips

Use MLX-optimized models. When you download a model on Apple Silicon, LM Studio automatically uses the MLX backend. No configuration needed.
Flash attention is always on. MLX uses optimized attention kernels by default. The flash attention setting has no effect.
Unified memory means full GPU offload. Apple Silicon’s GPU can access the full RAM pool, so full GPU offload works as long as the model and KV cache fit in unified memory.
Use q8_0 KV cache to stretch your memory budget. Halving the KV cache lets you run larger models or longer contexts.
Set TTL to “Until you close the app” for always-on inference. Models stay in unified memory with no reload penalty.

If you need cross-platform portability or want to run the same configuration on NVIDIA GPU and Mac, you can force LM Studio to use the llama.cpp backend on Apple Silicon. Contact LM Studio support or check their documentation for the setting to override the backend selection.

Example configurations

Mac Studio M2 Ultra — 128 GB unified memory

UI settings:

GPU Offload: Max
Context Length: 32768
Flash Attention: Enabled (auto on MLX)
KV Cache Quantization: q8_0
TTL: Until you close the app

This configuration runs a 70B Q4 model with a 32K context window. The 128 GB unified memory pool handles the ~40 GB model weights, ~16 GB KV cache at q8_0, and leaves headroom for multiple models. MLX delivers maximum throughput on Apple Silicon.

NVIDIA RTX 4090 — 24 GB VRAM

UI settings:

GPU Offload: Max
Context Length: 16384
Flash Attention: Enabled
KV Cache Quantization: q8_0
Eval Batch Size: 1024
TTL: Until you close the app

This configuration runs an 8B Q4 model with a 16K context window. The 24 GB VRAM handles the ~5 GB model weights, ~4 GB KV cache at q8_0 with flash attention. The larger batch size accelerates prompt ingestion.

Multi-GPU server — 2× RTX 4090 (48 GB total VRAM)

UI settings:

GPU Offload: Max
Context Length: 32768
Flash Attention: Enabled
KV Cache Quantization: q8_0
Eval Batch Size: 2048
TTL: Until you close the app

This configuration runs a 70B Q4 model across 2 GPUs. The 48 GB total VRAM handles the ~40 GB model weights, ~16 GB KV cache. The large batch size maximizes throughput for batch workloads.

CPU-only server — 64 GB RAM, no GPU

UI settings:

GPU Offload: Off
Context Length: 8192
Flash Attention: Disabled
KV Cache Quantization: q8_0
Eval Batch Size: 512
TTL: Until you close the app

This configuration runs an 8B Q4 model entirely on CPU. The small batch size and single parallel request keep memory usage manageable. Expect 5-10 tok/s — usable for interactive chat but not batch workloads.

Common mistakes and how to avoid them

Mistake 1: Setting context length too high

Problem: You set the context length to 128K “just in case,” but your prompts are under 2K tokens. You just wasted 64 GB of KV cache memory. Fix: Set the context length to match your actual workload. If your prompts are under 4K tokens, use 4096 or 8192. You can always increase it later for specific requests via the API.

Mistake 2: Forgetting flash attention

Problem: You are using 30-50% more VRAM than necessary because flash attention is not enabled. Fix: Always enable flash attention. There is no downside on supported hardware.

Mistake 3: Not controlling model unload

Problem: You restart your request after a 10-minute break, and the first request takes 30 seconds because LM Studio reloaded the model from disk. Fix: Set TTL to Until you close the app or -1 for always-on inference. The model stays in memory and every request gets instant response.

Mistake 4: Ignoring KV cache precision

Problem: You are running out of VRAM and LM Studio offloads layers to CPU, but you never considered that the KV cache is using 2× more memory than necessary. Fix: Set KV cache quantization to q8_0 to halve KV cache memory. This is the best balance of quality and memory savings.

Mistake 5: Using GPU offload on a model that does not fit

Problem: You set GPU offload to Max, but the model does not fit in VRAM. LM Studio runs out of memory or crashes. Fix: Set GPU offload to Auto and let LM Studio decide. If you must set it manually, start with a lower value and increase until you hit the limit.

Troubleshooting

LM Studio is slow after startup

Cause: The model was unloaded and had to reload from disk. Fix: Set TTL to Until you close the app or -1 to keep the model loaded indefinitely.

Out of memory errors

Cause: The model + KV cache exceeds available VRAM or RAM. Fix: Reduce the context length, enable flash attention, set KV cache quantization to q8_0, or use a smaller quantization.

GPU not being used

Cause: LM Studio did not detect the GPU, or VRAM is too small for even a few layers. Fix: Check the load panel to see if layers are on GPU. For NVIDIA GPUs, ensure the NVIDIA drivers and CUDA toolkit are installed. For AMD GPUs, ensure ROCm is installed.

Slow CPU inference

Cause: Layers split between GPU and CPU, or no GPU offload. Fix: If possible, offload more layers to GPU. Use a smaller quantization (Q4_K_M or Q5_K_M) to fit more layers on GPU.

Model fails to load

Cause: Not enough memory for the model + KV cache. Fix: Reduce the context length, reduce GPU offload, or use a smaller quantization. If the model is too large for your hardware, download a smaller model.

Next steps

LM Studio remote access

Expose your LM Studio model to teammates and agents through Tokios — without opening ports or serving on the local network.

Hardware reference

Compare hardware platforms for local LLM inference — specs, model capacity, and recommendations.

Model fit finder

Find which models run on your hardware, or which hardware can run your target model.

Test your deployment

Measure latency and throughput after tuning to confirm the improvement.

FAQ

What is the best GPU offload setting?

Use Auto in most cases. LM Studio detects available VRAM and offloads as many layers as fit. Set it to Max only when you know the full model fits in VRAM. Set it to Off for CPU-only inference.

How do I increase the context window?

In the load panel, drag the Context Length slider to your desired value (e.g., 8192, 16384, 32768). You can also set a per-model default in the model’s settings, or pass context_length in the API request.

Does LM Studio support flash attention?

Yes. Enable it in the load panel or per-model defaults. Flash attention reduces KV cache memory by 30-50% and speeds up inference. On Apple Silicon with MLX, flash attention is always on.

How do I keep the model loaded?

Set the TTL to Until you close the app or -1 via the API. The model stays in memory and every request gets instant response with no reload penalty.

Can I run multiple models at once?

Yes. LM Studio can load multiple models simultaneously. When memory is tight, it unloads the least-recently-used model to make room. Use lms status to see loaded models and lms unload to free memory.

What model format should I use?

Use GGUF for cross-platform compatibility (Windows, Linux, Mac). On Apple Silicon, LM Studio automatically uses MLX for faster inference. You do not need to choose manually.

How do I serve LM Studio on the local network?

In the Server tab, click Server Settings and change the Host to 0.0.0.0. This exposes LM Studio to your local network. For remote access over the internet, use Tokios instead.

What is the best KV cache quantization?

Use q8_0 for the best balance of quality and memory savings. It halves KV cache memory with negligible quality loss. Use q4_0 only when memory is extremely tight and you accept some quality degradation.

Sources

LM Studio documentation — official documentation on configuration and API
LM Studio per-model defaults — setting persistent configuration per model
LM Studio CLI reference — CLI commands for loading, unloading, and server management
LM Studio OpenAI compatibility — API endpoints and client integration
LM Studio changelog — release notes and new features
GGUF vs MLX on Apple Silicon — performance comparison and format selection
LM Studio GPU offload guide — GPU offload and context size configuration

After changing LM Studio configuration, re-register your Tokios deployment so the connector picks up the new upstream endpoint. See Register Model.

​Context window tuning

​How to increase the context window in LM Studio

​Context window memory cost

​Flash attention

​How to enable flash attention in LM Studio

​GPU offload strategy

​How LM Studio decides GPU layers

​Control GPU offload

​VRAM budgeting

​KV cache precision

​Batch size tuning

​Model memory management

​Keep models loaded with TTL

​Multi-model serving

​Local server configuration

​Server settings

​API endpoints

​Supported model formats

​GGUF (cross-platform)

​MLX (Apple Silicon optimized)

​Apple Silicon and MLX optimization

​Performance gains

​Apple Silicon optimization tips

​Example configurations

​Mac Studio M2 Ultra — 128 GB unified memory

​NVIDIA RTX 4090 — 24 GB VRAM

​Multi-GPU server — 2× RTX 4090 (48 GB total VRAM)

​CPU-only server — 64 GB RAM, no GPU

​Common mistakes and how to avoid them

​Mistake 1: Setting context length too high

​Mistake 2: Forgetting flash attention

​Mistake 3: Not controlling model unload

​Mistake 4: Ignoring KV cache precision

​Mistake 5: Using GPU offload on a model that does not fit

​Troubleshooting

​LM Studio is slow after startup

​Out of memory errors

​GPU not being used

​Slow CPU inference

​Model fails to load

​Next steps

LM Studio remote access

Hardware reference

Model fit finder

Test your deployment

​FAQ

​What is the best GPU offload setting?

​How do I increase the context window?

​Does LM Studio support flash attention?

​How do I keep the model loaded?

​Can I run multiple models at once?

​What model format should I use?

​How do I serve LM Studio on the local network?

​What is the best KV cache quantization?

​Sources

Context window tuning

How to increase the context window in LM Studio

Context window memory cost

Flash attention

How to enable flash attention in LM Studio

GPU offload strategy

How LM Studio decides GPU layers

Control GPU offload

VRAM budgeting

KV cache precision

Batch size tuning

Model memory management

Keep models loaded with TTL

Multi-model serving

Local server configuration

Server settings

API endpoints

Supported model formats

GGUF (cross-platform)

MLX (Apple Silicon optimized)

Apple Silicon and MLX optimization

Performance gains

Apple Silicon optimization tips

Example configurations

Mac Studio M2 Ultra — 128 GB unified memory

NVIDIA RTX 4090 — 24 GB VRAM

Multi-GPU server — 2× RTX 4090 (48 GB total VRAM)

CPU-only server — 64 GB RAM, no GPU

Common mistakes and how to avoid them

Mistake 1: Setting context length too high

Mistake 2: Forgetting flash attention

Mistake 3: Not controlling model unload

Mistake 4: Ignoring KV cache precision

Mistake 5: Using GPU offload on a model that does not fit

Troubleshooting

LM Studio is slow after startup

Out of memory errors

GPU not being used

Slow CPU inference

Model fails to load

Next steps

FAQ

What is the best GPU offload setting?

How do I increase the context window?

Does LM Studio support flash attention?

How do I keep the model loaded?

Can I run multiple models at once?

What model format should I use?

How do I serve LM Studio on the local network?

What is the best KV cache quantization?

Sources