Performance and cost reference values
How does each AI computer perform?
The overview below assigns published performance figures and an indicative cost calculation to each configuration in the configurator. It is meant as guidance before a quote.
As of:
The figures are compiled from published third-party benchmarks and from manufacturer data. They are reference values, not a performance guarantee. Actual performance depends on the model, quantisation, software and workload. Figures marked “Approximation” come from similar hardware that is not identical, or are extrapolated.
Notes in the tables and in the source list follow the German compilation, so each figure stays tied to the same source wording.
Methodology and assumptions
The tables name the quantities by which a local AI computer can be judged in day-to-day use.
The RTX6000 option refers to the NVIDIA RTX PRO 6000 Blackwell with 96 GB of graphics memory.
- Generation (decode)
- How fast text is produced, in tokens per second. In German, one token is roughly three quarters of a word. The tiers below apply per user.
Each lower bound is inclusive. 20.0 tokens per second therefore count as usable, and 50.0 tokens per second as smooth. Generation per user Tier Assessment under 10 tok/s too slow Suitable only for batch and background tasks. 10–20 tok/s limited Adequate for chat and text work, sluggish for coding agents. 20–35 tok/s usable Usable for software development and agents. 35–50 tok/s comfortable Pleasant for agentic coding. over 50 tok/s smooth Comparable to cloud services. - Prompt processing (prefill)
- How quickly long inputs, for example documents or a chat history, are read in. The wait until the first token depends on this. For coding agents with a large context, prompt processing matters as much as generation. The reference value is at least 500 tokens per second, ideally 1,000 tokens per second or more, at realistic context lengths (for example 32,000 tokens) and with prefix caching. Below that, every step waits seconds to minutes for the first token.
- Single user and parallel requests
- The tiers and the column for generation per user apply to one request. If several people share a computer, speed per user falls. Total throughput can rise at the same time.
- Software
- vLLM and SGLang are servers for many parallel users. llama.cpp is widely used and also runs on AMD graphics cards and on the CPU. The table shows which software was used.
- Quantisation
- Compressed model weights, for example Q4_K_M at about 4 to 5 bits, FP8 at 8 bits, and NVFP4 or MXFP4 at 4 bits. Fewer bits means less memory and higher speed, with a slight drop in quality.
- Memory requirement
- Rule of thumb: number of parameters times bits, divided by 8, plus a reserve for the context (KV cache). Example: 120 billion parameters at 4 bits correspond to about 60 GB.
G4 without a GPU
G4 without a GPU (32-core EPYC, DDR4)
CPU-only inference for experiments. In the measurements on hand, the 8B class sits in the “usable” tier, so it is usable for software development and agents. Dense models from the 30B class fall below 10 tokens per second per user and are too slow for interactive work; they suit batch and background tasks. With 128 GB of memory, large models can be loaded as well. Theoretical memory bandwidth is 204.8 GB/s (eight channels of DDR4-3200, reference configuration 128 GB).
With 32 GB, not all memory channels are likely to be populated. Generation speed is then lower as well. The actual population is checked in the quote.
- Applies to
- G4, 32 GB / 128 GB, Keine GPU
- Memory
- 32 or 128 (system RAM)
- Memory bandwidth
- 204.8 GB/s Eight channels of DDR4-3200, theoretical, reference configuration 128 GB. At 32 GB, not all channels are likely to be populated; bandwidth is then lower.
Suitable models
| Model | Quantisation | Memory | Assessment |
|---|---|---|---|
| Ministral 3 8B / Llama 3.1 8B / Qwen3-8B | Q4_K_M (~5 GB) | 32 GB: ja | Stufe brauchbar |
| gpt-oss-20b | MXFP4 (~12 GB) | 32 GB: ja | MoE (3,6B aktiv) – auf CPU deutlich schneller als dichte Modelle gleicher Größe; kein CPU-Benchmark gefunden |
| Llama 3.3 70B | Q4_K_M (~40 GB) | nur 128 GB | ~3 tok/s, Stufe zu langsam |
| gpt-oss-120b | MXFP4 (~61 GB) | nur 128 GB | passt in RAM; kein passender CPU-Benchmark gefunden (Lücke) |
Published performance figures
| Model | Quantisation | Software | Generation tok/s, 1 user | Tested on | Note | Source |
|---|---|---|---|---|---|---|
| Llama 3.1 8B Instruct | Q4_K_M | llama.cpp (CPU, BLAS) | 22.7 usable. Usable for software development and agents. | 2x EPYC 7282 (2x16 Kerne, 2x8 Kanäle DDR4-3200, ~130 GB/s gemessen) | Approximation Andere CPU: zwei 16-Kern-Rome-CPUs statt einer 32-Kern-CPU; ähnliche Kernzahl und DDR4-3200, Bandbreite eher niedriger als ein voll bestückter 32-Kern-EPYC | ahelpme.com – llama.cpp CPU-Benchmarks 2x AMD EPYC 7282 (2x16 Kerne, 2x8 DDR4-3200) 1 April 2025 |
| QwQ-32B (≈ Qwen3-32B-Klasse) | Q4_K_M | llama.cpp (CPU, BLAS) | 5.8 too slow. Suitable only for batch and background tasks. | 2x EPYC 7282 | Approximation wie oben | ahelpme.com – llama.cpp CPU-Benchmarks 2x AMD EPYC 7282 (2x16 Kerne, 2x8 DDR4-3200) 1 April 2025 |
| Llama 3.3 70B Instruct | Q4_K_M | llama.cpp (CPU, BLAS) | 2.9 too slow. Suitable only for batch and background tasks. | 2x EPYC 7282 | Approximation wie oben; erfordert 128 GB RAM | ahelpme.com – llama.cpp CPU-Benchmarks 2x AMD EPYC 7282 (2x16 Kerne, 2x8 DDR4-3200) 1 April 2025 |
Power consumption
325 W Approximation
Estimate: CPU TDP 225 W (example EPYC 7543) plus about 100 W for the platform (memory, board, SSD, fans). No measured system power draw.
Cost per 1 million tokens
Reference: Llama 3.1 8B Instruct Q4_K_M. Hardware: €14,200 net. Indicative price from the configurator, without additional storage.
There is no batch benchmark for the CPU. Every scenario uses the single-stream figure. That is an upper bound on cost.
| Scenario | Euro per 1 million tokens |
|---|---|
| Single seat 8 hours × 220 working days, generating 30% of the time, one request 22.7 tok/s | €67.01 (€110.89) |
| Team 8 hours × 220 working days, 50% utilisation, 4–8 parallel requests. Main scenario. 22.7 tok/s | €40.68 (€67.01)* |
| Team, highly parallel 8 hours × 220 working days, 50% utilisation, batch from 32 requests 22.7 tok/s | €40.68 (€67.01)* |
| 24/7 batch 24 hours × 365 days, 90% utilisation, batch from 32 requests 22.7 tok/s | €5.60 (€8.54)* |
* No measured value is available for this level of concurrency. The calculation uses the next-lower measured throughput. Actual costs are then more likely to be lower.
€0.30/kWh net (commercial). Euro per 1 million generated tokens, net. Depreciation over 5 years, in parentheses over 3 years. Assumptions and method
Comparison with API prices
Per 1 million output tokens plus 3 million input tokens, a typical ratio of 3:1. List prices, converted at the ECB reference rate. In this calculation the local computer processes the prompt at no additional token price.
| API model | Input / output, USD per 1 million | Euro per 1 million output incl. 3 million input | Source |
|---|---|---|---|
| Ministral 3 8B (Mistral API) | 0.15 / 0.15 | €0.54 | Mistral Docs – API pricing |
| OpenAI gpt-6-luna | 0.10 / 0.50 | €0.72 | OpenAI – API Pricing (Standard, Short context) |
| Anthropic Claude Haiku 5.5 | 0.10 / 0.50 | €0.72 | Anthropic – Claude Pricing |
G5 without a GPU
G5 without a GPU (32-core EPYC Genoa, DDR5)
Clearly higher memory bandwidth than the G4: theoretically 460.8 GB/s with twelve channels of DDR5-4800 (reference configuration 192 GB). For text generation, bandwidth is the decisive factor. The advantage of the G5 platform shows up with 192 GB of memory. In the measurements on hand, the 8B class sits in the “comfortable” tier, and dense 30B-class models in the “limited” tier. Larger dense models are too slow.
With 32 GB, only some of the twelve memory channels can be populated. Bandwidth, and therefore generation speed, is then clearly lower. The actual population is checked in the quote.
MoE models such as gpt-oss-120b, Mistral Small 4 or Qwen3.5-122B-A10B fit in memory at 192 GB. There is no reliable measurement for a 32-core Genoa platform.
- Applies to
- G5, 32 GB / 192 GB, Keine GPU
- Memory
- 32 or 192 (system RAM)
- Memory bandwidth
- 460.8 GB/s Twelve channels of DDR5-4800, theoretical, reference configuration 192 GB. At 32 GB, not all channels can be populated; bandwidth is then clearly lower.
Suitable models
| Model | Quantisation | Memory | Assessment |
|---|---|---|---|
| 8B-Klasse (Ministral 3 8B, Qwen3-8B) | Q4_K_M (~5 GB) | 32 GB: ja | Stufe komfortabel |
| Qwen3.6-27B / Gemma 4 31B | Q4_K_M (~17–19 GB) | 32 GB: knapp; 192 GB: ja | dicht, Stufe eingeschränkt (~10–14 tok/s, Proxy QwQ-32B) |
| Llama 3.3 70B | Q4_K_M (~40 GB) | nur 192 GB | ~7 tok/s (Proxy), Stufe zu langsam |
| gpt-oss-120b / Mistral Small 4 119B / Qwen3.5-122B-A10B | MXFP4 / Q4 (~61–70 GB) | nur 192 GB | MoE-Modelle passen; kein verlässlicher Single-Socket-CPU-Benchmark gefunden (Lücke) |
Published performance figures
| Model | Quantisation | Software | Generation tok/s, 1 user | Tested on | Note | Source |
|---|---|---|---|---|---|---|
| Llama 3.1 8B Instruct | Q4_K_M | llama.cpp (CPU, BLAS) | 49.6 comfortable. Pleasant for agentic coding. | EPYC 9554 (64 Kerne, 12x DDR5-5600) | Approximation Doppelte Kernzahl und schnellerer RAM als die 32-Kern-G5-Plattform; Decode ist bandbreitenlimitiert, daher eher Obergrenze. Nur mit 192 GB (12 Kanäle) erreichbar. | ahelpme.com – llama.cpp CPU-Benchmarks AMD EPYC 9554 (64 Kerne, 12x DDR5-5600) 25 March 2025 |
| QwQ-32B (≈ Qwen3-32B-Klasse) | Q4_K_M | llama.cpp (CPU, BLAS) | 14 limited. Adequate for chat and text work, sluggish for coding agents. | EPYC 9554 | Approximation wie oben | ahelpme.com – llama.cpp CPU-Benchmarks AMD EPYC 9554 (64 Kerne, 12x DDR5-5600) 25 March 2025 |
| Llama 3.3 70B Instruct | Q4_K_M | llama.cpp (CPU, BLAS) | 7.1 too slow. Suitable only for batch and background tasks. | EPYC 9554 | Approximation wie oben; erfordert 192 GB | ahelpme.com – llama.cpp CPU-Benchmarks AMD EPYC 9554 (64 Kerne, 12x DDR5-5600) 25 March 2025 |
Power consumption
380 W Approximation
Estimate: CPU TDP 280 W (example EPYC 9354) plus about 100 W for the platform. No measured system power draw.
Cost per 1 million tokens
Reference: Llama 3.1 8B Instruct Q4_K_M. Hardware: €20,700 net. Indicative price from the configurator, without additional storage.
There is no batch benchmark for the CPU. Every scenario uses the single-stream figure. The comparison CPU has 64 cores rather than 32; the figure is more likely an upper bound on speed and therefore a lower bound on cost.
| Scenario | Euro per 1 million tokens |
|---|---|
| Single seat 8 hours × 220 working days, generating 30% of the time, one request 49.6 tok/s | €44.55 (€73.82) |
| Team 8 hours × 220 working days, 50% utilisation, 4–8 parallel requests. Main scenario. 49.6 tok/s | €26.99 (€44.55)* |
| Team, highly parallel 8 hours × 220 working days, 50% utilisation, batch from 32 requests 49.6 tok/s | €26.99 (€44.55)* |
| 24/7 batch 24 hours × 365 days, 90% utilisation, batch from 32 requests 49.6 tok/s | €3.58 (€5.54)* |
* No measured value is available for this level of concurrency. The calculation uses the next-lower measured throughput. Actual costs are then more likely to be lower.
€0.30/kWh net (commercial). Euro per 1 million generated tokens, net. Depreciation over 5 years, in parentheses over 3 years. Assumptions and method
Comparison with API prices
Per 1 million output tokens plus 3 million input tokens, a typical ratio of 3:1. List prices, converted at the ECB reference rate. In this calculation the local computer processes the prompt at no additional token price.
| API model | Input / output, USD per 1 million | Euro per 1 million output incl. 3 million input | Source |
|---|---|---|---|
| Ministral 3 8B (Mistral API) | 0.15 / 0.15 | €0.54 | Mistral Docs – API pricing |
| OpenAI gpt-6-luna | 0.10 / 0.50 | €0.72 | OpenAI – API Pricing (Standard, Short context) |
| Anthropic Claude Haiku 5.5 | 0.10 / 0.50 | €0.72 | Anthropic – Claude Pricing |
4× AMD RX 9070 XT
4× AMD Radeon RX 9070 XT (4× 16 GB = 64 GB VRAM)
Four graphics cards with 16 GB each, 640 GB/s per card and 304 W board power according to AMD. Smaller models run on one card at high speed. Several instances or groups of users can be spread across the four cards. Larger models are split across two to four cards.
RDNA4 support in vLLM was still immature in 2026. What runs reliably is mainly llama.cpp (ROCm or Vulkan). For multi-user serving with vLLM, only community builds are available so far.
Llama 3.3 70B in Q4 (about 40 GB) fits, on paper, on three to four cards; there is no measurement for that. gpt-oss-120b (about 61 GB) fits only very tightly and leaves little room for context. Mistral Small 4 (119B) and Qwen3.5-122B do not fit fully in graphics memory.
- Applies to
- G4 or G5, 32 GB / 128 GB / 192 GB, 4× AMD 9070XT
- Memory
- 64 , 16 GB per GPU
- Memory bandwidth
- 640 GB/s per GPU; 4× = 2,560 GB/s aggregate, usable only with tensor parallelism
- Power consumption
- 304W
Suitable models
| Model | Quantisation | Memory | Assessment |
|---|---|---|---|
| gpt-oss-20b | MXFP4 (~12 GB) | 1 GPU | bis zu 4 parallele Instanzen möglich |
| Ministral 3 8B/14B, Qwen3-8B/14B | Q4_K_M–Q8_0 | 1 GPU | – |
| Qwen3.6-35B-A3B | Q4–Q6 bzw. AWQ-INT4 (~19–28 GB) | 2 GPUs | gutes Preis-Leistungs-Modell für Teams |
| Gemma 4 26B-A4B / 31B, Qwen3.6-27B / Qwen3.8-27B, Mistral Small 3.2 24B | Q4–Q6 | 2 GPUs | – |
| Llama 3.3 70B | Q4_K_M (~40 GB) | 3–4 GPUs | passt rechnerisch, kein Benchmark für 4× 9070 XT gefunden (Lücke) |
| gpt-oss-120b | MXFP4 (~61 GB) | 4 GPUs, sehr knapp | kaum Platz für KV-Cache – nicht empfohlen |
| Mistral Small 4 119B, Qwen3.5-122B | Q4 (~67–70 GB) | passt nicht vollständig | nur mit CPU-Offload |
Published performance figures
| Model | Quantisation | Software | Context | Generation tok/s, 1 user | Parallel throughput | Prefill tok/s | Tested on | Note | Source |
|---|---|---|---|---|---|---|---|---|---|
| gpt-oss-20b | MXFP4 | llama.cpp ROCm 7.2 | kurz (llama-bench) | 148.5 smooth. Comparable to cloud services. | – | 5,529.9 | 1× RX 9070 XT | Vulkan: 180,8 tok/s Decode / 4.952 tok/s Prefill; bei 131k Kontext ROCm 70,5 tok/s | GitHub kr4ckhe4d/local-llm-benchmarks – llama.cpp auf RX 9070 XT / R9700 (ROCm 7.2) 6 October 2026 |
| Qwen3-8B | Q4_K_M | llama.cpp HIP | pp512/tg (llama-bench) | 98.8 smooth. Comparable to cloud services. | – | 4,245 | 1× RX 9070 XT | llama.cpp Issue #26663 – RX 9070 XT, HIP vs. Vulkan (Qwen3 Q4_K_M) 7 August 2026 | |
| Qwen3-14B (≈ Ministral 3 14B-Klasse) | Q4_K_M | llama.cpp HIP | pp512 (llama-bench) | 54.3 smooth. Comparable to cloud services. | – | 1,104 | 1× RX 9070 XT | Approximation Für Ministral 3 14B kein direkter Wert; Qwen3-14B gleicher Größe als Näherung | llama.cpp Issue #26663 – RX 9070 XT, HIP vs. Vulkan (Qwen3 Q4_K_M) 7 August 2026 |
| Qwen3.6-35B-A3B | AWQ-INT4 | vLLM (Community-Build, TP=2) | – | – | 957 (32 Requests) | – | RX 9070 XT + RX 9070 (TP=2) | Approximation Offline-Batch (32 Prompts × 128 Tokens) auf 2 GPUs, eine davon 9070 non-XT; nicht offizielles vLLM; gestreamte Parallel-Last lief laut README in OOM. Mit 4 GPUs wären 2 Instanzen denkbar (nicht gemessen). | GitHub patcarter883/rdna4-vllm – vLLM TP=2 auf RDNA4 (9070 XT + 9070) 8 October 2026 |
| Qwen3.6-35B-A3B | UD-Q6_K | llama.cpp ROCm | leer; Prefill 3.170 tok/s bei 128k Tiefe | 72 smooth. Comparable to cloud services. | – | – | RX 9070 XT + Radeon AI PRO R9700 (16 + 32 GB) | Approximation Zwei-GPU-System mit gleicher RDNA4-Architektur, aber einer 32-GB-Karte | GitHub kr4ckhe4d/local-llm-benchmarks – llama.cpp auf RX 9070 XT / R9700 (ROCm 7.2) 6 October 2026 |
| Gemma 4 26B-A4B | Q4 + MTP | llama.cpp ROCm | leer; 53 tok/s bei 128k gefülltem Kontext | 143 smooth. Comparable to cloud services. | – | – | RX 9070 XT + Radeon AI PRO R9700 | Approximation wie oben; mit spekulativem Decoding (MTP) | GitHub kr4ckhe4d/local-llm-benchmarks – llama.cpp auf RX 9070 XT / R9700 (ROCm 7.2) 6 October 2026 |
| Qwen3.8-27B (dicht) | Q4_K_M (UD) | llama.cpp ROCm 7.2 (Fork) | ~8,4k Prompt | 24.7 usable. Usable for software development and agents. | – | 2,061 | 2× RX 9070 XT (Layer-Split, Windows) | mit MTP-Spekulation 36,7 tok/s; unter Linux/ROCm 10 MXFP4 29,4 tok/s | GitHub MrLordCat/llama.cpp-rdna-lab – 2x RX 9070 XT, Qwen3.8-27B 26 September 2026 |
Power consumption
1,541 W Approximation
Upper bound, estimated: 4 × 304 W board power (AMD) plus 225 W CPU TDP plus about 100 W for the platform. With split models the cards work one after another; actual power draw is lower. No measured system power draw.
Cost per 1 million tokens
Reference: Qwen3.6-35B-A3B (AWQ-INT4 Batch / Q6_K Einzelstrom). Hardware: €18,100 net. Indicative price from the configurator, without additional storage.
There is no measurement for moderate parallelism (4–8 requests). The team scenario therefore uses the single-stream figure of 72 tok/s; the costs are more likely an upper bound. 957 tok/s come from an offline batch of 32 prompts on only two of the four cards (community build, approximation).
| Scenario | Euro per 1 million tokens |
|---|---|
| Single seat 8 hours × 220 working days, generating 30% of the time, one request 72 tok/s | €28.23 (€45.87) |
| Team 8 hours × 220 working days, 50% utilisation, 4–8 parallel requests. Main scenario. 72 tok/s | €17.65 (€28.23)* |
| Team, highly parallel 8 hours × 220 working days, 50% utilisation, batch from 32 requests 957 tok/s | €1.33 (€2.12) |
| 24/7 batch 24 hours × 365 days, 90% utilisation, batch from 32 requests 957 tok/s | €0.27 (€0.36) |
* No measured value is available for this level of concurrency. The calculation uses the next-lower measured throughput. Actual costs are then more likely to be lower.
€0.30/kWh net (commercial). Euro per 1 million generated tokens, net. Depreciation over 5 years, in parentheses over 3 years. Assumptions and method
Comparison with API prices
Per 1 million output tokens plus 3 million input tokens, a typical ratio of 3:1. List prices, converted at the ECB reference rate. In this calculation the local computer processes the prompt at no additional token price.
| API model | Input / output, USD per 1 million | Euro per 1 million output incl. 3 million input | Source |
|---|---|---|---|
| Qwen3.6-35B-A3B (OpenRouter) | 0.15 / 1.00 | €1.30 | OpenRouter – Models API (Listenpreise gehosteter Open-Weight-Modelle) |
| Mistral Small 4 (Mistral API) | 0.15 / 0.60 | €0.94 | Mistral Docs – API pricing |
| OpenAI gpt-6-luna | 0.10 / 0.50 | €0.72 | OpenAI – API Pricing (Standard, Short context) |
1× RTX PRO 6000
1× NVIDIA RTX PRO 6000 Blackwell (96 GB)
One NVIDIA RTX PRO 6000 Blackwell with 96 GB of graphics memory and 1,792 GB/s. Maximum power draw is 600 W (Workstation Edition) or 300 W (Max-Q). Independent lab measurements with vLLM exist for this card, including parallel requests.
The name RTX6000 in the configurator means the NVIDIA RTX PRO 6000 Blackwell with 96 GB. The figures in this profile apply to that card. The older RTX 6000 Ada (48 GB) is not what these figures describe.
With 96 GB of graphics memory, at least 128 GB of system memory is advisable. For multi-user serving, MoE models with few active parameters have the advantage.
Llama 3.3 70B fits tightly in FP8 and comfortably in Q4; a current measurement is not included in this compilation.
- Applies to
- G4 or G5, 32 GB / 128 GB / 192 GB, 1× NVIDIA RTX6000
- Memory
- 96
- Memory bandwidth
- 1,792 GB/s
- Power consumption
- 600 (Workstation Edition) bzw. 300 (Max-Q)
RTX6000 in the configurator means the NVIDIA RTX PRO 6000 Blackwell (96 GB). The performance figures apply to that card. The quote distinguishes Workstation Edition (up to 600 W) and Max-Q (up to 300 W).
Suitable models
| Model | Quantisation | Memory | Assessment |
|---|---|---|---|
| gpt-oss-120b | MXFP4 (~61 GB) | ja | Referenzmodell, gut belegt |
| Mistral Small 4 (119B, 6,5B aktiv) | NVFP4 | ja | europäisches Modell, gemessen |
| Nemotron 3 Super 120B-A12B | NVFP4 | ja | – |
| Qwen3.6-35B-A3B / Qwen3.6-27B | FP8 | ja | viele parallele Nutzer möglich |
| Gemma 4 26B-A4B / 31B | FP8 / NVFP4 | ja | – |
| Llama 3.3 70B | FP8 (~70 GB) / Q4 | ja (FP8 knapp) | kein aktueller Benchmark ausgewertet |
| Qwen3.5-122B-A10B | FP8 (~122 GB) | nein (nur in Q4/NVFP4) | → 2× RTX |
Published performance figures
| Model | Quantisation | Software | Context | Generation tok/s, 1 user | Parallel throughput | Parallel at 32k context | Prefill tok/s | To first token | GPU peak | Tested on | Note | Source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| gpt-oss-120b | MXFP4 | vLLM | 1k (Einzel: 79,7 tok/s bei 128k) | 181.1 smooth. Comparable to cloud services. | 361 (4 Requests) | 139.5 | – | 63 ms | 530 W | 1× RTX Pro 6000 Blackwell | Millstone AI – gpt-oss-120b MXFP4 auf 1x RTX Pro 6000 Blackwell (vLLM) 30 January 2026 | |
| gpt-oss-120b | MXFP4 | vLLM | 4k max-model-len 8k | 172.1 smooth. Comparable to cloud services. | 1,088.2 (32 Requests) | – | – | – | – | 1× RTX Pro 6000 Blackwell | Batch 8: 618,7 tok/s | GitHub kishan5111/gptoss-benchmark – gpt-oss-120b vLLM, H100 vs RTX Pro 6000 (Community) 8 October 2026 |
| gpt-oss-120b | MXFP4 | llama.cpp (CUDA) | 4k (131k: 113,7 tok/s Decode, 1.177 tok/s Prefill) | 210.7 smooth. Comparable to cloud services. | – | – | 6,495 | – | – | 1× RTX Pro 6000 Blackwell Workstation Edition | Hardware Corner – llama.cpp-Update: GPT-OSS 120B auf RTX Pro 6000 Blackwell 8 October 2026 | |
| Mistral Small 4 119B (2603) | NVFP4 | SGLang | 1k (Einzel: 64,2 tok/s bei 256k) | 131.3 smooth. Comparable to cloud services. | 261.7 (5 Requests) | 148.6 | – | 477 ms | – | 1× RTX Pro 6000 Blackwell | Millstone AI – Mistral-Small-4-119B-2603 NVFP4 auf 1x RTX Pro 6000 Blackwell (SGLang) 17 March 2026 | |
| Qwen3.6-35B-A3B | FP8 | vLLM | 1k | 196.4 smooth. Comparable to cloud services. | 449 (5 Requests) | 253.8 | – | 55 ms | 532 W | 1× RTX Pro 6000 Blackwell | Millstone AI – Qwen3.6-35B-A3B FP8 auf 1x RTX Pro 6000 Blackwell (vLLM) 25 May 2026 | |
| Qwen3.6-27B (dicht) | FP8 | vLLM | 1k | 46.1 comfortable. Pleasant for agentic coding. | 189.3 (5 Requests) | 92.5 | – | 170 ms | 521 W | 1× RTX Pro 6000 Blackwell | Millstone AI – Qwen3.6-27B FP8 auf 1x RTX Pro 6000 Blackwell (vLLM) 27 May 2026 | |
| Gemma 4 26B-A4B | FP8 | vLLM | 1k | 139 smooth. Comparable to cloud services. | 674.2 (10 Requests) | 334.4 | – | 81 ms | 469 W | 1× RTX Pro 6000 Blackwell | Millstone AI – Gemma-4-26B-A4B FP8 auf 1x RTX Pro 6000 Blackwell (vLLM) 14 April 2026 | |
| Gemma 4 31B (dicht) | NVFP4 | vLLM | 1k | 40.7 comfortable. Pleasant for agentic coding. | 126 (4 Requests) | 68.1 | – | 113 ms | 517 W | 1× RTX Pro 6000 Blackwell | Millstone AI – Gemma-4-31B NVFP4 auf 1x RTX Pro 6000 Blackwell (vLLM) 2 April 2026 | |
| Nemotron 3 Super 120B-A12B | NVFP4 | vLLM | 1k | 69.9 smooth. Comparable to cloud services. | 178.3 (5 Requests) | 108.7 | – | 126 ms | 531 W | 1× RTX Pro 6000 Blackwell | Millstone AI – Nemotron-3-Super-120B-A12B NVFP4 auf 1x RTX Pro 6000 Blackwell (vLLM) 11 March 2026 |
Power consumption
912 W Approximation
GPU peak 532 W measured (Millstone, Qwen3.6-35B-A3B). System including CPU and platform estimated with an additional 280 W CPU TDP (G5 example) and about 100 W for the platform, as an upper bound.
Cost per 1 million tokens
Reference: gpt-oss-120b MXFP4 (vLLM). Hardware: Price on request. Indicative price from the configurator, without additional storage.
moderate: 361 tok/s = Millstone, 4 parallel requests, 1k context; high: 1,088 tok/s at batch 32 (community measurement kishan5111)
| Scenario | Euro per 1 million tokens |
|---|---|
| Single seat 8 hours × 220 working days, generating 30% of the time, one request 181.1 tok/s | €19.76 (€32.66) |
| Team 8 hours × 220 working days, 50% utilisation, 4–8 parallel requests. Main scenario. 361 tok/s | €6.03 (€9.92) |
| Team, highly parallel 8 hours × 220 working days, 50% utilisation, batch from 32 requests 1,088.2 tok/s | €2.00 (€3.29) |
| 24/7 batch 24 hours × 365 days, 90% utilisation, batch from 32 requests 1,088.2 tok/s | €0.29 (€0.43) |
€0.30/kWh net (commercial). Euro per 1 million generated tokens, net. Depreciation over 5 years, in parentheses over 3 years. Assumptions and method
Comparison with API prices
Per 1 million output tokens plus 3 million input tokens, a typical ratio of 3:1. List prices, converted at the ECB reference rate. In this calculation the local computer processes the prompt at no additional token price.
| API model | Input / output, USD per 1 million | Euro per 1 million output incl. 3 million input | Source |
|---|---|---|---|
| gpt-oss-120b (OpenRouter) | 0.037 / 0.17 | €0.25 | OpenRouter – Models API (Listenpreise gehosteter Open-Weight-Modelle) |
| Mistral Small 4 (Mistral API) | 0.15 / 0.60 | €0.94 | Mistral Docs – API pricing |
| OpenAI gpt-6-luna | 0.10 / 0.50 | €0.72 | OpenAI – API Pricing (Standard, Short context) |
| Anthropic Claude Haiku 5.5 | 0.10 / 0.50 | €0.72 | Anthropic – Claude Pricing |
2× RTX PRO 6000
2× NVIDIA RTX PRO 6000 Blackwell (192 GB)
192 GB of graphics memory for models in the 120 to 300 billion parameter class at 8 or 4 bits, or for two independent instances of a 120B-class model.
The same card mapping applies as for one RTX PRO 6000 Blackwell. The performance figures refer to two of these cards.
Also reported, without its own table row: Step-3.7-Flash (198B, NVFP4) at 218 tok/s peak throughput in the Millstone overview.
- Applies to
- G4 or G5, 32 GB / 128 GB / 192 GB, 2× NVIDIA RTX6000
- Memory
- 192
- Memory bandwidth
- 1,792 GB/s per GPU
- Power consumption
- 600 (Workstation Edition) bzw. 300 (Max-Q)
Two NVIDIA RTX PRO 6000 Blackwell cards with 192 GB together. The mapping matches the single card.
Suitable models
| Model | Quantisation | Memory | Assessment |
|---|---|---|---|
| Qwen3.5-122B-A10B | FP8 | ja | gemessen |
| DeepSeek-V4-Flash (284B, 13B aktiv) | FP4+FP8 | ja | gemessen, bis 1M Kontext |
| Step-3.7-Flash (198B) | NVFP4 | ja | Spitzendurchsatz 218 tok/s (Millstone-Übersicht) |
| gpt-oss-120b ×2 / Mistral Small 4 ×2 | MXFP4 / NVFP4 | je GPU eine Instanz | Durchsatz ≈ 2× Einzel-GPU (Hochrechnung, nicht gemessen) |
| Llama 3.3 70B | FP8/BF16 | ja | kein aktueller Benchmark ausgewertet |
Published performance figures
| Model | Quantisation | Software | Context | Generation tok/s, 1 user | Parallel throughput | Parallel at 32k context | To first token | GPU peak | Tested on | Note | Source |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.5-122B-A10B | FP8 | vLLM (TP=2) | 1k (Einzel: 81,7 tok/s bei 256k) | 106 smooth. Comparable to cloud services. | 236.8 (5 Requests) | 148.8 | 102 ms | 984 W | 2× RTX Pro 6000 Blackwell | Millstone AI – Qwen3.5-122B-A10B FP8 auf 2x RTX Pro 6000 Blackwell (vLLM) 1 March 2026 | |
| DeepSeek-V4-Flash | FP4+FP8 | vLLM (TP=2) | 1k (Einzel: 96,3 tok/s bei 1M) | 106.1 smooth. Comparable to cloud services. | 367.8 (8 Requests) | 170 | 1,700 ms | 865 W | 2× RTX Pro 6000 Blackwell | Millstone AI – DeepSeek-V4-Flash FP4+FP8 auf 2x RTX Pro 6000 Blackwell (vLLM) 28 July 2026 | |
| gpt-oss-120b (2 Instanzen) | MXFP4 | vLLM | – | 181.1 smooth. Comparable to cloud services. | 722 (8 Requests) | – | – | – | 1× RTX Pro 6000 Blackwell (×2 hochgerechnet) | Approximation Hochrechnung: 2 unabhängige Instanzen à 361 tok/s; nicht gemessen | Millstone AI – gpt-oss-120b MXFP4 auf 1x RTX Pro 6000 Blackwell (vLLM) 30 January 2026 |
Power consumption
1,364 W Approximation
GPU peak 984 W measured (Millstone, two GPUs, Qwen3.5-122B). System including CPU and platform estimated with an additional 280 W CPU TDP (G5 example) and about 100 W for the platform, as an upper bound.
Cost per 1 million tokens
Reference: Qwen3.5-122B-A10B FP8 (vLLM, TP=2). Hardware: Price on request. Indicative price from the configurator, without additional storage.
moderate: Millstone, 5 parallel requests, 1k context; no figure for batch ≥ 32 → high parallelism calculated at 236.8 tok/s (lower bound)
| Scenario | Euro per 1 million tokens |
|---|---|
| Single seat 8 hours × 220 working days, generating 30% of the time, one request 106 tok/s | €46.63 (€77.00) |
| Team 8 hours × 220 working days, 50% utilisation, 4–8 parallel requests. Main scenario. 236.8 tok/s | €12.72 (€20.87) |
| Team, highly parallel 8 hours × 220 working days, 50% utilisation, batch from 32 requests 236.8 tok/s | €12.72 (€20.87)* |
| 24/7 batch 24 hours × 365 days, 90% utilisation, batch from 32 requests 236.8 tok/s | €1.85 (€2.76)* |
* No measured value is available for this level of concurrency. The calculation uses the next-lower measured throughput. Actual costs are then more likely to be lower.
€0.30/kWh net (commercial). Euro per 1 million generated tokens, net. Depreciation over 5 years, in parentheses over 3 years. Assumptions and method
Comparison with API prices
Per 1 million output tokens plus 3 million input tokens, a typical ratio of 3:1. List prices, converted at the ECB reference rate. In this calculation the local computer processes the prompt at no additional token price.
| API model | Input / output, USD per 1 million | Euro per 1 million output incl. 3 million input | Source |
|---|---|---|---|
| Qwen3.5-122B-A10B (OpenRouter) | 0.26 / 2.08 | €2.56 | OpenRouter – Models API (Listenpreise gehosteter Open-Weight-Modelle) |
| Mistral Medium 3.5 (Mistral API) | 1.50 / 7.50 | €10.74 | Mistral Docs – API pricing |
| OpenAI gpt-6.1-sol | 2.00 / 10.00 | €14.32 | OpenAI – API Pricing (Standard, Short context) |
| Anthropic Claude Sonnet 5.5 | 2.00 / 10.00 | €14.32 | Anthropic – Claude Pricing |
Cost per 1 million tokens
A computer of your own has high acquisition costs and, after that, mainly electricity costs. What a token costs therefore depends above all on how heavily the computer is utilised.
Only generated tokens are counted. In local operation, prompt processing is included in these reference values with no surcharge. Idle power, cooling, maintenance, interest and staff are not included. Depreciation is straight-line.
Calculation method
- Active hours per year = operating hours per year × utilisation
- Tokens per year = tokens per second × 3,600 × active hours
- Electricity cost per year = system power in kW × active hours × electricity price
- Hardware per year = net hardware price ÷ depreciation period (3 or 5 years)
- Euro per 1 million tokens = (hardware per year + electricity cost per year) ÷ (tokens per year ÷ 1,000,000)
Euro per 1 million tokens = (net price ÷ years + kW × active hours × electricity price) ÷ (tokens per second × 3,600 × active hours ÷ 1,000,000). Active hours = operating hours × utilisation.
- Main assumption for commercial electricity, net. The basis is a market observation of 27.15 ct/kWh net at 10,000 kWh per year (February 2026), rounded up. For comparison, the BDEW electricity-price analysis for August 2026 cites 17.2 ct/kWh in industry and 37.0 ct/kWh for households. MySmartEnergy – Gewerbestrom 2026 (Marktbeobachtung 27,15 ct/kWh netto bei 10.000 kWh, Feb. 2026), BDEW-Strompreisanalyse August 2026 (Industrie 17,2 ct/kWh; Haushalte 37,0 ct/kWh)
- Straight-line, excluding interest, maintenance, residual value, staff, room cooling and idle power.
- ECB reference rate of 7 October 2026. €1 = 1.1177 USD. EZB-Referenzkurs EUR/USD
Comparison of the reference configurations
For each profile, one reference configuration without additional storage is calculated. Additional storage does not change inference performance; it changes the purchase price and therefore the token cost slightly. The figures shown are reference values for the reference stated.
| Reference | Model | Hardware | Single seat | Team | Team, highly parallel | 24/7 batch |
|---|---|---|---|---|---|---|
| G4 without a GPU | Llama 3.1 8B Instruct Q4_K_M | €14,200 net | €67.01 (€110.89) | €40.68 (€67.01)* | €40.68 (€67.01)* | €5.60 (€8.54)* |
| G5 without a GPU | Llama 3.1 8B Instruct Q4_K_M | €20,700 net | €44.55 (€73.82) | €26.99 (€44.55)* | €26.99 (€44.55)* | €3.58 (€5.54)* |
| 4× AMD RX 9070 XT | Qwen3.6-35B-A3B (AWQ-INT4 Batch / Q6_K Einzelstrom) | €18,100 net | €28.23 (€45.87) | €17.65 (€28.23)* | €1.33 (€2.12) | €0.27 (€0.36) |
| 1× RTX PRO 6000 | gpt-oss-120b MXFP4 (vLLM) | Price on request Approximation | €19.76 (€32.66) | €6.03 (€9.92) | €2.00 (€3.29) | €0.29 (€0.43) |
| 2× RTX PRO 6000 | Qwen3.5-122B-A10B FP8 (vLLM, TP=2) | Price on request Approximation | €46.63 (€77.00) | €12.72 (€20.87) | €12.72 (€20.87)* | €1.85 (€2.76)* |
* No measured value is available for this level of concurrency. The calculation uses the next-lower measured throughput. Actual costs are then more likely to be lower.
Indicative. The token costs are based on an estimated system price and are not a quote. In the configurator, price on request applies to NVIDIA configurations.
Effect of the electricity price
In the team scenario with five years of depreciation, the electricity price changes the result only slightly. The purchase price and utilisation are what decide it.
| Reference | 17.2 ct/kWh, industry (BDEW) | 30 ct/kWh, commercial (main assumption) | 37 ct/kWh, households (BDEW) |
|---|---|---|---|
| G4 without a GPU | €40.18 | €40.68 | €40.96 |
| G5 without a GPU | €26.71 | €26.99 | €27.13 |
| 4× AMD RX 9070 XT | €16.89 | €17.65 | €18.07 |
| 1× RTX PRO 6000 | €5.94 | €6.03 | €6.08 |
| 2× RTX PRO 6000 | €12.51 | €12.72 | €12.83 |
Reading this against API prices
On token price alone, hosted open-weight models are very inexpensive in 2026. A computer of your own reaches this price range at high, sustained utilisation, for example with many users or with batch processing around the clock. The more common reason for a computer of your own is a different one: data sovereignty, predictable fixed costs, no rate limits, independence from providers, and the option of fine-tuning on your own data. Against premium APIs in the Sonnet and gpt-6.1 class, a computer of your own is already lower in the team scenario, provided an open model is sufficient for the task.
German and European models
Mistral (France)
Mistral Small 4 was measured on the RTX PRO 6000: 131 tokens per second for one request, 262 tokens per second with parallel requests. For Ministral 3 8B and 14B, no direct figures are available on this hardware. Models of the same size (Qwen3-8B and Qwen3-14B) serve as an approximation. Millstone AI – Mistral-Small-4-119B-2603 NVFP4 auf 1x RTX Pro 6000 Blackwell (SGLang)
Soofi S (German consortium)
30B-A3B, German and English. As of October 2026, available only as a closed beta preview. Hardware benchmarks on the systems offered here are not available. The model uses the Nemotron-3-Nano architecture (30B, around 3B active) and is likely to behave similarly to other MoE models of this class. This is an assessment, not a measured value. Hugging Face – Soofi-Project/Soofi-S-Base (Closed Beta)
Teuken
Current inference benchmarks on the hardware offered here are not available.
Hybrid: daytime use, out-of-hours rental
This scenario supplements team use. On working days the computer is available for your own work (8 hours × 220 working days). In the remaining time, that is at night, at weekends and on public holidays, spare graphics-card capacity is assumed to be rented out, so that part of the hardware cost can be offset.
The amounts are reference values from published marketplace and price data, not a revenue guarantee. Utilisation of the out-of-hours time, that is the share in which someone actually rents, is the largest uncertainty. It depends on region, price, reliability, network connection and the rest of the supply.
For NVIDIA configurations, the configurator states price on request. The token costs and the shares of depreciation in this section are indicative and are based on an estimated system price. They are not a quote.
Configurations without a graphics card are barely in demand on the marketplaces considered. No extra income is modelled for them.
Host programmes
The overview summarises published programmes through which operators can offer computing capacity. It is a factual assessment, not a recommendation and not a brokerage.
| Programme | Company seat | Suitability for a single system | Hardware | Programme’s share | Out of hours only | Assessment |
|---|---|---|---|---|---|---|
| Primcast Marketplace | Company in the US (NY) plus an office in Bucharest; renter filter “EU and adequate countries”; hosts worldwide | high | NVIDIA/AMD/Intel single box | 0% | ja (Listing pausierbar) | Outbound TCP 2222, CGNAT ok, KYC, Monatszahlung Bank. Isolation per KVM-Passthrough. |
| Vast.ai | US platform; hosts worldwide, including the EU | medium | NVIDIA only | 0% | nein (dedizierte Maschine; Eigennutzung bricht Verification) | 500 Mbps symmetrisch, Public IPv4, Ubuntu Server. Host-Fee laut Juni-2024-Update 0 % (FAQ teils noch 25 % – Agreement prüfen). |
| Salad / SaladCloud | US | medium-low | NVIDIA plus selected AMD | not stated | teilweise (Idle-Netzwerk verdient mit) | Verdienst = Durchschnitt inkl. Wartezeit. AMD-Karten verdienen deutlich weniger als NVIDIA. |
| Nosana | Netherlands (EU) | medium | NVIDIA (4090/5090 campaign) | not stated | nein (85 % Tages-Uptime für Incentive) | DePIN, Auszahlung in NOS; idle capacity ok, aber Uptime-Anforderung hoch. |
| Golem Network GPU Provider | Poland / close to the EU | low-medium | NVIDIA RTX 30xx+ | not stated | nein (Maschine während Betrieb dediziert) | Live-USB-Provider-OS; Nachfrage und USD-Äquivalent schwankend (GLM). |
| OpenRouter Provider Program | US | very low (single box) | your own inference endpoint | not stated | nein (95 %+ Uptime für normales Routing) | Backlog; Priorität für proprietäre Modelle; OpenAI-API, /models, Invoicing, Privacy-Policy Pflicht. Nacht-only fällt unter 80 % → nur Fallback. |
| OpenCode Zen | US-hosted models according to the Zen docs | no host programme | — | not stated | n/a | Kuratierter Pay-per-use-Gateway für Endnutzer. Kein Marktplatz zum Vermieten eigener Hardware. |
| Runpod Community Cloud | US; Secure Cloud possible in EU regions | closed | NVIDIA | not stated | n/a | Community Cloud für neue Hosts geschlossen; Secure Cloud nur Data-Center-Partner. |
| TensorDock | US (Delaware) | low | NVIDIA (AMD in parts, according to the docs) | 25% | nein (SLA-Strafen bei Downtime) | 25 % Fee, Mindestauszahlung 250 $, Strafzahlungen bei Reboots/Downtime. |
| Akash Network | decentralised / global | low | NVIDIA only | not stated | nein | Kubernetes-Cluster, Domain, Public IP; für Single-Box-KMU aufwendig. |
| io.net | US / DePIN | low | NVIDIA consumer possible | not stated | teilweise | Staking; Consumer-Geräte nur kleiner Block-Reward-Anteil; Verdienst stark tokenabhängig. |
| Chutes (Bittensor SN64) | DePIN, global | very low | data-centre NVIDIA | not stated | nein | Bare-metal, static IP, K8s; shared Marketplace-GPUs ungeeignet. |
| Hivenet | France (EU) | unclear | NVIDIA (its own cloud) | not stated | unklar | EU-Anbieter mit Compute-Angebot; öffentliches Single-Box-Host-Programm nicht dokumentiert. |
Data protection, liability and commercial activity
- Other people’s workloads on a company computer: isolation (VM/passthrough) and network separation are required; otherwise there is an access and liability risk.
- Personal data of third parties (tenants’ prompt or training data) means processing on behalf of a controller, a data-processing agreement and technical and organisational measures; for US platforms, check a third-country transfer (SCCs or an adequacy decision).
- Your own company data must not be reachable on the same system during rental (separate volumes, no host login for tenants).
- Business income is taxable (VAT, and where relevant the German small-business rule under § 19 UStG; income tax as business income). Check whether a trade registration is required.
- Liability: misuse by a tenant, for example for attacks or unlawful content. The programme’s terms and your own safeguards, such as a firewall and monitoring, have to be taken into account.
Out-of-hours utilisation
Own use: 1,760 hours per year. Off-hours: 7,000 hours per year.
Always-on verified RTX PRO 6000 WS on Vast.ai, about 87.5% occupancy (Hashrate.no, 8 October 2026). Night-and-weekend-only use, combined with your own use in parallel, typically prevents verification and premium routing. Utilisation of the off-hours time is therefore set deliberately lower: 15, 30 and 50 percent.
| Scenario | Share of off-hours | hours per year |
|---|---|---|
| Pessimistic | 15 % | 1,050 |
| Realistic | 30 % | 2,100 |
| Optimistic | 50 % | 3,500 |
Hashrate.no – Vast.ai RTX PRO 6000 Workstation (Occupancy & Host-Raten), Hashrate.no – Vast.ai Verification Stats (Gesamt-Occupancy), Aura Digital – Vast.ai GPU Server Earnings (Nutzung unbekannt, 3 Szenarien)
Renting out the graphics cards
Main path of the model calculation: during out-of-hours time the card is rented out by the hour. The electricity price and the exchange rate match the cost section. System power is the same upper bound as in the profile in question.
Calculation of net additional income. Net per year = (hourly rate in US dollars × number of graphics cards × rented hours ÷ exchange rate) − (system power in kW × rented hours × electricity price).
4× AMD RX 9070 XT
Approximation: Salad cloud renter price for the RX 9070 XT, $0.07–0.196/h (ComputePrices, 8 October 2026); host net including idle time is lower. No Vast.ai AMD market. A thin market; the figures are speculative. Approximation
ComputePrices – Salad Cloud RX 9070 XT Mietpreise, ComputePrices – Salad Cloud RX 7800 XT, Salad – GPU earning rates (Durchschnitt inkl. Idle)
| Scenario | Net per year | Share of depreciation, 5 years | Share of depreciation, 3 years | Euro per 1 million tokens after offsetting, 5 years | Team without offsetting, 5 years |
|---|---|---|---|---|---|
| Pessimistic | -€372.68 | -10.3 % | -6.2 % | €19.29 | €17.65 |
| Realistic | -€595.06 | -16.4 % | -9.9 % | €20.26 | €17.65 |
| Optimistic | -€741.25 | -20.5 % | -12.3 % | €20.90 | €17.65 |
1× RTX PRO 6000
Vast.ai host P50 for the RTX PRO 6000 WS, about $1.05/h verified / about $0.90 unverified (Hashrate.no, 8 October 2026); floor about $1.30 (GPU Economy). Discount for part-time use. Approximation
Hashrate.no – Vast.ai RTX PRO 6000 Workstation (Occupancy & Host-Raten), GPU Economy – Vast.ai Preise (Marktplatz-Floor je GPU), Vast.ai – How Much Money Can You Earn Renting Out Your GPU (Mai 2026)
| Scenario | Net per year | Share of depreciation, 5 years | Share of depreciation, 3 years | Euro per 1 million tokens after offsetting, 5 years | Team without offsetting, 5 years |
|---|---|---|---|---|---|
| Pessimistic | €464.26 | 7 % | 4.2 % | €5.63 | €6.03 |
| Realistic | €1,304.30 | 19.6 % | 11.8 % | €4.89 | €6.03 |
| Optimistic | €2,800.12 | 42 % | 25.2 % | €3.59 | €6.03 |
2× RTX PRO 6000
The same hourly rate per graphics card as for one RTX PRO 6000, here for two cards. Approximation
Hashrate.no – Vast.ai RTX PRO 6000 Workstation (Occupancy & Host-Raten), GPU Economy – Vast.ai Preise (Marktplatz-Floor je GPU)
| Scenario | Net per year | Share of depreciation, 5 years | Share of depreciation, 3 years | Euro per 1 million tokens after offsetting, 5 years | Team without offsetting, 5 years |
|---|---|---|---|---|---|
| Pessimistic | €1,073.43 | 11.7 % | 7 % | €11.28 | €12.72 |
| Realistic | €2,898.40 | 31.6 % | 18.9 % | €8.85 | €12.72 |
| Optimistic | €6,083.23 | 66.3 % | 39.8 % | €4.61 | €12.72 |
Selling tokens
A supplementary, illustrative calculation: during out-of-hours time, tokens are sold through a provider, below the published end-customer price, and actually generating during only half of the rented hours. This is not the main scenario.
4× AMD RX 9070 XT. Illustrative; there is practically no AMD inference-provider market.
1× RTX PRO 6000. Illustrative, below the retail floor of 0.03/0.17 (OpenRouter CoreWeave, 8 October 2026). Electricity cost alone is in the same range.
2× RTX PRO 6000. Illustrative, below retail 0.26/2.08 (OpenRouter).
| Reference | Scenario | Net per year | Share of depreciation, 5 years | Share of depreciation, 3 years | Euro per 1 million tokens after offsetting, 5 years | Team without offsetting, 5 years |
|---|---|---|---|---|---|---|
| 4× AMD RX 9070 XT | Pessimistic | -€389.23 | -10.8 % | -6.5 % | €19.36 | – |
| 4× AMD RX 9070 XT | Realistic | -€778.46 | -21.5 % | -12.9 % | €21.07 | – |
| 4× AMD RX 9070 XT | Optimistic | -€1,297.44 | -35.8 % | -21.5 % | €23.34 | – |
| 1× RTX PRO 6000 | Pessimistic | -€168.24 | -2.5 % | -1.5 % | €6.18 | – |
| 1× RTX PRO 6000 | Realistic | -€336.49 | -5.1 % | -3 % | €6.33 | – |
| 1× RTX PRO 6000 | Optimistic | -€560.81 | -8.4 % | -5.1 % | €6.52 | – |
| 2× RTX PRO 6000 | Pessimistic | €231.04 | 2.5 % | 1.5 % | €12.41 | – |
| 2× RTX PRO 6000 | Realistic | €462.07 | 5 % | 3 % | €12.10 | – |
| 2× RTX PRO 6000 | Optimistic | €770.12 | 8.4 % | 5 % | €11.69 | – |
For a single system that you use yourself during the day, selling tokens is not economically robust at present. Programmes for this path generally require high continuous availability. Renting the graphics card by the hour is the more natural of the two calculations.
The extra income covers only part of the depreciation. For the profile with AMD graphics cards the rental market is thin and the model calculation is negative. Selling tokens from a single system that is also used during the day is not economically robust. Utilisation of the off-hours time is the largest uncertainty. The figures are reference values, not a promise of revenue.
Limits
Data sovereignty during the rented hours
While the system is rented out, other parties’ workloads run on the company computer, arranged through the programme in question. A statement that no data leaves the premises does not hold for these hours. Your own company data must not be reachable on the system during this time. For other parties’ personal data, commissioned processing, a contract and, for providers outside the EU, the rules on transfers to third countries may need to be considered.
AMD does not pay off in this model calculation
For four Radeon RX 9070 XT, net additional income is below zero at all three utilisation levels. The power consumption assumed, deliberately chosen as an upper bound, exceeds the revenue derived from public price observations. The market for these cards is thin. Without markedly higher rental prices or a markedly lower real power draw, out-of-hours rental does not pay its way here.
The computer does not pay for itself
For the NVIDIA profiles as well, rental in the realistic case covers only part of depreciation over five years. The cost per million of your own tokens falls, and a substantial remainder of the purchase cost remains. The optimistic case with two cards assumes high out-of-hours utilisation, which is hard to combine with your own use during the day.
Availability, network and liability
Several programmes require near-continuous availability, high symmetric bandwidth or a public IPv4 address. That fits poorly with use during the day and rental only afterwards. Other parties’ workloads need proper isolation, otherwise access and liability risks arise. Income is relevant for tax; whether it counts as a commercial activity has to be checked in the individual case.
Sources
All pages were retrieved on the date stated. The date on the source is the measurement or publication date, where one is given.
- Empiria Insight – KI-Rechner-Konfigurator (Optionen & Richtpreise) 8 October 2026 Datenquelle im Repo: lukaskawerau/empiriainsight.com, src/lib/data/hardware.ts
- tiny corp – tinybox Konfigurator (Referenz der Optionen) 8 October 2026
- tiny shop – tinybox green v2 with 4x RTX Pro 6000 Blackwell 8 October 2026 Beleg, dass tiny corp mit 'RTX6000' die RTX PRO 6000 Blackwell (96 GB) verbaut
- MegaOne AI – Tinybox red v2 / green v2 Spezifikationen (Sekundärquelle) 8 October 2026 G4: 128 GB @ 204,8 GB/s; G5: 192 GB @ 460,8 GB/s; 4x 9070XT = 64 GB @ 2.560 GB/s
- NVIDIA RTX PRO 6000 Blackwell Workstation Edition – Datenblatt 8 October 2026 96 GB GDDR7, 1.792 GB/s, max. 600 W
- NVIDIA RTX PRO 6000 Blackwell Max-Q – Datenblatt 8 October 2026 96 GB GDDR7, 1.792 GB/s, max. 300 W
- AMD Radeon RX 9070 XT – Spezifikationen 8 October 2026 16 GB GDDR6, 640 GB/s, 304 W Total Board Power
- AMD EPYC 7543 (32 Kerne, DDR4) – Spezifikationen 8 October 2026 TDP 225 W; Beispiel-SKU für G4, konkrete CPU nicht festgelegt
- AMD EPYC 9354 (32 Kerne, Genoa) – Spezifikationen 8 October 2026 TDP 280 W; Beispiel-SKU für G5, konkrete CPU nicht festgelegt
- Millstone AI – gpt-oss-120b MXFP4 auf 1x RTX Pro 6000 Blackwell (vLLM) 30 January 2026
- Millstone AI – Qwen3.6-35B-A3B FP8 auf 1x RTX Pro 6000 Blackwell (vLLM) 25 May 2026
- Millstone AI – Qwen3.6-27B FP8 auf 1x RTX Pro 6000 Blackwell (vLLM) 27 May 2026
- Millstone AI – Mistral-Small-4-119B-2603 NVFP4 auf 1x RTX Pro 6000 Blackwell (SGLang) 17 March 2026
- Millstone AI – Gemma-4-26B-A4B FP8 auf 1x RTX Pro 6000 Blackwell (vLLM) 14 April 2026
- Millstone AI – Gemma-4-31B NVFP4 auf 1x RTX Pro 6000 Blackwell (vLLM) 2 April 2026
- Millstone AI – Nemotron-3-Super-120B-A12B NVFP4 auf 1x RTX Pro 6000 Blackwell (vLLM) 11 March 2026
- Millstone AI – Qwen3.5-122B-A10B FP8 auf 2x RTX Pro 6000 Blackwell (vLLM) 1 March 2026
- Millstone AI – DeepSeek-V4-Flash FP4+FP8 auf 2x RTX Pro 6000 Blackwell (vLLM) 28 July 2026
- Millstone AI – Übersicht LLM Inference Benchmarks 8 October 2026
- GitHub kishan5111/gptoss-benchmark – gpt-oss-120b vLLM, H100 vs RTX Pro 6000 (Community) 8 October 2026 Datum der Messung nicht angegeben
- Hardware Corner – llama.cpp-Update: GPT-OSS 120B auf RTX Pro 6000 Blackwell 8 October 2026 Veröffentlichungsdatum auf der Seite nicht maschinenlesbar
- llama.cpp Issue #26663 – RX 9070 XT, HIP vs. Vulkan (Qwen3 Q4_K_M) 7 August 2026
- GitHub kr4ckhe4d/local-llm-benchmarks – llama.cpp auf RX 9070 XT / R9700 (ROCm 7.2) 6 October 2026 Community-Messungen; Einzelkarte 9070 XT bzw. 9070 XT + R9700
- GitHub MrLordCat/llama.cpp-rdna-lab – 2x RX 9070 XT, Qwen3.8-27B 26 September 2026 Community-Fork von llama.cpp; Layer-Split über 2 GPUs
- GitHub patcarter883/rdna4-vllm – vLLM TP=2 auf RDNA4 (9070 XT + 9070) 8 October 2026 Community-Build; Offline-Batch 32 Prompts x 128 Output-Tokens; gestreamte Last führte laut README zu OOM
- digtvbg – RX 9070 XT: vLLM (ROCm 7.2) vs. llama-server (Vulkan) 22 March 2026
- ahelpme.com – llama.cpp CPU-Benchmarks AMD EPYC 9554 (64 Kerne, 12x DDR5-5600) 25 March 2025
- ahelpme.com – llama.cpp CPU-Benchmarks 2x AMD EPYC 7282 (2x16 Kerne, 2x8 DDR4-3200) 1 April 2025
- Geizhals – NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96 GB 27 August 2026 Öffentlicher Preisvergleich, herangezogen als Eingangsgröße für die indikative Tokenkostenrechnung der NVIDIA-Profile. Kein Angebotspreis von Empiria Insight.
- MySmartEnergy – Gewerbestrom 2026 (Marktbeobachtung 27,15 ct/kWh netto bei 10.000 kWh, Feb. 2026) 12 July 2026
- BDEW-Strompreisanalyse August 2026 (Industrie 17,2 ct/kWh; Haushalte 37,0 ct/kWh) 2026-08
- Mistral Docs – API pricing 8 October 2026
- OpenAI – API Pricing (Standard, Short context) 8 October 2026
- Anthropic – Claude Pricing 8 October 2026
- OpenRouter – Models API (Listenpreise gehosteter Open-Weight-Modelle) 8 October 2026 Aggregator; Preise je Anbieter schwankend
- EZB-Referenzkurs EUR/USD 7 October 2026 1 EUR = 1,1177 USD
- Hugging Face – Soofi-Project/Soofi-S-Base (Closed Beta) 8 October 2026
- OpenRouter – Become a Provider (Anforderungen, Uptime-Routing, Priorität für proprietäre Modelle) 8 October 2026 Backlog; Priorität für proprietary models; 95 %+ Uptime für normales Routing; OpenAI-kompatible API, /models, Invoicing, Privacy-Policy Pflicht
- OpenRouter Docs – Provider Integration 8 October 2026
- OpenCode Zen – kuratierter Modell-Gateway (kein Host-/Provider-Programm für eigene Hardware) 8 October 2026 Pay-per-use API für Endnutzer; kein Marktplatz, um eigene Boxen zu vermieten
- Vast.ai Docs – Host Verification Stages (Anforderungen) 8 October 2026 Ubuntu Server, NVIDIA only, 500 Mbps Up/Down, Public IPv4, RAM ≥ 95 % VRAM, dedizierte Maschine (keine persönliche Last); RTX PRO 6000 priorisiert
- Vast.ai Docs – Host Payouts 8 October 2026
- Vast.ai – June 2024 Product Update (Host-Fee 0 %) 2024-06 Host-seitige Fee entfernt; gesetzter Preis = Host-Einnahme. FAQ nennt teils noch 75/25 – vor Kalkulation im Host-Agreement prüfen
- Vast.ai – How Much Money Can You Earn Renting Out Your GPU (Mai 2026) 18 May 2026 RTX 5090 typ. 0,30–0,60 $/GPU-h; Live-Mietpreise ca. 25 % über Host-Einnahme
- GPU Economy – Vast.ai Preise (Marktplatz-Floor je GPU) 8 October 2026 RTX PRO 6000 Blackwell Floor ca. 1,30 $/GPU-h; RTX 5090 ca. 0,48–0,50 $; RTX 4090 ca. 0,39 $
- Hashrate.no – Vast.ai RTX PRO 6000 Workstation (Occupancy & Host-Raten) 8 October 2026 Verified Occupancy 87,5 %, Host-P50 ca. 1,05 $/h (rented); Unverified Occupancy 59,5 %, Host-P50 ca. 0,90 $/h
- Hashrate.no – Vast.ai Verification Stats (Gesamt-Occupancy) 8 October 2026 Gesamt-Occupancy ca. 79 %; RTX PRO 6000 WS 4,2 % der Verifications, 30-Tage-Anteil ca. 60 %
- Salad – GPU earning rates (Durchschnitt inkl. Idle) 8 October 2026 Durchschnitt über alle Chopping-Stunden inkl. Wartezeit
- Salad – Live demand / earning rates 8 October 2026
- ComputePrices – Salad Cloud RX 9070 XT Mietpreise 8 October 2026 0,070–0,196 $/GPU-h je Priority-Tier (Renter-Preis, nicht Host-Netto)
- ComputePrices – Salad Cloud RX 7800 XT 8 October 2026 0,042–0,132 $/GPU-h
- Primcast – GPU-Marketplace (Host setzt Miete, 0 % Host-Fee) 8 October 2026 US-Firma (NY) + Büro Bukarest; NVIDIA/AMD/Intel; Outbound TCP 2222, CGNAT ok; EU-Filter für Renter; KYC; Monatszahlung Bank
- Primcast – About (NY & Bucharest) 8 October 2026
- Runpod – Community Cloud Hosting Application Update (geschlossen für neue Hosts) 8 October 2026 Spiegel des Runpod-Blogposts; neue Community-Hosts nicht mehr aufgenommen
- Runpod – Secure Cloud / Compute Partners 8 October 2026 Nur Data-Center-/Enterprise-Partner, nicht Single-Box
- Akash Network – Provider Hardware Requirements 8 October 2026 Nur NVIDIA; Kubernetes-Setup; Domain, Public IP, 100 Mbps min.
- TensorDock – Supplier Hosting Agreement (25 % Fee, SLA-Strafen) 26 March 2024 25 % Plattformgebühr; Uptime-SLA mit Strafzahlungen; Mindestauszahlung 250 $
- io.net – Device / Worker Onboarding 8 October 2026 NVIDIA Consumer möglich; Staking; Consumer nur geringer Block-Reward-Anteil
- Chutes (Bittensor SN64) – Miner Overview 8 October 2026 Bare-metal/single-tenant, static IP, K8s; keine shared Vast/Runpod-GPUs
- Nosana – RTX 4090/5090 Provider Campaign (85 % Tages-Uptime) 8 October 2026 Niederländisches DePIN; 85 % daily uptime für Incentive; NVIDIA Consumer
- Nosana Docs – Host Getting Started 8 October 2026
- Golem Network – AI/GPU Provider Roadmap 8 October 2026 EU-nah (Polen); NVIDIA RTX 30xx+; dedizierte Maschine während Betrieb
- Hivenet – Distributed Cloud Compute (FR) 8 October 2026 Französischer Anbieter; öffentliches Host-Programm für Einzelboxen undokumentiert
- Aura Digital – Vast.ai GPU Server Earnings (Nutzung unbekannt, 3 Szenarien) 2 September 2026 Betont: Occupancy nicht vorab kenntlich; Median RTX PRO 6000 WS Ende Aug. 2026 ca. 1,20 $/h
Community measurements from individuals’ GitHub projects are secondary sources. With new software versions, the values can rise or fall substantially.