https://network-switch.com/pages/david-lorame
Published: October 28, 2026 | Last Technically Reviewed: October 28, 2026
- 1. Quick Summary
- 2. What the 10-Person Office Actually Needs
- 3. Why PDF Count Is Not a GPU Sizing Metric
- 4. Separate Ingestion, Retrieval and Inference
- 5. When Existing Servers and Networking Are Enough
- 6. What to Measure Before Selecting a GPU Server
- 7. Build the Hardware BOM Around Verified Workloads
- 8. Frequently asked questions (FAQs)
- 9. Sources and References
Quick Summary
A small civil engineering office already uses Open WebUI with Windows Active Directory. About ten employees use external AI models for research and drafting. Now they want to search internal standards, prepare contracts, and work with roughly 500 PDFs. They have $5,000 to $6,000 for local hardware. Does that justify a GB10-based system or a dedicated GPU server?
That is the question in a September 2026 r/LocalLLaMA discussion. The office has an AI interface, but it has not established how much processing must run locally or whether its current equipment is limiting performance.
Quick Answer
Test the workload before buying hardware. Check document accuracy, retrieval quality, model memory, and overlapping requests. Reuse existing servers if they meet the requirements. Consider a GPU when local inference cannot meet the agreed response time or capacity. Upgrade the network only if measured storage or inter-server traffic calls for it. A small RAG installation may work on the existing LAN.
What the 10-Person Office Actually Needs
The original post concerns contracts, quotations, and engineering documents, not ten people generating answers continuously. The poster mentions conversion problems but says most important PDFs are not scans. Replies discuss tables, document versions, memory, and cloud costs. The thread records no completed purchase or benchmark.
Before specifying equipment, ask what a useful answer looks like. Must it identify the correct edition and clause? Is it finding reference material or drafting a contract for human review? Can excerpts reach an approved external model, or must processing stay local? These answers matter more than PDF count.
Open WebUI does not, by itself, require a new GPU appliance for embeddings, retrieval, and inference. Windows AD login also does not prove document-level authorization. Open WebUI documents group-based knowledge permissions, but the office must test the accounts, grants, and documents returned to each user.
Inventory CPU and RAM headroom, storage, backups, NICs, switch ports, document locations, and update frequency. A supported server with available capacity may be a better starting point than an untested GPU purchase.
If a new chassis eventually makes sense, our rack server versus blade server guide covers the differences in hardware format. For now, the more useful question is whether this office needs another server at all.
Why PDF Count Is Not a GPU Sizing Metric
Five hundred PDFs could be short text documents or lengthy standards with tables, diagrams, and revisions. That count says little about extracted text, parsing accuracy, index size, or inference memory.
In the Reddit discussion, a commenter warns about damaged tables and clauses and shares results from a different document set. Those figures cannot size this office. The poster confirms conversion problems but says most key files are not scans.
Start with a sample that includes merged tables, columns, clauses, drawings, and different document editions. Keep the correct text, page number, table values, and version alongside the parsed result. A faster GPU cannot restore a technical value lost during extraction.
The Unstructured documentation on PDF partitioning explains the difference between text extraction, OCR, and layout-aware processing. The suitable method depends on the file. Some workflows benefit from acceleration, while others can run comfortably on a CPU. The documentation does not tell us how this particular collection will perform.
Inference memory is a different problem. Model weights, quantization, runtime, context length, KV cache, and overlapping requests all matter. A long excerpt from one PDF can occupy more context than a short answer drawn from hundreds of files. Quantization can save memory, but its effects on technical answers should be tested.
Prepare a document profile before requesting hardware: page counts, scanned-page proportion, extracted text volume, update frequency, and versioning needs. Track concurrent queries separately. The model processes the prompt assembled for a question, not the full library of 500 PDFs.
Check document accuracy and answer performance separately. A system that replies quickly while quoting the wrong engineering clause is not ready to support real quotations. Better extraction and retrieval may be the first investments, with no new GPU needed.
Separate Ingestion, Retrieval and Inference
Document preparation and live queries have different requirements. Parsing and embeddings run when files arrive or change; retrieval and inference run when users ask questions. One GPU can serve several components, but that should be a design choice supported by tests.
RAG Component-to-Hardware Matrix
| Component | Typical role | CPU / GPU and memory considerations | Storage / network dependency | Evidence before purchase |
| Ingestion | Read, version, and queue source files | Usually CPU-led; RAM depends on file and job size | Source storage, temporary files, and access permissions | File size, processing window, and errors |
| OCR / document parsing | Recover scanned text, tables, and layout | CPU or a supported accelerated model, depending on the parser | Temporary images and any remote processing service | Table and clause accuracy; time per sample document |
| Embeddings | Convert selected passages into vectors | CPU may be enough for modest batches; GPU acceleration is optional | Model files and vector output | Indexing time and how often files change |
| Vector retrieval | Find relevant passages and apply access filters | Often uses CPU, RAM, and storage, depending on the database | Local NVMe or measured traffic to a remote database | Search quality and p95 query latency |
| Reranking | Reorder retrieved passages | CPU or GPU, depending on model and response-time target | Transfer of candidate passages between services | Improvement in relevance versus added delay |
| LLM inference | Generate an answer from the retrieved context | Model weights, quantization, runtime, and KV cache affect memory | Model loading; LAN traffic if inference runs on another host | TTFT, total response time, concurrency, and peak memory |
| Backup / recovery | Preserve files, indexes, and settings | Normally does not require an inference GPU | Separate backup storage and a usable restore path | Successful restore and recovery time |
Keep the document preparation path separate from the live query path. They create different peaks in CPU, storage, and model demand.
DOCUMENT PREPARATION (batch / on updates)
Source PDFs -> Parsing / OCR -> Embeddings -> Vector Index
|
v
LIVE QUESTION Retrieval
Office User -> Authorization Check ----------> |
v
Optional Reranker
|
v
LLM Inference
|
v
Answer with Sources
Open WebUI describes a single-instance setup. Multiple application instances bring additional database and coordination requirements. Ten employees alone do not justify that complexity.
If components run on different hosts, their connections matter. Ethernet file shares and Fibre Channel SANs are not interchangeable. Our HBA versus NIC versus CNA guide explains the options. An ordinary Ethernet share does not need an FC HBA.
Local indexing can be combined with an approved external inference API, provided company policy permits sending the relevant queries and document excerpts and accepts the provider's handling terms. This is not an option when inference must remain fully local.
Also note when each component runs. Document conversion can often happen after hours, while live answers cannot wait. An extra CPU worker and more GPU memory solve different problems. Review the full data path, including OCR, embeddings, reranking, storage, logs, and telemetry, before describing any deployment as private.
When Existing Servers and Networking Are Enough
Try existing supported hardware first. A server may already have enough capacity for parsing, a vector index, and even inference. Test it before ruling it out.
Existing Server Reuse Checklist
- Record the server model, operating system support, CPU load, and competing workloads.
- Check available RAM and the manufacturer's supported expansion options.
- Confirm NVMe or other storage capacity, backups, and a working restore process.
- Check supported GPU options, PCIe slots, power supply capacity, airflow, and physical fit.
- Record NIC and switch-port speeds, driver support, and measured traffic.
- Identify where documents, indexes, and inference services run and how they communicate.
- Review authentication, document permissions, firewall rules, and relevant logs.
- Run the pilot without putting production virtual machines or backups at risk.
A free PCIe slot is not proof that a GPU is supported. Check vendor limits on firmware, cards, power, and cooling before quoting an upgrade.
When is 1GbE enough? If parsing, retrieval, and inference run on one host and users mostly exchange text, 1GbE may be sufficient. A faster switch cannot accelerate calculations inside that server. Check imports, backups, and normal query traffic first.
When does 10GbE matter? Consider it when large NAS transfers, remote model loading, or separated compute create measurable traffic. Check storage, NICs, and switch counters together. Our AI Compute Center deployment solutions offer broader context, not proof that this office needs data center switches.
Cisco's Unified Edge for Red Hat Edge Design Guide, updated January 26, 2026, describes single-node and multi-node edge designs. It is useful architecture documentation, but it does not provide a benchmark for this ten-person RAG workload.
Review existing firewall rules, permitted API traffic, device capacity, and support status. Keep the firewall if it can enforce the required access policy.
Ask whether ordinary queries still access the NAS after the model and index are loaded. If not, a faster NAS link may do little for response time. If so, measure transfers and latency, and separate network limits from disk or encryption overhead. Every proposed switch upgrade should identify the constrained path and its endpoint interfaces.
What to Measure Before Selecting a GPU Server
Ten accounts are not ten continuous inference streams. Staff may be reading or editing, but several long requests could overlap near a deadline. Measure those patterns instead of guessing at concurrency.
Build a fixed question set with known answers, tables, document editions, cross-references, and cases where the correct response is that no source supports an answer. Check retrieval and citations before timing generation.
Measure retrieval, reranking where used, prompt assembly, TTFT, and total response time. Record p95 latency, runtime and model versions, quantization, context size, RAM/GPU peaks, and offload behavior.
Concurrent Request Test
| Scenario | How to run it | What to capture | Decision it supports |
| Single reference question | One user submits a short question with a verified answer | Retrieval time, TTFT, total time, and correctness | Establishes the basic service baseline |
| Long-context question | Repeat with a longer approved set of retrieved passages | Prompt tokens, TTFT, peak memory, and citation accuracy | Tests context and memory requirements |
| Overlapping requests | Start with 2 and 4 simultaneous users, then test the observed peak | Queue time, p95 completion time, errors, and memory use | Helps size request scheduling and concurrency |
| Background indexing | Refresh selected documents while queries are running | Parsing load, storage I/O, and changes in response time | Shows whether tasks should be scheduled separately |
| Remote storage access | Repeat an indexing run using the approved NAS or file share | Link throughput, disk latency, and total ingestion time | Shows whether the storage connection is limiting progress |
| Access separation | Have users with different document permissions ask about restricted content | Retrieved passages, denied access, and audit records | Checks access control, not GPU capacity |
These are proposed tests, not results from the Reddit office. Agree on accuracy and response-time targets, repeat trials, and keep failed cases. A fast answer with a wrong citation still fails.
The Ollama FAQ describes request queues and configurable parallel processing. Its documented default for OLLAMA_NUM_PARALLEL is one, but the setting can be increased, with additional memory requirements. The Ollama context-length documentation explains how context settings affect memory use. vLLM's production metrics include queue time, TTFT, end-to-end latency, and KV cache utilization. Verify the versions in the actual deployment rather than relying on a general claim that a particular runtime can or cannot serve multiple requests.
Use the results to target the remedy. Parsing problems need extraction work; slow retrieval calls for index, RAM, or storage checks; slow generation may warrant runtime tuning or a GPU. Occasional indexing can often move outside working hours.
Build the Hardware BOM Around Verified Workloads
The BOM should address requirements the current system cannot meet. The Reddit budget does not necessarily cover backup, power, warranty, integration, or ongoing support.
The main choices are to reuse existing equipment, add the inference capacity that testing requires, or fix a measured storage or network bottleneck. Treat security separately.
| Verified finding | Preferred hardware direction | What must be confirmed before quoting |
| Current host meets ingestion, retrieval, and response targets | Keep the existing infrastructure | Spare capacity, backups, vendor support, and permissions |
| Current host handles documents but cannot meet required local inference performance | Consider a supported GPU upgrade or a separate inference server | Model and runtime compatibility, VRAM, KV cache, PCIe, power, cooling, and measured load |
| Compute is adequate, but tests find a storage or inter-server link bottleneck | Review NICs, storage interfaces, and switch connectivity where needed | Interfaces at both ends, switch software, cabling or optics, link speed, recovery behavior |
| Current firewall supports the required access controls and capacity | Retain the firewall and adjust approved rules | User and API paths, management access, logs, and backup |
| The required isolation or capacity is beyond the existing firewall | Evaluate a suitable security gateway | Features, licenses, inspection workload, supported throughput, and change scope |
For a server quote, include model and runtime, quantization, context, tested concurrency, RAM/VRAM, storage, and expansion requirements. Verify GPU support, power, and cooling before treating an add-in card as a simple upgrade.
Map traffic between devices. A single host may need only its existing NIC. A NAS or separate inference host may need more bandwidth. Choose cables and optics only after checking both interfaces, distance, and supported protocols.
Also list what the project does not need. The facts in this discussion do not establish a requirement for a 25GbE or 100GbE fabric, a dedicated SAN, extra racks, or a replacement firewall.
Provide the current and proposed topology, workload measurements, vendor support checks, storage and backup plan, and a BOM marked reuse, required now, or optional later. Confirm stock, firmware, licensing, and warranty when requesting a quote.
For an initial Private AI hardware and compatibility review, share the current server and switch models, runtime, document mix, storage location, and concurrency results. Network-Switch can review interfaces and prepare a compatible shortlist without assuming a full replacement.
Request an Initial Private AI Hardware & Compatibility Review.
Frequently asked questions (FAQs)
Does a ten-person office need a dedicated GPU server for RAG?
Not necessarily. It depends on whether inference must run locally, the selected model, memory requirements, context size, overlapping requests, and the performance of the existing host. Test those conditions before ordering new hardware.
Will 500 PDFs fit in a small vector database?
They might. File count alone cannot answer that question. The index depends on extracted content, chunk size, embedding dimensions, metadata, and the database design. Build a representative index and measure its growth and search quality.
Can embedding and LLM inference run on different hardware?
Yes, if the software supports separate services and the access policy permits it. Embeddings can often run as scheduled batch work while inference serves live requests. Separate hardware adds another connection and something else to maintain, so measure the benefit first.
Is 1GbE sufficient for a small private RAG deployment?
It can be, particularly when the main components share one server and users mostly exchange text. Large imports, backups, NAS access, and traffic between servers may change that answer. Measure those paths before planning 10GbE.
How many concurrent requests should be tested for ten users?
Begin with one request, then test realistic overlaps and extend to the busiest pattern you observe. Record queue time, p95 response time, answer accuracy, and peak memory. Do not assume all ten staff members are generating answers simultaneously.
What should be sent for an initial server and network BOM review?
Provide current server and switch models, available RAM and storage, whether processing must stay local, the selected model, representative documents, expected peak requests, test results, and any access restrictions. That is enough to begin a useful compatibility review without guessing at a complete equipment list.
Sources and References
- r/LocalLLaMA, Looking for advice for small office looking for local AI RAG (September 2026). Original question and community replies. The thread does not establish a final purchase or measured outcome.
- Cisco Unified Edge for Red Hat Edge Design Guide (updated January 26, 2026). Edge compute architecture reference, not a performance benchmark for this office.
- Open WebUI, Scaling Open WebUI. Single-instance deployment and the extra requirements of scaling horizontally.
- Open WebUI, Groups and Resource Access. Documented group and knowledge-resource permissions.
- Unstructured, PDF Partitioning. PDF text extraction, OCR, and parsing options.
- Ollama, FAQ. Model loading, parallel requests, and queue behavior.
- Ollama, Context Length. Context settings and their memory implications.
- vLLM, Production Metrics. Monitoring queue time, response latency, and KV cache use.