Overview
For highly sensitive research, ITS is in the early stages of building support for running open-source AI models via LM Studio and oMLX on local hardware. Unlike every other CMC AI tool, local models process all data entirely on-device — no data ever leaves your machine.
This approach is required for projects involving P4 Restricted data, controlled unclassified information (CUI), or any research where cloud transmission is prohibited by data governance requirements.
Zero Network Transmission
All inference runs on-device. No prompts, data, or outputs ever reach the internet.
Runs on Your Hardware
Models run on the machine in front of you — an M-series Mac, a GPU workstation, or ITS-hosted hardware.
Full Transparency
Open-source models — you can inspect the model architecture, weights, and code.
P4 Restricted Capable
The only CMC AI pathway approved for use with P4 Restricted and highly sensitive research data.
Available Tools
LM Studio
A user-friendly desktop application for downloading, running, and experimenting with open-source language models. Features a chat interface, a model library browser, and configurable inference parameters. Best for users who prefer a graphical interface without command-line setup.
oMLX
An open-source inference server built specifically for Apple Silicon and managed from the macOS menu bar. It runs models through Apple’s MLX framework with continuous batching and a tiered KV cache that spills from memory to SSD, and it exposes OpenAI- and Anthropic-compatible endpoints on localhost — so existing scripts and tools can point at a local model without code changes. It also serves vision, embedding, and reranker models alongside text. Requires macOS 15 (Sequoia) or later on an M-series Mac.
Recommended Models
| Model | Size | Best For | Min RAM |
|---|---|---|---|
| Gemma 4 E4B | ~5 GB 8B total / 4.5B effective |
On-device text, image, and audio; laptops and edge use. 128K context. | 8 GB |
| Gemma 4 12B | ~7 GB 12B dense |
General reasoning and multimodal work on a standard laptop. 256K context. | 16 GB |
| Qwen 3.8 27B | ~16 GB 27.8B dense |
Vision-language analysis, long documents, video input. 262K context. | 32 GB |
| Laguna S 2.1 | ~60 GB 118B MoE / 8B active |
Agentic coding and long-horizon research tooling. 1M context. | 96 GB |
| Qwen 3.8 Flash Next | ~94 GB 176B MoE / 6B active |
High-throughput multimodal reasoning; sparse activation keeps it fast. 262K context. | 128 GB |
| GLM 5.3 Flash | ~160 GB 320B MoE / 18B active |
Frontier-scale multimodal reasoning over text, image, and video. 1M context. | 192 GB |
| DeepSeek V4 Flash (0731) | ~155 GB 284B MoE / 13B active |
Coding, chat, and agent workflows with adjustable reasoning effort. 1M context. | 192 GB |
Sizes are approximate 4-bit quantized weights; add headroom for context and the operating system. “Active” parameters describe compute per token, not memory — a mixture-of-experts model still has to hold all of its weights in memory.
How to Get Started
General Users — Download LM Studio
Download LM Studio from lmstudio.ai on your institutional device. Browse the model library and select a model suited to your needs. Start with Gemma 4 E4B or Gemma 4 12B before trying larger ones.
Apple Silicon Users — Install oMLX
On an M-series Mac running macOS 15 or later, install oMLX with brew install jundot/omlx/omlx or the DMG from the project’s releases page. It serves models on localhost and is managed from the menu bar, so scripts and notebooks can call a local model through the same API they already use.
Larger Models — Request CMC-Hosted Access
Models above roughly 100B parameters need more memory than a laptop provides. Contact the ITS Infrastructure team to discuss access to CMC-hosted hardware, and ITS will advise on approved handling procedures for your data classification level.
Run and Iterate
Use specific, detailed prompts — local models may be less powerful than large cloud models, so prompt quality matters more. Monitor system performance and close unnecessary applications to maximize available RAM.
Hardware Requirements
Up to ~27B
16–32 GB RAM or unified memory
20–60 GB storage
Apple Silicon, or a CPU with a dedicated GPU
Runs Gemma 4 E4B, Gemma 4 12B, Qwen 3.8 27B
100B-class MoE
96–192 GB unified memory
80–250 GB storage
High-memory M-series Mac or multi-GPU workstation
Runs Laguna S 2.1, Qwen 3.8 Flash Next
Frontier open models
Up to 512 GB unified memory
80-core GPU
Provisioned and managed by ITS
Runs GLM 5.3 Flash, DeepSeek V4 Flash (0731)
Data Security
-
Zero Network Transmission
Local models process all data on-device. No prompts, responses, or documents are ever transmitted to the internet or any external service.
-
Air-Gapped Option for P4 Data
ITS maintains dedicated workstations with no network connection for maximum isolation. Required for P4 Restricted data including certain health records, CUI, and highly sensitive research.
-
No Vendor Data Sharing
There is no vendor involved in inference — open-source models run entirely under your control. No usage telemetry, no data sharing, no third-party involvement.
-
Model Integrity
ITS only approves installation of signed, verified models from vetted sources. Model provenance is documented for compliance.
Ready to Try Local Models?
Contact the Solutions team to discuss your research data classification requirements and get provisioned on the appropriate local AI infrastructure.
Request Local AI Access → Back to LLMs