RassyCodex is the local OpenAI-compatible inference and retrieval fabric for Ian's stack. It gives Hermes, OpenFang, Qwen Code, Aider, OpenCode, Open WebUI, and compatible clients stable aliases, request normalization, routing, queues, session affinity, retrieval services, media endpoints, health, metrics, and safe upstream proof. The stack includes high-context coding lanes on the paired V100 cards and RTX 2080 Ti media work for image, STT, and TTS. It is intentionally separate from the stopped RassyGPT Runtipi app.
Stable logical aliases front local model runners. Clients ask for names like rassy-agent, rassy-general, rassy-code-deep, rassy-code-agent, rassy-utility, rassy-embed, rassy-rerank, rassy-image, rassy-stt, and rassy-tts.
The general and deep reasoning path is rassy-general and rassy-code-deep to rassy-codex, backed by qwen3.6:35b on the paired 32 GB V100 cards with the high-context policy used for the hardest coding and operator work.
rassy-code-agent is the dedicated North Mini Code route for coding, repository editing, tool use, and long-horizon agent loops on the paired 16 GB V100 cards.
Embeddings, retrieval, reranking, provenance, and P100 emergency embedding failover stay in RassyCodex. Memory choice and prompt assembly stay with Hermes or the calling agent.
The RTX 2080 Ti lane is treated as a queued media appliance for images, speech recognition, and speech synthesis rather than an unbounded shared scratchpad.
Hermes, OpenFang, coding agents, Open WebUI, or a curl client sends an OpenAI-compatible request.
RassyCodex authenticates, normalizes protocol fields, resolves aliases, applies queue and session rules, and records safe request metadata.
The chosen logical lane routes to Ollama, LocalAI, Qdrant, or the local reranker with bounded backpressure and no silent streamed retry.
Responses include route headers such as request ID, logical route, upstream, queue timing, gateway version, and deprecation metadata where applicable.
Image, STT, and TTS traffic share the RTX 2080 Ti lane with explicit queue limits, backpressure, cancellation behavior, and separate public aliases so clients know what they are asking for.
This table is generated from the live route configuration. Preferred aliases are listed by /v1/models and tested capabilities by /v1/capabilities; legacy aliases remain for compatibility while clients migrate.
| Model | Kind | Backend | Model Used | Hardware | Description |
|---|---|---|---|---|---|
rassy-codex | chat | codex | qwen3.6:35b | 2x Tesla V100-PCIE-32GB | Qwen3.6 35B high-context coding and OpenFang operator lane on the paired 32 GB V100 cards. Features: agentic coding, thinking preservation, tool use, vision-capable upstream |
rassy-codex-lite | chat | codex_lite | qwen3.6:35b | 2x Tesla V100-PCIE-32GB lower-context coding lane | Lower-context Qwen3.6 coding lane on the paired 32 GB V100 cards for normal-sized agent work. Features: lower-context coding, isolated normal-sized coding queue, faster agent loops |
rassy-code-agent | chat | general | north-mini-code-1.0:q4_K_M | 2x Tesla V100 16GB | North Mini Code 1.0 dedicated coding-agent lane on the paired 16 GB V100 cards. Features: agentic coding, coding agent, repository editing, tool use, long-horizon repo work, interleaved thinking |
rassy-fast | chat | fast | unsloth/Qwen3-8B-GGUF / Qwen3-8B-Q4_K_M.gguf | Tesla P40 24GB utility chat lane | Qwen3-8B utility chat lane on the P40 for summaries, routing, compaction, and former worker aliases. Features: utility summaries, routing, compaction, compatibility worker aliases |
rassy-embed | embeddings | embed | qwen3-embedding:8b | Tesla V100-PCIE-12GB primary embedding lane | Ollama Qwen3-Embedding-8B document and memory embedding lane on the V100 12GB, with P100 standby after primary failure or bounded primary stall. Features: 4096-dim Ollama document embeddings, OpenFang memory retrieval, P100 emergency failover after failure or bounded stall |
rassy-embed-query | embeddings | embed_query | qwen3-embedding:8b | Tesla V100-PCIE-12GB shared query/document embeddings | Query embeddings share the V100 12GB embedding service with RassyCodex retrieval instructions and P100 standby after primary failure, bounded primary stall, or query-only overflow pressure. Features: query-prefixed embeddings, same V100 12GB service as document embeddings, P100 emergency failover after failure or bounded stall |
rassy-embed-p100 | embeddings | embed_p100 | qwen3-embedding:8b | Tesla P100 16GB emergency embedding failover | Conservative Qwen3-Embedding-8B standby on the P100; explicit use or automatic failover only. Features: 4096-dim Ollama embeddings, emergency failover, P100 co-resident with rerank |
rassy-rerank | rerank | rerank | ExpedientFalcon/Qwen3-Reranker-4B-Q4_K_M-GGUF / qwen3-reranker-4b-q4_k_m.gguf | Tesla P100 16GB | Qwen3-Reranker-4B Q4 lane on the P100 for retrieval self-tests, with Qwen embedding cosine fallback. Features: retrieval reranking, cosine fallback |
rassy-image | images | image | flux.2-klein-4b | RTX 2080 Ti 11GB media lane | LocalAI image/photo generation lane on the RTX 2080 Ti. Features: local photo/image generation, RTX 2080 Ti media lane |
rassy-audio | audio | audio | whisper-large-turbo-q8_0 / qwen3-tts-cpp | RTX 2080 Ti 11GB media lane | LocalAI STT/TTS lane on the RTX 2080 Ti. Features: local STT, local TTS, RTX 2080 Ti media lane |