Psilon democratizes AI access for consumers. It removes the NVIDIA ecosystem dependency from AI by porting PyTorch to open, standardized GPU APIs — Vulkan, DirectX 12, and WebGPU — and it makes frontier-scale models runnable, unmodified, on ordinary consumer gaming hardware.
Today, serious frontier AI models run on one vendor's hardware, through one vendor's software stack. If your GPU doesn't speak CUDA — or your model doesn't fit in your VRAM — you're locked out. Psilon enables everyone to participate in the singularity.
A highly optimized PyTorch backend for Vulkan, DX12, and WebGPU that aims to match or exceed CUDA performance on the same silicon. AMD, Intel, NVIDIA, Apple, and the browser — one model, any GPU.
General-purpose AI model streaming: frontier models run in their unmodified entirety on low-cost hardware by streaming layers through the GPU instead of holding the whole model in VRAM.
First release · Pillar 1
psiTorch is a highly optimized PyTorch library and backend for Vulkan,
DirectX 12, and WebGPU. It keeps the PyTorch name and API surface, so
existing models and code run unchanged — torch.device("vulkan")
works the way you'd expect.
The Vulkan port is often as fast or faster than CUDA on the same hardware — and it is positioned to pull further ahead as new drivers expose more tensor-core primitives to Vulkan and DX12.
| Operation | Vulkan vs. best CUDA |
|---|---|
torch.mm | 1.49× faster |
torch.matmul | 1.45× faster |
F.nll_loss | 1.0× (parity) |
| NVMe→GPU weight streaming | 11 GB/s |
Ratios vs. torch.compile(mode="max-autotune") + cuDNN SDPA on an
RTX 5090 Laptop GPU, Windows. Coverage is expanding: these are targeted
AI-critical operators, not yet the full operator surface.
2,691 ATen operator registrations, with matrix-core acceleration (tensor cores, AMD WMMA, Intel XMX) and automatic hardware tier detection.
A single shader codebase compiles to SPIR-V for Vulkan, DXIL for DirectX 12, and WGSL for WebGPU.
Not inference-only. psiTorch targets the full PyTorch experience — training and inference — all on open APIs.
First release · Pillar 2
Streaming an AI model does not require quantizing it, pruning it, or otherwise cutting it down to fit in limited VRAM. It is a new technique for processing frontier models in their unmodified entirety on modest consumer hardware. They may not be fast — but they work, today, uncompromised.
Streaming and quantization are complementary approaches to running AI models on consumer hardware. psiCache streams and caches AI models to enable models of any scale to run unmodified on consumer hardware. psiCache also supports adaptive quantization of model weights and KV cache values. Because streaming an AI model transforms an inherently compute bound problem into an IO bound problem on consumer hardware, abundant GPU cycles become available for adaptive quantization that enables psiCache to adjust weight and KV values at run-time to achieve a target perplexity goal. No more hand-tuning AI models, psiCache adaptive quantization works with all pyTorch based models out of the box.
First release · Pillar 3
Psilon ships a family of the top Hugging Face AI Python libraries — fused and optimized to perform better on CUDA, Vulkan, DX12, and WebGPU alike. High-overhead Python paths in the standard libraries are replaced with fused native GPU implementations, so the ecosystem you already use gets faster on every backend, including CUDA.
Keep your transformers-style workflows. The fused libraries
preserve the familiar APIs while eliminating interpreter overhead on the hot
path.
The same fused optimizations accelerate CUDA, Vulkan, DX12 and WebGPU backends. PsiTorch is a faster backend for all pyTorch workloads.
Video didn't become universal because everyone bought broadcast equipment — it became universal because it became streamable. Psilon does the same for AI models: any model, streamed to any machine, on open standards.
By transforming AI into a streamable consumer media type, Psilon enables everyone to participate in the singularity. AI is no longer a monopoly for a few big companies to define or control.
Signed Windows builds are published on the GitHub releases page. Every release
includes a SHA256SUMS.txt covering all assets — verify your
download before running anything. Mac and Linux builds are planned.
Source and issues live at github.com/stjohnalex/psilon.
Psilon is under active vibe development. Mac and Linux versions will be available soon as well as additional opensource as soon as I get a chance to review the code. The best place to reach me for business contact is via LinkedIn, or via Discord if you just want to join the revolution and help support the project.
Connect with Alex St. John.
linkedin.com/in/alexstjohn
Join the revolution and support the project.
discord.gg/NAQhQdn8t
Watch demos and deep dives.
youtube.com/@psilonai
Visit the Playcast site.
playcast.io
Source, releases, and issues.
github.com/stjohnalex/psilon