I recently linked to CalDigit’s line of new Thunderbolt 5 docks with excitement. Then, my buddy Jim Metzendorf sent me a link to this press release about the products. Just soak this in:
CalDigit’s new Element 5, is more than just a next-gen connectivity device—it’s a performance backbone built for the future of AI. With three fully-featured downstream Thunderbolt 5 ports and up to 120 Gbps of bandwidth, the Element 5 enables local developers and researchers to run large language models (LLMs) like Llama, Mistral, and Phi-2 at full speed, entirely offline.
Ehhhh…
Running quantized or full-precision models locally requires GPU horsepower—and the Element 5 delivers by enabling external GPU (eGPU) connectivity at full PCIe Gen 4 speeds. Whether you’re pairing a MacBook Pro with a high-end RTX 4090 in an enclosure, or distributing compute across multiple eGPUs, the Element 5’s three Thunderbolt 5 ports give you the freedom to scale.
This is particularly useful for local LLM inference, where token-per-second performance can double or triple with the right GPU pipeline in place. With Thunderbolt 5’s rock-solid throughput, model loading and execution remain fluid and responsive.
A couple of things here:
- Apple silicon Macs do not support eGPUs.
- Intel Macs that included this feature only supported AMD GPUs, so that RTX 4090 isn’t going to do you any good.
They go on:
Modern LLMs aren’t light. Models like Mistral-7B or fine-tuned Llama derivatives often require tens of gigabytes just to load into memory, not including embeddings, vector databases, or training datasets. The Element 5 supports Thunderbolt-connected NVMe SSDs that provide read/write speeds exceeding 3,000 MB/s, making it ideal for:
- Storing and loading models quickly
- Streaming large training or inference datasets
- Running multiple AI tools without storage lag
🤷♂️