Gojira v3.0
Gojira 2.0 has similar parts as Bear 6.0, but a bit newer and faster.
In addition, quantity two: ZOTAC GAMING GeForce RTX 3060 Twin Edge GPUs with 12 GB GDDR6 VRAM, PCIe 4.0 x16, 3 x DisplayPort, HDMI.
The 2 video cards are not connected to any monitors. Instead, the integrated CPU/GPU drives the monitor, which allows the two NVIDIA 3060s to be dedicated to compute tasks.
You can use two ZOTAC ZT-A30600H-10M cards in the MSI PRO Z790-A WIFI motherboard for running large language models.
The most straightforward, cost-effective, and highest-performance option is to install the second ZOTAC GeForce RTX 3060 directly into the second physical PCIe 4.0 x16 slot on your motherboard. The slot runs electrically at x4 speeds, which provides the necessary bandwidth for the card to function effectively for your LLM tasks.
To clarify, the second Zotac card can be used in your “Bear” computer system for LLM tasks, but not in the way that traditional gaming multi-GPU (SLI/NVLink) systems worked.
Multi-GPU for LLMs (Compute Tasks)
The key difference is that SLI/NVLink is a hardware bridge designed specifically for gaming (splitting frame rendering), which the RTX 3060 does not support. For AI, machine learning, and LLM workloads, the software handles the parallel processing differently:
VRAM Stacking/Pooling is Possible (via software): You can use libraries in frameworks like PyTorch or Hugging Face’s Accelerate which use model parallelism (or pipeline parallelism) to automatically split the model layers across the VRAM of both cards.
This means you effectively get access to nearly 24GB of total VRAM (12GB + 12GB), which is a huge advantage for loading larger LLMs that wouldn’t fit on a single 3060 or even a single 3060 Ti.
The communication between the GPUs happens over the motherboard’s PCIe lanes (the second slot runs at x4 speed). This connection is slower than NVLink, which means there is a performance penalty (data transfer time between cards), but the workload can still be completed. You can expect performance somewhat less than double that of a single card, but significantly more capable in terms of model size.
The cards will physically fit into the two PCIe x16 slots. You need specific software configuration (e.g., using device_map='auto' in Hugging Face or specific PyTorch distributed training configurations) to manage the memory across both GPUs. The primary benefit is the combined 24GB VRAM capacity, allowing you to run much larger models than a single GPU can handle.
The MSI MAG Z790 TOMAHAWK WIFI has two full-length PCIe x16 slots.
The top slot is a reinforced PCIe 5.0 x16 slot (running at x16 speed when used alone). The bottom full-length slot is wired for PCIe 4.0 x4 speeds, meaning it has the physical slot size for a large card but uses fewer lanes. There is a PCIe x1 slot located between the two x16 slots, which provides enough vertical separation to accommodate the standard dual-slot thickness of the ZOTAC GeForce RTX 3060 AMP White Edition cards.
would ubuntu os present any issues for this dual gpu configuration on this motherboard
Using Ubuntu with a dual NVIDIA RTX 3060 configuration on the MSI MAG Z790 TOMAHAWK WIFI motherboard should work without major operating system issues, but proper driver installation and software configuration are crucial. The primary challenges often lie in initial setup and software compatibility rather than the OS itself.
Potential Areas for Smooth Operation (or Issues)
The most common hurdle is ensuring the correct proprietary NVIDIA drivers are installed and recognized by the system.
Solution: Use Ubuntu’s built-in “Additional Drivers” tool or the recommended PPA method for AI/LLM work to install the correct, compatible drivers and CUDA Toolkit version for your specific Ubuntu release (e.g., Ubuntu 22.04 LTS).
Without the right drivers, tools like nvidia-smi might not detect one or both GPUs, resulting in a “No devices were found” error. BIOS Settings: The motherboard’s BIOS settings can sometimes interfere with Linux and GPU detection.
Solution: Ensure Secure Boot is disabled in the BIOS, as this can sometimes prevent proprietary NVIDIA drivers from loading correctly. Also, check for and potentially disable features like Intel VMD if you encounter SSD detection issues during installation.
The way the software utilizes the GPUs is critical for performance.
Solution: Software frameworks like PyTorch or tools like Ollama and LM Studio can manage memory and split model layers across both GPUs for LLM tasks without needing SLI or NVLink.
Performance might not scale perfectly due to data transfer overhead across the PCIe lanes (the second slot runs at a slower x4 speed). For daily desktop use, managing which GPU drives the display can sometimes be tricky with multi-GPU setups.
Solution: Use the NVIDIA settings panel in Ubuntu to configure your primary display output. For compute tasks, both GPUs are generally available regardless of which one is connected to a monitor.
Overall, Ubuntu is a popular and robust choice for AI and deep learning workloads and is well-supported by NVIDIA’s driver ecosystem. The configuration is achievable with careful setup.