Phison aiDAPTIV+: Hybrid AI Solution Combines SSD Caching and GPUs for Training Large Language Models

Phison has recently showcased its aiDAPTIV+, a hybrid hardware and software solution designed to make large-language model (LLM) training more cost-effective and accessible, leveraging NAND flash technology to enhance efficiency. AI workstations and servers use it to support models like Llama, Falcon, Whisper, and ResNet.

The Hybrid AI hardware solution allows small and medium-sized businesses to train significantly larger AI models beyond previous limitations, scaling up to Llama-3 70B and Falcon 180B. It features seamless integration with GPU memory, utilizing aiDAPTIVCache for low-latency processing. It also includes high-endurance AI100E SSDs with a 100 DWPD rating for durability. Besides, with a focus on security, ease of deployment, and cost savings, aiDAPTIV+ is ideal for AI applications in industries requiring on-premises data control and scalable model training.

This design enables organizations to train complex models like Llama-2 7B on standard off-the-shelf hardware while ensuring low-latency processing and high-endurance storage (100 DWPD) with SLC NAND.

Additionally, the aiDAPTIV+ middleware optimizes data flow between GPU memory and SSDs by assessing AI model requirements, detecting inactive data segments, and offloading them to SSDs. This approach expands the GPU’s available memory, enabling the training of significantly larger models than its built-in capacity would allow.

Previously, we have covered some NVME SSDs like WD Blue SN500 and Greenliant SATA M.2 ArmourDrive. These two are not anything like the Phison aiDAPTIV+.

Phison aiDAPTIV+ Hybrid AI Solution Specifications:

• Processor: Not specified (depends on workstation configuration)
• GPU: Up to 4x RTX 6000 Ada
• GPU Memory (GDDR): 192GB
• Memory (RAM): 512GB
• Storage:
  • AI100E SSD (High-endurance, 100 DWPD over 3 years)
  • 2x 2TB aiDAPTIVCache for GPU memory extension
• Supported AI Models:
  • LLMs: Llama, Llama-2, Llama-3, CodeLlama, Vicuna, Falcon
  • Vision and Audio: Whisper, Clip Large, ResNet, Metaformer
  • Others: TAIDE, Deit base, Mistral, and more
• AI Framework: PyTorch
• Middleware: Optimized for GPU memory extension
• Scalability: Supports single-node AI training, expandable by adding nodes
• Latency: Low-latency SSD caching for large models (e.g., Llama-3 70B)
• Security: On-premises AI training for better data privacy
• Power: Not Specified (1000W or more, depending on the full system configuration)
• Dimension: TBD

According to the company, they have a technique similar to OS-level swapping, where less frequently used data is offloaded from RAM to storage to free up memory for active processes. Likewise, aiDAPTIV+ utilizes high-speed, high-capacity AI100E M.2 SSDs as aiDAPTIVCache, effectively extending GPU memory. Engineers specifically optimize these SSDs to handle the intensive demands of AI workloads, ensuring efficient data management and seamless model training.

The above chart compares training times for large language models—7B, 13B, and 70B—using traditional methods versus the aiDAPTIV+ system. It highlights that training a 70B model with 4x RTX 6000 Ada GPUs alone would fail, while aiDAPTIV+ completed it in 4.41 hours. This performance boost comes from Phison’s middleware, which optimizes GPU memory by segmenting the model. However, inactive segments are temporarily stored in aiDAPTIVCache and efficiently swapped between the cache and GPU as needed.

ADLINK’s new DLAP Supreme series integrates Phison’s aiDAPTIV+ technology to overcome memory and performance constraints in edge AI devices. This series delivers 8x faster inference speeds, supports 4x longer token lengths, and enables large language model training on devices with limited memory, such as the NVIDIA Jetson AGX Orin 32GB running the Gemma 27B model. By leveraging aiDAPTIV+, edge devices can now handle AI tasks that typically require high-end GPUs like the H100 or A100. Companies including ADLINK, Advantech, ASRock, ASUS, and GIGABYTE have already integrated this technology into their AI solutions.

The Phison aiDAPTIV+ AI100E M.2 2280 1TB is listed on Newegg but is currently out of stock. You can visit the product page for more information.

Please follow and like us: