The Best Hardware for Machine Learning

Do you want to learn more about machine learning hardware?

State-of-the-art machine learning models require serious processing power. More complex AI learning methods, like deep learning (DL), can provide even more interesting results — but the amount of processing power they need jumps up significantly.

If you regularly experiment with ML/DL, you may want to build a high-power computer dedicated to running these models.

For a special-use rig with a lot of power, there are also some special considerations that you’ll need to keep in mind — like disk transfer speed, cooling and power supply.

In this post, we’ll break down the key components of a computer built for deep and machine learning, and show you how to select the best parts for the job.

GPU

You’ll need both a GPU and CPU for your machine, but the GPU will provide most of the processing power for your build.

Why focus on the GPU instead of the CPU? Modern, low to high-grade CPUs have anywhere between 2 and 32 cores, each of which can work on one task at a time. GPUs, by contrast, are built with many more, smaller-sized logical cores — with state-of-the-art cards having as many as — making them effective for performing a large number of simpler tasks in parallel.

GPUs are also typically designed to handle computations necessary for generating computer graphics — frequently matrix and vector operations — which are similar to the operations your computer will perform when running deep learning models.

As a result, GPUs can be as much as four to five times faster than CPUs when running deep learning models.

Other, processing power-intensive builds that need to make many parallel computations, like computers built for mining bitcoin, also rely heavily on GPUs. This is why the price of GPUs often fluctuates along with the rise and fall of the bitcoin market.

Tips for Selecting a GPU

When selecting a GPU, a few specifications will be the most important. A good amount of memory (VRAM) is critical. At a minimum, you’ll want a GPU with around 8 to 11 GB of memory and a high memory bandwidth. GPUs with these specs, however, aren’t cheap. A GPU with 8 GB of VRAM, like the RTX 2070 or GTX 1080 Ti, will cost between $500 and $800.

GPUs with more memory will be even more effective and expensive. An RTX 8000, which has 48 GB of  memory, will cost around $5,500.

With enough PCIe slots, you can use multiple GPUs in the same rig. Most deep learning builds will have between two to four GPUs. Keep in mind, however, that extra GPUs provide diminishing returns. A second GPU will offer a 20 to 90 percent boost in performance. A third may only provide a 50 percent boost.

Some manufacturers, like Nvidia, no longer support three- or four-way SLI, meaning you may be limited to two GPUs depending on which cards you select.

Storage

Storage likely won’t be a bottleneck — you need some, but nothing extravagant. The right storage, however, can make your build a little more efficient.

You have two types of storage to choose from — solid state drives (SSDs) and the more conventional hard disk drives (HDDs). SSDs have faster read/write times than HDDs and are more reliable, but they’re also more expensive.

RAM

RAM will only have an impact on your build’s ability to run deep learning models if you don’t have enough. You should have at least enough RAM to match the VRAM of your most powerful GPU — 1 GB of RAM for each GB of VRAM on that GPU.

Because RAM is modular, upgrading isn’t too difficult, so long as you have extra empty RAM slots. If you feel like matching your GPU’s VRAM will bottleneck you, consider purchasing some extra RAM.

CPU

Your CPU will support your GPU(s). For a deep learning build, you’ll want a higher-end CPU with multiple cores — at least 8. Consumer-grade CPUs with 16 cores exist, and will almost guarantee that your CPU doesn’t bottleneck system performance, but power always comes with a price tag.

There are CPUs with greater than 16 cores available, but they are either cutting-edge consumer equipment or manufactured for use in servers. Both kinds are very expensive.

Motherboard

You’ll need to select a motherboard that can support both your CPU and GPU(s).

For each GPU you have, you’ll need a PCIe slot on your motherboard. Most high-end GPUs will require a slot with a minimum of 8 PCIe lanes. Theoretically, you can get a performance boost by using a slot with a greater number of PCIe lanes, but the difference won’t be significant.

Your motherboard’s CPU socket will also need to be compatible with your CPU.

Other Hardware to Consider

You’ll also need a power supply unit (PSU) that can deliver enough wattage to all of your system’s components. You’ll need to calculate this wattage using the specifications of the other components for your build. There are few calculators online, like Newegg’s, that can help you determine how much power you need from your PSU.

Cooling will likely be an issue — even mid-grade GPUs use a lot of power and produce serious amounts of heat.

Air-cooling with fans is the usual approach for cooling. Because a ML/DL relies on such powerful GPUs, however, you’ll likely need to pay close attention to fan design with this approach.

Water-cooling is also a possibility. A functional water-cooling system may be harder to put together and more expensive than an air cooling system. A water-cooling will be much, much more effective at removing heat than air cooling, however. If you have two or more GPUS or more in your finished build, you should seriously consider water-cooling.

You’ll also need a case big enough to fit your components, and select peripherals — mouse, keyboard and monitors — for your preferred workstation configuration.

Choosing the Right Hardware for Machine and Deep Learning

Deep and machine learning requires some serious hardware. All of the parts listed above will be important. You’ll want to put the most focus, however, on choosing your GPU, which will provide the power for your machine. Your GPU will also likely be the most expensive component of your build.

Aside from these components, you’ll also need to consider how you’ll cool your system and which peripherals you’ll want.