Quotes from Japan's Supercomputer Fugaku article in IEEE Spectrum (6-29-2020)-
Another plus for Fugaku is that it follows the K computer by maintaining an all-CPU design. Shimizu says this makes memory access and CPU interconnectivity more efficient. Most other supercomputers rely on graphic processing units (GPUs) to accelerate performance.
Dongarra points out an additional benefit here: “A CPU-only system simplifies the programming. Just one program is needed, not two: One for the CPU and one for the GPU.”