NVIDIA Created CUDA, But a Startup Aims to Make It Universal
For many developers of artificial intelligence and high-performance computing (HPC), CUDA has now become synonymous with NVIDIA GPUs. The language and its software ecosystem represent the de facto standard of the industry, with a reach that involves about 80% of the existing HPC code today.
A London-based startup, Spectral Compute, believes that this close association between software and hardware is not inevitable and has developed a solution aimed at separating the two worlds. Founded in 2018 by four engineers with a combined experience of about 60 years in HPC optimization, Spectral Compute emerged from dissatisfaction with the cost of NVIDIA GPUs and the limitations of available alternatives to run CUDA code on different platforms. From this experience, SCALE was born, a compiler developed using LLVM and Clang that aims to serve as a direct replacement for NVCC, NVIDIA's official compiler. This was discussed by HPCwire.
The approach chosen differs from that adopted by other tools already present in the market. Solutions like AMD's HIPIFY convert CUDA code into C++ intended for the ROCm ecosystem, while SYCLomatic, developed at Intel, automates migration to Data Parallel C++, still requiring manual intervention on part of the code. There are also projects like ZLUDA, which operate directly on compiled binaries, but introduce an intermediate layer that may penalize performance.
SCALE, on the other hand, follows a different philosophy: it recompiles the CUDA source code directly for the target architecture, maintaining a workflow similar to that of traditional compilers used in the CPU world. According to Spectral, this allows for performance differences primarily related to the hardware used rather than the compiler itself. After recompilation, the result is compared with that generated by NVCC to verify the numerical correctness of the execution.
The company claims that this methodology allows for significant performance improvements as well. In benchmarks published on its website, SCALE claims to have achieved increases of up to around six times compared to code conversion via HIPIFY on AMD GPUs. Although such results come directly from Spectral and have not been independently verified, they represent a central element of the startup's technological offering.
The initial goal was to support AMD GPUs, but the roadmap includes extending compatibility to other accelerators dedicated to artificial intelligence, with the involved manufacturers yet to be disclosed. At the same time, Spectral continues to support NVIDIA GPUs, asserting that there are still margins to further enhance performance through more advanced compilation techniques.
The challenge involves not only the CUDA language but also the vast ecosystem of libraries built over the years by NVIDIA. Tools like cuDNN, cuTENSOR, and cuDF are now essential for numerous AI and HPC applications, which is why Spectral is working to gradually expand compatibility with these libraries. Support for PyTorch is also planned, facilitating integration with AI workflows.
Despite the goal of reducing technological lock-in associated with NVIDIA hardware, Spectral emphasizes that it does not wish to compete directly with the American giant. In June, it joined the NVIDIA Inception program dedicated to startups and claims to collaborate with various industry players while maintaining a neutral position. The company currently has about 30 employees, markets SCALE to enterprises, and offers it for free to universities and nonprofit organizations.
The compiler has been on the market for about two years and has already been used on large systems such as Frontier, the exascale supercomputer at the Oak Ridge National Laboratory. For the research world and HPC developers, the ability to recompile existing CUDA applications without facing costly porting processes represents the main strength of the solution proposed by Spectral.