NVIDIA A100 Tensor Core GPU
Get it at affordable price

A100 adds a powerful new third-generation Tensor Core that boosts throughput over V100 while adding comprehensive support for DL and HPC data types, together with a new Sparsity feature that delivers a further doubling of throughput.

New TensorFloat-32 (TF32) Tensor Core operations in A100 provide an easy path to accelerate FP32 input/output data in DL frameworks and HPC, running 10x faster than V100 FP32 FMA operations or 20x faster with sparsity. For FP16/FP32 mixed-precision DL, the A100 Tensor Core delivers 2.5x the performance of V100, increasing to 5x with sparsity.

New Bfloat16 (BF16)/FP32 mixed-precision Tensor Core operations run at the same rate as FP16/FP32 mixed-precision. Tensor Core acceleration of INT8, INT4, and binary round out support for DL inferencing, with A100 sparse INT8 running 20x faster than V100 INT8. For HPC, the A100 Tensor Core includes new IEEE-compliant FP64 processing that delivers 2.5x the FP64 performance of V100.

The NVIDIA A100 GPU is architected to not only accelerate large complex workloads, but also efficiently accelerate many smaller workloads. A100 enables building data centers that can accommodate unpredictable workload demand, while providing fine-grained workload provisioning, higher GPU utilization, and improved TCO. The NVIDIA A100 GPU delivers exceptional speedups over V100 for AI training and inference workloads as shown in Figure 2. Similarly, Figure 3 shows substantial performance improvements across different HPC applications.

The NVIDIA A100 GPU is a technical design breakthrough that is fueled by five key innovations, and the outcome of these innovations result in 6x higher performance than NVIDIA’s previous generation Volta architecture for training and 7x higher performance for inference. The innovations are mentioned below:

1. NVIDIA Ampere Architecture: At the core of this chip is the NVIDIA Ampere GPU architecture that contains more than 54 billion transistors and hence, making it the largest 7-nanometer processor in the world


2. Third-generation Tensor Cores with TF32: The A100 includes new TF32 for artificial intelligence that allows for up to 20x the AI performance of FP32 precision, without making any changes in the code.


3. Multi-Instance GPU: Multi-instance GPU or MIG is a new technology that allows multiple networks to operate simultaneously on a single A100 GPU for optimal utilization of computing resources. It enables a single A100 GPU to be partitioned into as many as seven separate GPUs in order to deliver varying degrees of computing for jobs of different sizes while providing optimal utilization as well as maximizing return on investment.


4. Third-Generation NVIDIA NVLink: The 3rd-generation NVIDIA NVLink helps in doubling the high-speed connectivity between GPUs, and thus provide efficient performance scaling in a server.


5. 5.Structural Sparsity: Structural sparsity is a new technique that harnesses the inherently sparse nature of AI math to double the performance.