CPU VS GPU VS TPU, What is CUDA?

• CPU 

Central Processing Unit

 

• GPU

Graphics Processing Unit

- Graphics Processing Unit(GPU) is a microprocessor having hundreds of cores.

- GPU is a co-processor of CPU which speeds up the CPU performance by doing heavy computations, leads to faster results.

- It’s a parallel architectural processor which follows parallel programming model and made possible for running Deep Learning models efficiently and effectively.

- Some GPU manufacturers are as NVIDIA, AMD, ASUS, Intel etc. The top most is NVIDIA manufacturing its best GPUs for handling heavy computations by introducing its core technology called CUDA and thus CUDA cores got introduced.

- CUDA and NVIDIA GPUs dominates the market.

 

 

• TPU

Tensor Processing Unit, illustrates additional optimization over GPUs in how computation and memory are distributed for neural network execution. Google's custom-developed application-specific integrated circuits (ASICs) used to accelerate machine learning workloads.

 

• CUDA 

- NVIDIA’s GPU performs parallel computations by using its CUDA cores. These hundreds of cores in GPU is a parallel computing platform for performing intensive applications on GPU.

- Hundreds of cores made it superior than CPU in terms of bandwidth and computing time.

- Many Deep Learning technologies such as tensorflow, torch, CNTK, H2O.ai, Keras, Theano and PyTorch entrust CUDA for GPU support.

- GPU is been heavily utilized for Deep Learning models and heavy Machine Learning models which require intensive computations.