In short
A GPU is a processor that executes thousands of operations in parallel and is used to train and run AI models.
Large language models and vision models usually need GPU servers to respond in reasonable time. Required GPU memory depends on model size, concurrent users and the latency target.
In on-premise AI projects, GPU selection should be planned together with power consumption, cooling and redundancy.