Rather than answering it in the context of “Go”, I would answer this question in the context of any language (e.g. C, C++, Go) which implements the concept of "pointers"; and the same reasoning can be applied to “Go” as well.
There are typically two memory sections where the memory allocation takes place: the Heap Memory and the Stack Memory (let’s not include “global section/memory” as it would go out of context).
Heap Memory: this is what most of the languages make use of: be it Java, C#, Python… But it comes with a penalty called the “Garbage Collection” which is a direct performance hit.
Stack Memory: variables can be allocated in the stack memory in languages like C, C++, Go, Java. Stack memory doesn’t require garbage collection; hence it is a performant alternative to the heap memory.
But there is a problem: when we allocate an object in the heap memory, we get back a “Reference” which can be passed to “multiple methods/functions” and it is through the reference, “multiple methods/functions” can read/update the same object(allocated in the heap memory) directly. Sadly, the same is not true for the stack memory; as we know whenever a stack variable is passed to a method/function, it is “passed by value”(e.g. Java) provided you have the “concept of pointers”(as in the case of C, C++, Go).
Here is where pointers come into picture. Pointes let “multiple methods/functions” read/update the data which is placed in the stack memory.
In a nutshell, “pointers” allow the use of “stack memory” instead of the heap memory in order to process variables/structures/objects by “multiple methods/functions”; hence, avoiding performance hit caused by the garbage collection mechanism.
Another reason for introducing pointers in Go could be: Go is ought to be an "Efficient System Programming Language" just like C, C++, Rust etc. and work smoothly with the system calls provided by the underlying Operating System as many of the system call APIs have pointers in their prototype.
One may argue that it can done by introducing a pointer-free layer on top of the system call interface. Yes, it can be done but having pointers would be like acting very close to the system call layer which is trait of a good System Programming Language.