Visual Studio Code does come with in integrated Git source control provider. However, in order for that to work, Git itself needs to be installed on your system as well. So just download and install Git on your machine, and Visual Studio Code will also be able to use it.
While the source control panel does not tell you this, but unfortunately just stays quiet about it, not offering anything, there is a way to have VS Code tell you this: On the “Welcome” page that is displayed when you start the application, there is a link “Clone Git repository…”. If you click there, VS Code will tell you that Git is missing:

Once you have installed Git and restarted Visual Studio Code, you will see a small Git icon at the top of the source control panel (unless you already have a repository opened of course):

This will tell you that Git is there and you can click the icon to initialize a repository—or just open an existing repository.