You can work with remote repositories without cloning the whole repository, yes. However you are limited in what you do:
To inspect a repository for available branches:
git ls-remote git://url/to/repository.git
To fetch a single branch:
git fetch git://url/to/repository.git branch
This will fetch the branch as FETCH_HEAD
you then need to checkout that branch, and can save it to a local branch (otherwise you don't have any direct reference to its head):
git checkout FETCH_HEAD
git checkout -b my-external-branch
If you plan to work with an external repository more often, it makes sense to add it as a remote (even if you don't plan to fetch everything):
git remote add ext-repository git://url/to/repository.git
Then you can either fetch the whole repository:
git fetch ext-repository
or again just single branches:
git fetch ext-repository branch