You want to add libraries to your project and in doing so you have two options as you yourself identified:
Compile-time libraries are libraries which is needed to compile your application. They are not included when your application is assembled (e.g., into a war-file). Libraries of this kind must be provided by the container running your project.
This is useful in situation when
you want to vary API and implementation, or when the library is supplied by the container (which is typically the case with javax.servlet which is required to compile but provided by the application server, e.g., Apache Tomcat).
Run-time libraries are libraries which is needed both for compilation and when running your project. This is probably what you want in most cases. If for instance your project is packaged into a war/ear, then these libraries will be included in the package.
As for the other alernatives you have either global libraries using Library Manager
or jdk libraries. The latter is simply your regular java libraries, while the former is just a way for your to store a set of libraries under a common name. For all your future projects, instead of manually assigning the libraries you can simply select to import them from your Library Manager
.