The files in the lib directory allow you to extend GAE beyond its built in third-party libraries. Per the documentation on the subject, you do need to upload any third-party libraries that you wish to use.
Cloud Endpoints Frameworks v2 recently came out of beta, so it is entirely possible that it is now included in the built in libraries, but that's not something should be counted on as the package endpoints 2.0 is not currently listed as being available in the documentation.
So, to answer your question, yes, there are certainly serious problems that you could run into by not including the lib folder. For libraries that are not known to be built in, it seems to me that it would be wise to control the versions you need via the lib folder.
Further info, per @Dan Cornilescu's comment:
If you need to use a built in library, you should list it in the libraries:
section of your app.yaml
, as detailed in the app.yaml documentation.