EJBs, or enterprise java beans are java classes that can be managed by Java EE container that guarantees services like
- bean life cycle
- thread management
- transaction management etc
Yes, JPA is not a part of Java EE spec now. It moved to JSE. However in past when Entity Beans provided the "standard" bridge between java and relational databases world.
What you cannot do without EJB? I'd say nothing. I mean you can do everything without EJB. And the reason is that there are alternative solutions like Spring or Guice.
And as always you can write lower level code without any framework.