There are basic two considerations here: the needs of the compiler and the goal of good source code organization. In general, I find that header files should contain as little code as possible while still keeping the compiler happy. Header files should also #include as few other header files as possible, preferring forward type declarations to #include statements whenever possible. This reduces the number of dependencies between your source files and reduces compile time. It also helps avoid circular dependencies among source files.
While almost all function implementations go in the .cpp file, template functions and inline functions need to be available to the compiler when compiling dependent types. Rather than putting these in the header file, I prefer to put them in a third file (I use the extension .icpp for 'included' implementation). This file is #included at the bottom of the header file (inside the #ifdef guard), keeping the .hpp file free from all implementation details. While not required by the compiler, the header file does contain documented declarations of all template and inline functions. Personally I never write 'implicitly' inline functions where the function definition is enclosed within the class definition.
So for all classes, types and functions that are used by more than one .cpp file, the class definitions, type definitions and function declarations go in the header file, and the corresponding function definitions (implementations) go in the .cpp file, except definitions of inline and template functions go in the .icpp file. I successfully avoid global data so I don't have any rules for that. I usually end up using static member data instead, accessed through global or static member functions, and the guidelines above apply.
Classes, types and functions that are used within one .cpp file only I sometimes put in the .cpp file. These will always be simple utility functions that have very low probability of reuse elsewhere. These go in an anonymous namespace, shielding them from name collisions with other types and functions in other .cpp files.
As to private data, conceptually the users of some class has no business knowing what private data (or functions for that matter) a class has. However, C++ syntax requires these to be declared as part of the class definition, hence they do appear in the header file. This is not usually a problem, but if it is the Pimpl design pattern can be used as a work-around. Here you define two classes, Foo and FooImpl, where FooImpl holds all your data, and Foo holds an instance of FooImpl. FooImpl.hpp is not #included in Foo.hpp, thereby hiding implementational details from the users of Foo.)