Just to extend @Andy's answer
Dependency Injection is one of the forms of the Dependency Inversion Principle
To achieve the decoupling of dependencies (as typically found in layered architecture),
DI is commonly used for instantiation scenarios such as basic new() and patterns like Factory method. In addition to being able to inject a new dependency instance every time (e.g. like factory), containers can also be set up to inject named instances, singleton instances, etc - i.e. IoC containers usually also take on the responsibility of managing the lifespans of objects as well.
One potential 'mindset shift' is that dependencies now potentially become publicly visible on concrete classes, since DI typically injects via constructors or public Get / Set properties. This may seem strange if you are used to using OO encapsulation, where dependencies of a class are seen as implementation and should be hidden from the 'outside' i.e. class method signatures.
However, by implementing Interface / Concrete class separation (as you should, not only for decoupling but also for testing / mocking purposes), the injection constructors / property injection methods will not be on the interface, so encapsulation is again in place.
Re : "Doing DI by hand" without Unity etc
What you would need to do is to code your own IoC container, which then is responsible for 'building up' instances of classes - during each 'build up', you would scan the class for dependencies (which are configured in the container, e.g. by config, by attributes, or simply just by convention, e.g. all public settable properties, or any class parameters on a constructor will be assumed to be dependencies). You would then create (if necessary) and inject this 'dependency' instance onto the object (e.g. by using reflection). And then recursively, all dependencies of these dependencies need to be built up etc. You would then also need to provide lifespan management for each of the objects, e.g. Singletons etc.