tl;dr, To have an expression is like having the source code of an application, and a delegate is an executable to run the application. An expression can be thought of as the "source" (i.e., syntax tree) of the code that would run. A delegate is a specific compilation that you would run and does the thing.
By storing a lambda as a delegate, you are storing a specific instance of a delegate that does some action. It can't be modified, you just call it. Once you have your delegate, you have limited options in inspecting what it does and whatnot.
By storing a lambda as an expression, you are storing an expression tree that represents the delegate. It can be manipulated to do other things like changing its parameters, changing the body and make it do something radically different. It could even be compiled back to a delegate so you may call it if you wish. You can easily inspect the expression to see what its parameters are, what it does and how it does it. This is something that a query provider can use to understand and translate an expression to another language (such as write an SQL query for a corresponding expression tree).
It is also a whole lot easier to create a delegate dynamically using expressions than it is emitting the code. You can think of your code at a higher level as expressions that is very similar to how a compiler views code instead of going low-level and view your code as IL instructions.
So with an expression, you are capable to do much more than a simple anonymous delegate. Though it's not really free, performance will take a hit if you run compiled expressions compared to a regular method or an anonymous delegate. But that might not be an issue as the other benefits to using expressions may be important to you.