Stack: The stack is used as a sort of temporary scratch pad for use by the block of code that's currently executing, and whatever block called the current one, and whatever block called that one, and so on. When the current block exits, the local variables it was using are forgotten. As the name indicates, the stack is used in a last-in, first-out manner.
One of the most important uses of the stack is to keep track of the current call chain. When one function calls another, the caller pushes the address of the next instruction (the return address) onto the stack. As each function exits, it pops its caller's return address off the stack and continues executing code starting at that address. It's also used for communicating function parameters and return values between caller and callee.
Heap: The heap is different -- there's no particular order to it. If you want to allocate memory in a block of code and have that memory stick around beyond the end of the block, you allocate it on the heap. Of course, you'll also need to store a pointer/reference to it somewhere so that other code can find that memory; most languages provide accommodation for that.
Speed: Differences in speed aren't due to any property of the memory itself -- as you say in your question, both stack and heap typically inhabit the same physical memory. Allocating space on the stack is quick due to the stacks LIFO nature: if you push something onto the stack, there's only one place it can end up. By contrast, allocating a block on the heap requires finding a large enough contiguous free region in memory. A stack allocation can be as quick as a single instruction; a heap allocation requires a call to a memory allocation function like malloc().
Static v. dynamic: Allocating memory on the heap is dynamic -- whether to allocate a block, and the size of the block, can be determined according to input the program receives while it's running. Regions of memory allocated on the heap can even be resized if necessary. It's possible to dynamically allocate memory on the stack, too (see the C standard library function alloca()), but that memory will be lost as soon as the current function exits. Stack allocations are usually static -- the compiler determines how much space is needed for the (non-register) parameters, return data, and local variables, and it generates code to reserve the necessary space on the stack when the function is called.
Example: Imagine that you're creating a word processor. You can't know ahead of time how large the document will be, or even how many documents will be in use at the same time. At the same time, you want the user's documents to remain in memory as long as the user wants to keep them open. If you try to allocate memory for the documents on the stack you'll find it difficult to have more than one open at once, and you'll need to create a single function that creates, edits, saves, and closes the document. Allocating the space on the heap allows you to create as many documents as you like, each sized appropriately for the data it contains, and to avoid tying the lifetime of the documents to the lifetime of any particular function.
Summary: In a nutshell, the stack holds the values of variables (sometimes registers are used instead), while the heap is used for allocating memory that will be used beyond the lifetime of the current block.
