What you're trying to do is come up with a function that maps serial numbers to activation keys. Without knowing more about the nature of the function, this could be anywhere from very easy (a polynomial with only a few terms) to very hard (a multi-tiered function involving lots of block XORs, substitution tables, complicated key schedules, ...).
If you have access to the key verifier routine (e.g. by disassembly - which is almost always against the EULAs of commercial software), then you have a routine that returns whether or not a given activation key is correct for a given serial number. If this was done by computing an activation key for a serial number, then you are practically done. If this was done by computing the inverse function on the key, then your task is a little harder: you need to invert that function to retrieve the key derivation algorithm, which may not be so easy. If you end up having to solve some hard mathematical problems (e.g. the discrete logarithm problem) because the scheme depends on public-key cryptography, then you're hoping that the values you're dealing with are small enough that you can brute-force or use a known algorithm (e.g. Pollard's rho algorithm) in computationally feasible time.
In any case, you'll need to get comfortable with disassembly and debugging, and hope that there are no anti-debugger measures in place.
Otherwise, the problem is much harder - you'd need to make some educated guesses and try them (e.g. by trying to do a polynomial fit), and hope for the best. Because of the very large variety of different possible functions that can fit any set of inputs and outputs (mathematically uncountable, though in practice limited by source code size), trying to do a known-plaintext attack on the algorithm itself is generally infeasible.
