Framework design is much more difficult than application design. I just wrote an article on my blog about it and came up with three ideals that I follow for framework design.
- Get design correct before a release
- Keep the visible sdk small & focused
- Make the sdk easy to use
Points 1 and 2 are important because once a release is made, refactoring the visible sdk results in breaking changes. Point 3 is important if you want developers to use your sdk.
For me, these ideals govern every decision I make in my framework design. I had to consider them over and over throughout the whole process. I put a bunch of tips in my article to achieve these goals.
Framework/SDK/API Design Tips
In terms of design patterns, I think I followed a similar path to you. I learned about design patterns a few years ago and realized how powerful (and destructive) they can be. I used some in some programs, and eventually got to use some on a framework. I found that in my framework, I got a chance to apply new patterns that I did not get to use for applications (off the top of my head, I can only think of template method and I believe I used factories more in my framework than in an application. I can't remember all the details though).
Good luck. I think framework design is difficult but also rewarding.
Disclaimer: I am not an expert in framework design...just something I've dabbled in.