When you're using a Library project one of the things you're in fact doing is compiling your Library project into a jar and then referencing that jar in your calling Project.
If you right click the Project, and select "Configure Build Path" you'll see a tab called "Libraries", if you look inside "Android Dependencies" you'll notice a list of jar's corresponding to your Library projects.
These jars are expandable, showing you that they have a slot for a source attachment. Usually this would be editable allowing you to directly link the source but in terms of ADT these are already filled and are uneditable.
When debugging these files you're linked to a read-only class file with this attached source. This is because you're not running against source files directly, you're running against a pre-compiled class file. Until the ADT team get this functionality in place, you're pretty much forced to jump to the direct source code and rebuild everything.
EDIT
See @Steven linked answer :)