To change the favicon of the notebook you have to create individual profiles for the desired servers. For each profile (e.g. custom) add the favicon to .ipython/profile_custom/static/base/images/favicon.ico
.
Now start your server with the newly created profile like
ipython notebook --profile custom
and voila you have your custom favicon.
Note, that the favicons are typically .ico files as discussed e.g. here.
I tested this with a prominent favicon and now have notebooks looking like

This is also used by some of the available kernels like e.g. IJulia.