I think what you want could be similar to what I've been doing...
Make a directory in home called .host_configs/ . This is version controlled. Or in my case it lives in a special folder on a central computer, I scp it down on any new machine. Inside it make a folder for every host that you want different configurations for. The folder for each host should be named after the short hostname for that machine. So in your git repo you have:
.host_configs/
homecomp1/
girlfriendcomp1/
workcomp1/
workcomp2/
In each host specific folder, put the .vimrc, .irbrc, etc., configuration files for that specific box.
And also, in each host folder make a file called .[SHORT_HOST]_rc. For instance, if your machine is name "sane" have a file named .sane_rc ... This file will contain the lines that would normally be in .bashrc that are unique to that host. For instance, if it's a mac and it needs alias ls='ls -GF' instead of alias ls='ls --color=auto' which works for most nix machines for ls with colors, put that line in the .[SHORT_HOST]_rc for that machine, along with whatever special functions, declarations, etc, that would normally go into the .bashrc or .profile etc. (or .zshrc, .tschrc, as the case may be). So the version controlled ~/.host_configs/ folder looks like:
.host_configs/
homecomp1/
.homecomp1_rc #special shell configs for this hostname
.vimrc #you know the rest
.irbrc
.Xresources
girlfriendcomp1/
.girlfriendcomp1_rc
.vimrc
.bubblebathrc
workcomp1/
.workcomp1_rc
.bashrc
.vimrc
workcomp2/
.workcomp2_rc
.bashrc
.vimrc
I use all the same barebones $HOME/.bashrc (or ~/.tshrc etc) on all of my machines. I just take the basic one that comes with the distro in question and move all of the host-specific configuration into the .host-configs/[SHORT_HOST]/.[SHORT_HOST]_rc file.
Put this at the bottom (of $HOME/.bashrc):
export SHORT_HOST="sane"
for file in `find ~/.host_configs/$SHORT_HOST -name ".*"`
do
ln -s $file `basename $file`
done
source ~/`.$SHORT_HOST`_rc
(Finds all of the dot-files for the host and makes a symlink in home to the~/.host_configs/foo_host folder).
Your dot files are in their normal location but they are symlinked to version control. The above also sources all of the lines in your [$SHORT_HOST]_rc file into .bashrc
You can commit back to git from the ~/.host_configs/ folder whenever you have changes.
That's what it looks like in shell, which is probably all you need, but if you need other features, I would write something that uses the same principles (sourcing an external .rc file into .bashrc and symlinking all the config files to the structured version control folder) in something more versatile/less ugly than shell. So instead of the above in your .bashrc, there could be:
export SHORT_HOST="sane"
ruby ~/import_conf.rb $SHORT_HOST
...and write your import_conf.rb to do more complex conf management, like placing a specific configuration file in some directory besides home, or handling a config folder like .ssh/, .subversion/ etc. That's what I do, it's pretty elegant for me, but there may be better solutions. Dropbox with some creative symlinks is also an excellent idea, though you're relying on a third party, and you need to be in a graphical environment. Also note there are inconsistencies between what you can do with symlinks + dropbox in Linux and shortcuts + dropbox in Windows if you implement something that wants to play with Windows.
