I tested this on OSX 10.10.x using the following commands (my mergetool was set to something else before this):
# locate xcode utilities
sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer
# set "opendiff" as the default mergetool globally
git config --global merge.tool opendiff
And then simulated a merge conflict on a freshly created git repo. Following this a git mergetool
opened up the diff tool without any problem:

I tested this again with another mergetool git config --global merge.tool bc3
:

Can you check that your mergetool is being setup correctly by doing the following:
git config --list --show-origin
If that does not provide a clue to the problem, then here is the VERBOSE output that might hold more clues:
$ GIT_CURL_VERBOSE=1 GIT_TRACE=2 git mergetool
+zbell_begin:1> zbell_timestamp=1478783158
+zbell_begin:2> zbell_lastcmd='GIT_CURL_VERBOSE=1 GIT_TRACE=2 git mergetool'
+iterm2_preexec:2> PS1='
%F{135}%n%f at %F{166}%m%f in %F{118}%~%f ${vcs_info_msg_0_}
$python_info[virtualenv]$ '
+iterm2_preexec:3> ITERM2_SHOULD_DECORATE_PROMPT=1
+iterm2_preexec:4> iterm2_before_cmd_executes
+iterm2_before_cmd_executes:1> printf '\033]133;C;\007'
+-zsh:70> GIT_CURL_VERBOSE=1 GIT_TRACE=2 git mergetool
13:05:58.178595 git.c:562 trace: exec: 'git-mergetool'
13:05:58.179015 run-command.c:336 trace: run_command: 'git-mergetool'
13:05:58.189681 git.c:562 trace: exec: 'git-sh-i18n--envsubst' '--variables' 'usage: $dashless $USAGE'
13:05:58.190101 run-command.c:336 trace: run_command: 'git-sh-i18n--envsubst' '--variables' 'usage: $dashless $USAGE'
13:05:58.196721 git.c:562 trace: exec: 'git-sh-i18n--envsubst' 'usage: $dashless $USAGE'
13:05:58.197349 run-command.c:336 trace: run_command: 'git-sh-i18n--envsubst' 'usage: $dashless $USAGE'
13:05:58.211241 git.c:349 trace: built-in: git 'config' '--bool' 'mergetool.prompt'
13:05:58.214550 git.c:349 trace: built-in: git 'rev-parse' '--git-dir'
13:05:58.218507 git.c:349 trace: built-in: git 'rev-parse' '--git-path' 'objects'
13:05:58.231731 git.c:349 trace: built-in: git 'config' 'merge.tool'
13:05:58.236048 git.c:349 trace: built-in: git 'config' 'mergetool.opendiff.cmd'
13:05:58.240211 git.c:349 trace: built-in: git 'config' '--bool' 'mergetool.keepBackup'
13:05:58.245948 git.c:349 trace: built-in: git 'config' '--bool' 'mergetool.keepTemporaries'
13:05:58.250250 git.c:349 trace: built-in: git 'rev-parse' '--show-toplevel'
13:05:58.254733 git.c:349 trace: built-in: git 'ls-files' '-u'
Merging:
another-file
13:05:58.259422 git.c:349 trace: built-in: git 'ls-files' '-u' '--' 'another-file'
13:05:58.265199 git.c:349 trace: built-in: git 'config' '--bool' 'mergetool.writeToTemp'
13:05:58.268606 git.c:349 trace: built-in: git 'ls-files' '-u' '--' 'another-file'
13:05:58.273569 git.c:349 trace: built-in: git 'ls-files' '-u' '--' 'another-file'
13:05:58.280265 git.c:349 trace: built-in: git 'ls-files' '-u' '--' 'another-file'
13:05:58.318106 git.c:349 trace: built-in: git 'rev-parse' '--show-cdup'
13:05:58.350195 git.c:349 trace: built-in: git 'rev-parse' '--show-cdup'
Normal merge conflict for 'another-file':
{local}: created file
{remote}: created file
13:05:58.356881 git.c:349 trace: built-in: git 'config' 'mergetool.opendiff.cmd'
13:05:58.361075 git.c:349 trace: built-in: git 'config' 'mergetool.opendiff.path'
13:05:58.365342 git.c:349 trace: built-in: git 'config' 'mergetool.opendiff.cmd'
13:05:58.370113 git.c:349 trace: built-in: git 'config' 'mergetool.opendiff.cmd'