Include this library in your page: https://github.com/vimeo/player-api/tree/master/javascript like this
<script src="//f.vimeocdn.com/js/froogaloop2.min.js"></script>
This code will send an API call to vimeo player to set the volume to 0 once the player is ready, based on http://developer.vimeo.com/player/js-api
// you may need another way of getting reference to the iframe:
var iframe = document.getElementsByTagName('iframe')[0];
var player = $f( iframe );
player.addEvent('ready', function() {
player.api('setVolume', 0);
});
http://jsfiddle.net/87dsjL8q/
Or, without the external library:
iframe.contentWindow.postMessage('{"method":"setVolume", "value":0}','*');
http://jsfiddle.net/87dsjL8q/1/