4

I've been working in a web project that uses mouse scroll wheel for different actions over a video. At some point I have to establish a coefficient for the relation between the deltaY and the number of frames that deltaY should roll. So different types of mouses return very different deltaY, specially smooth scroll ones.

In the fiddle I provide bellow this is done in:

targetOffset = targetOffset + (e.deltaY/1000); // 16000 aprox for smooth scroll mice

And 1000 is the coefficient that works well for a Notched Scroll Wheel common mouse. But if I use that coefficient with a Smooth Scroll Touch "wheel", like those of mac computers (that don't have a wheel really) that coefficient is "just too much", like 16 times "too much".

Is there something that could be done to detect this or to callibrate the coefficient in some way?

var FF = !(window.mozInnerScreenX == null); // is firefox?
var vid = document.getElementById("v");
var canvas = document.getElementById("c");
var context = canvas.getContext('2d');
var targetFrame = document.getElementById('t');
var cw = 200;
var ch = Math.round(cw/1.7777);
canvas.width = cw;
canvas.height = ch;
var directionScroll = 0;
var targetOffset = 0;
var coefficient = 1000;
var modes = ['pixels', 'lines', 'page'];
vid.pause();
 vid.addEventListener('seeked', function() {
    context.drawImage(vid, 0, 0, cw, ch);
 });
window.addEventListener('wheel', function(e) {
  e.preventDefault();
  // Normally scrolling this should be a substraction 
  //   not a sum but "I like it like this!"
  
  // Changed this with help of @Kaiido 's answer as partially solves the discrepancies between Firefox and Chrome
  // alert(modes[e.deltaMode]);
  if (modes[e.deltaMode]=='pixels') coefficient = 1000;
  else if (modes[e.deltaMode]=='lines') coefficient = 30; // This should correspond to line-height??
  else return false; // Disable page scrolling, modes[e.deltaMode]=='page'
  
  targetOffset = targetOffset + (e.deltaY/coefficient); // e.deltaY is the thing!!
  if (e.deltaY < 0) directionScroll = 1;
  if (e.deltaY > 0) directionScroll = -1;
  targetFrame.value = targetOffset;
  return false;
});

var renderLoop = function(){
  requestAnimationFrame( function(){
      context.drawImage(vid,0,0,cw,ch);
    if (vid.paused || vid.ended) {
      targetOffset = targetOffset*0.9;
      targetFrame.value=Math.round(targetOffset*100)/100;
      var vct = vid.currentTime-targetOffset;
      if (vct<0) {
        vct = vid.duration + vct;
      } else if (vct>vid.duration) {
        vct = vct - vid.duration;
      }
      vid.currentTime = vct;
    }
    renderLoop();
  });
};
renderLoop();
.column {
    float: left;
    width: 50%;
}

/* Clear floats after the columns */
.row:after {
    content: "";
    display: table;
    clear: both;
}
#c {
  border:1px solid black;
}
<h3>
  scroll up is forward
</h3>
<div class="row">
  <div class="column">
<div>
  Video element:
</div>
<video controls height="120" id="v" tabindex="-1" autobuffer="auto" preload="auto">
    <source type="video/webm" src="https://www.html5rocks.com/tutorials/video/basics/Chrome_ImF.webm"></source>
</video>
</div>
  <div class="column">
<div>
  Canvas element:
</div>
<canvas id="c"></canvas>
<div>
  Momentum: <input type=text id="t">
</div>
  </div>
</div>

Any help appreciated.

Edit 1:

I've updated the code so that a simple condition is applied to the coefficient, but that does not quite solve the issue as many variants are posible due to browser/plattform/mouse. Some way of callibrate the mouse could work?

Edit 2:

@Kaiido 's answer turned to resolve Firefox and Chrome differences. Firefox returns lines as deltaMode while Chrome returns pixels. I've edited the snippet to consider this.

But the problem still stands with the 'smooth scroll' mouse. To puzzle me even more, that mouse needs a coefficient opposite to the one of lines, it needs a coefficient larger instead of smaller.

lalengua
  • 509
  • 3
  • 15
  • I can't answer your question, but this scroll-to-control-video effect is neat – Jonathan Lam Apr 26 '18 at 00:18
  • I don't really have a good way to test this, but have you found a value that works well on smooth scroll mice? It's entirely possible that some value in-between would be "close enough" that it works okay in both situations. – Chris Hall Apr 26 '18 at 00:51
  • No, in the snippet (30 FPS), for `smooth scroll` mouse (at least the one I can test on) coefficient is around `16000`, 16 times compared to `1000` in the snippet which corresponds to a common `notched scroll` mouse. – lalengua Apr 26 '18 at 00:58
  • The wheel event should have a `deltaMode` property, can you test to see if this property is set to the same value everywhere? And as an experienced user of Apple's mice, and of smooth scrolling, I guess I can say I know how to trigger a *frame by frame* event or a scroll the whole page continuously one. And in your [previous question](https://stackoverflow.com/questions/49992447), I did remove your rAF loop, but if you really want to use it (e.g to threshold your wheel events even though the video's seeked one will take care of this threshold), then don't redraw every frame. – Kaiido Apr 26 '18 at 01:49

4 Answers4

3

See UPDATE at the end!


My original answer:

I don't have a mac nor a 'smooth' mouse, but I've tested your snippet both on Chrome and Firefox both on Windows and Linux boxes.

Works great on Chrome both on Windows and Linux but...

looks like the coefficient isn't the right one for Firefox... it works better (not as good as in Chrome) with 200.

One more thing:

Have you tested the mac fancy mouse on windows and vice-versa? Could it be a mac related problem?

UPDATE:

Other answers are great but I got puzzled by your question and learned a lot with the code and with what other answers pointed out, but something kept in my mind like a bug.

Searching for this topic I found this question very informative. It included a possible mouse scroll calibration script in this answer and a function getScrollLineHeight for Detecting the line-height used by DOM_DELTA_LINE triggered scroll events.

I've copied this function in the snippet for completeness, but at the end it's not needed for what I've thought. I've commented out the line that calls getScrollLineHeight because it does not work in this site for security reasons, but works in this fiddle.

My confusion was to think of scrolling as I normally do, in terms of pixels on a page. But your code really doesn't care about that. I mean, does not care about mouse scroll wheel event.deltaY magnitude. Only if it's positive or negative and consider that one step forward or backwards in a video timeline.

So this does not resolve the problem of "touch sensitive scroll mice", but it does resolve easily Firefox/Chrome and any Pixel/Line/Page deltaMode also. Now it runs smoothly both in Chrome and Firefox. I can't test on other browser because of WEBM video format, and I haven't been able to create a video in any format that works (look at my P.D. at the end).

So, every call is just one step: -1 or 1. Though it seems that only Firefox returns anything than "pixels" for deltaMode. I used this fiddle to test... Now you can focus on that smooth scrolling mouse and see how fast it sends every call, that is what really matters in this particular case (note that many macs have smooth scrolling software or inverted scrolling).

I've commented every line of your code and my modifications for my self but may be useful for others.

// detect if browser firefox as it appears to be the only
//  browser that return deltaModes different than DOM_DELTA_PIXEL
//  Ref: https://stackoverflow.com/a/37474225/4146962
var FF = !(window.mozInnerScreenX == null);

// Function grabbed from the reference above
// It tries to read current line-height of document (for 'lines' deltaMode)
function getScrollLineHeight() {
    var r;
    var iframe = document.createElement('iframe');
    iframe.src = '#';
    document.body.appendChild(iframe);
    var iwin = iframe.contentWindow;
    var idoc = iwin.document;
    idoc.open();
    idoc.write('<!DOCTYPE html><html><head></head><body><span>a</span></body></html>');
    idoc.close();
    var span = idoc.body.firstElementChild;
    r = span.offsetHeight;
    document.body.removeChild(iframe);
    return r;
}

// html5 elements
var vid = document.getElementById("v"); // HTML5 video element
var canvas = document.getElementById("c"); // HTML5 canvas element
var context = canvas.getContext('2d'); // Canvas context
var momentum = document.getElementById('m'); // Current momentum display
var delta = document.getElementById('d'); // Current deltaMode display
var lineheight = document.getElementById('l'); // Current deltaMode display

// global variables
var ch = 120; // canvas with (could be window.innerHeight)
var cw = Math.round(ch * (16 / 9)); // 16/9 proportion width
var targetOffset = 0; // Video offset target position when scrolling

// deltaY to FPS coefficients (for fine tuning)
// Possible mouse scroll wheel 'event.deltaMode'
//  modes are: 0:'pixels', 1:'lines', 2:'page'
var pc = 1000; // 'pixels' deltaY coefficient
var lh = "disabled"; //getScrollLineHeight(); // get line-height of deltaMode 'lines'
lineheight.value = lh; // display current document line height
coefficient = 30;
var deltaModes = ['pixels', 'lines', 'page']; // For deltaMode display

// Sets canvas dimensions
canvas.width = cw;
canvas.height = ch;

// Pauses video (this also starts to load the video)
vid.pause();

// Listens video changes time position
vid.addEventListener('seeked', function() {
  // Updates canvas with current video frame
  context.drawImage(vid, 0, 0, cw, ch);
});

// Listens mouse scroll wheel
window.addEventListener('wheel', function(e) {

  // Don't do what scroll wheel normally does
  e.preventDefault();

  // You don't need an amount, just positive or negative value: -1, 1
  var deltabs = 1;
  if (e.deltaY<0) deltabs = -1;

  // Disable page scrolling, modes[e.deltaMode]=='page'
  if (e.deltaMode>1) return false;

 delta.value = deltaModes[e.deltaMode];
  // Normally scrolling this should be a subtraction 
  //   not a sum but "I like it like this!"
  // targetOffset = targetOffset + (e.deltaY / coefficient); // e.deltaY is the thing!!
  targetOffset = targetOffset + (deltabs/coefficient);

  // Shows current momentum
  momentum.value = targetOffset;

  return false;
});

// Updates canvas on a loop (both for play or pause state)
var renderLoop = function() {
  requestAnimationFrame(function() {

    // This parts updates canvas when video is paused
    // Needs 'seeked' listener above
    if (vid.paused || vid.ended) {

      // Reduce target offset gradually
      targetOffset = targetOffset * 0.9;
      // Show current momentum
      momentum.value = Math.round(targetOffset * 100) / 100;

      // this part joins start and end of video when scrolling
      // forward & backwards
      var vct = vid.currentTime - targetOffset;
      if (vct < 0) {
        vct = vid.duration + vct;
      } else if (vct > vid.duration) {
        vct = vct - vid.duration;
      }
      vid.currentTime = vct;

      // This parts updates canvas when video is playing
    } else {
      // update canvas with current video frame
      context.drawImage(vid, 0, 0, cw, ch);
    }

    renderLoop(); // Recursive call to loop
  });
};
renderLoop(); // Initial call to loop
input {
  width: 50px;
}

.column {
  float: left;
  width: 50%;
}

/* Clear floats after the columns */
.row:after {
  content: "";
  display: table;
  clear: both;
}
<h3>
  mouse scroll video
</h3>
<div class="row">
  <div class="column">
    <div>
      Video element:
    </div>
    <video controls height="120" id="v" tabindex="-1" autobuffer="auto" preload="auto">
      <source type="video/webm" src="https://www.html5rocks.com/tutorials/video/basics/Chrome_ImF.webm"/>
    </video>
  </div>
  <div class="column">
    <div>
      Canvas element:
    </div>
    <canvas id="c"></canvas>
    <div>
      Momentum: <input type=text id="m">
    </div>
    <div>
      deltaMode: <input type=text id="d">
    </div>
    <div>
      lineHeight: <input type=text id="l">
    </div>
  </div>
</div>

P.D. I have a question (too specific for explaining elsewhere)... I've tested with my own videos and got very bad results... why is that? Something to do with specific video encoding settings? Do you know which encoding cmd would be needed for FFMPEG conversion to WEBM format like the video used in your example?

Sanxofon
  • 759
  • 9
  • 20
  • Thanks for that 'one more thing'. I'll have to check that. Many thanks for the testing. – lalengua Apr 26 '18 at 01:44
  • 1
    Wow! You are right! I't works great of firefox & now I think I know what to look for. Your testing fiddle is great I will look into it. About the encoding, I'm no expert, but by trial and error I've found that KeyFraming does the trick, at least in WEBM. Look for `-g` (max keyframe interval) in FFMPEG. The more keyframes the smoother but larger file. Something like `-g 6` or `-g 10` works for me. Example: `ffmpeg -i input.mov -aspect 16:9 -c:v libvpx -qmin 0 -deadline best -qmax 46 -crf 1 -b:v 100K -g 10 -an output.webm`. I play with the variables. I really don't know. – lalengua Apr 27 '18 at 02:40
  • Hi i'm on chrome in windows with a notched mouse and both directions of wheel make the video scroll backward :( – Steven Lu Sep 27 '18 at 06:05
  • @steven-lu, thank you there was an unnecessary `Math.abs()` applied on `deltabs` variable. I've edited the code. – Sanxofon Oct 21 '18 at 17:38
3

That's a wild guess, because I don't have such a Notched mouse to test with, but this 16 times factor really sounds like your delta values are not set on the same mode.

Indeed, the wheel event has 3 possible modes:

  • 0 => pixels (probably smooth scroll / Apple mice)
  • 1 => lines (probably the Notched Mice)
  • 2 => pages (keyboard ?)

So you may have to check your wheel event's deltaMode property to react accordingly.

onwheel = e => {
  var modes = ['pixels', 'lines', 'page'];
  console.log('scrolled by %s %s', e.deltaY, modes[e.deltaMode]);
}
<h1> scroll </h1>
Kaiido
  • 123,334
  • 13
  • 219
  • 285
  • This ended up resolving Chrome vs Firefox differences. Firefox returns `lines` for `deltaMode` and Chrome returns `pixels`. That makes Firefox run much better now, but the problem stills happens with the 'smooth scroll' mouse wich needs an inverse coefficient in relation to this!!. I am beginning to wish never have meet that mouse... – lalengua Apr 26 '18 at 03:20
  • 1
    @lalengua and to add to your frustration, some users (like myself) change this *natural scrolling* direction because I don't feel it's natural at all. Wheel is an hard beast to deal with... – Kaiido Apr 26 '18 at 03:21
  • That's so cruel! But many thanks, I'll consider it (a button for changing direction or something). You also helped me to detect 'page' scroll and disable it! Much appreciated. – lalengua Apr 26 '18 at 03:39
1

Why not just detect if the delta is greater than some threshold, and if so, divide by 16. I would think something like:

if (Math.abs(e.deltaY) > 400) slowdown = 16;
else if (Math.abs(e.deltaY) < 15) slowdown = 1;
targetOffset = targetOffset + (e.deltaY/1000/slowdown);

would do the trick.

var vid = document.getElementById("v");
    var canvas = document.getElementById("c");
    var context = canvas.getContext('2d');
    var targetFrame = document.getElementById('t');
    var cw = 200;
    var ch = Math.round(cw/1.7777);
    canvas.width = cw;
    canvas.height = ch;
    var directionScroll = 0;
    var targetOffset = 0;
    var maxDelta = 0;
    // vid.pause();
  vid.addEventListener('seeked', function() {
    context.drawImage(vid, 0, 0, cw, ch);
  });
    var slowdown = 1;
    window.addEventListener('wheel', function(e) {
      e.preventDefault();
      // Normally scrolling this should be a substraction 
      //   not a sum but "I like it like this!"
      if (Math.abs(e.deltaY) > 400) slowdown = 16;
      else if (Math.abs(e.deltaY) < 15) slowdown = 1;
      targetOffset = targetOffset + (e.deltaY/1000/slowdown); // e.deltaY is the thing!!
      if (e.deltaY < 0) directionScroll = 1;
      if (e.deltaY > 0) directionScroll = -1;
      targetFrame.value = targetOffset;
      return false;
    });
    
    var renderLoop = function(){
      requestAnimationFrame( function(){
       context.drawImage(vid,0,0,cw,ch);
        if (vid.paused || vid.ended) {
          // setTimeout(function(){
          targetOffset = targetOffset*0.9;
          targetFrame.value=Math.round(targetOffset*100)/100;
          var vct = vid.currentTime-targetOffset;
          if (vct<0) {
            vct = vid.duration + vct;
          } else if (vct>vid.duration) {
            vct = vct - vid.duration;
          }
          vid.currentTime = vct;
          // }, 0);
        }
        renderLoop();
      });
    };
    renderLoop();
.column {
    float: left;
    width: 50%;
}

/* Clear floats after the columns */
.row:after {
    content: "";
    display: table;
    clear: both;
}
#c {
  border:1px solid black;
}
<h3>
  scroll up is forward
</h3>
<div class="row">
  <div class="column">
<div>
  Video element:
</div>
<video controls height="120" id="v" tabindex="-1" autobuffer="auto" preload="auto">
    <source type="video/webm" src="https://www.html5rocks.com/tutorials/video/basics/Chrome_ImF.webm"></source>
</video>
</div>
  <div class="column">
<div>
  Canvas element:
</div>
<canvas id="c"></canvas>
<div>
  Momentum: <input type=text id="t">
</div>
  </div>
</div>
dave
  • 62,300
  • 5
  • 72
  • 93
  • That's smart! Thanks! But as noted by @sanxofon this is not quite what I'm looking for if many variants are posible when cross-browsing/plattform are posible. – lalengua Apr 26 '18 at 01:43
  • 1
    you may want to look at https://github.com/basilfx/normalize-wheel - it's code from facebook that they use to normalize wheel events, probably would help – dave Apr 26 '18 at 16:47
1

I tried it in Firefox and is not comparable with the Chrome "performance", so you can try adding this.

var vid = document.getElementById("v");
var canvas = document.getElementById("c");
var context = canvas.getContext('2d');
var targetFrame = document.getElementById('t');
var cw = 200;
var ch = Math.round(cw/1.7777);
canvas.width = cw;
canvas.height = ch;
var directionScroll = 0;
var targetOffset = 0;
// vid.pause();
vid.addEventListener('seeked', function() {
  context.drawImage(vid, 0, 0, cw, ch);
});

function normalizeDelta(wheelEvent) {
  var delta = 0;
  var wheelDelta = wheelEvent.wheelDelta;
  var deltaY = wheelEvent.deltaY;
  // CHROME WIN/MAC | SAFARI 7 MAC | OPERA WIN/MAC | EDGE
  if (wheelDelta) {
    delta = -wheelDelta / 120; 
  }
  // FIREFOX WIN / MAC | IE
  if(deltaY) {
    deltaY > 0 ? delta = 1 : delta = -1;
  }
  return delta;
}

window.addEventListener('wheel', function(e) {
  e.preventDefault();
  // Normally scrolling this should be a substraction 
  //   not a sum but "I like it like this!"
  targetOffset = targetOffset + normalizeDelta(e); // e.deltaY is the thing!!
  if (e.deltaY < 0) directionScroll = 1;
  if (e.deltaY > 0) directionScroll = -1;
  targetFrame.value = targetOffset;
  return false;
});

var renderLoop = function(){
  requestAnimationFrame( function(){
    context.drawImage(vid,0,0,cw,ch);
    if (vid.paused || vid.ended) {
      // setTimeout(function(){
      targetOffset = targetOffset*0.9;
      targetFrame.value=Math.round(targetOffset*100)/100;
      var vct = vid.currentTime-targetOffset;
      if (vct<0) {
        vct = vid.duration + vct;
      } else if (vct>vid.duration) {
        vct = vct - vid.duration;
      }
      vid.currentTime = vct;
      // }, 0);
    }
    renderLoop();
  });
};
renderLoop();
.column {
    float: left;
    width: 50%;
}

/* Clear floats after the columns */
.row:after {
    content: "";
    display: table;
    clear: both;
}
#c {
  border:1px solid black;
}
<h3>
  scroll up is forward
</h3>
<div class="row">
  <div class="column">
<div>
  Video element:
</div>
<video controls height="120" id="v" tabindex="-1" autobuffer="auto" preload="auto">
    <source type="video/webm" src="https://www.html5rocks.com/tutorials/video/basics/Chrome_ImF.webm"></source>
</video>
</div>
  <div class="column">
<div>
  Canvas element:
</div>
<canvas id="c"></canvas>
<div>
  Momentum: <input type=text id="t">
</div>
  </div>
</div>

Reference: https://stackoverflow.com/a/49095733/9648076

In case of deleted question, the answer purposed is this:

private normalizeDelta(wheelEvent: WheelEvent):number {
  var delta = 0;
  var wheelDelta = wheelEvent.wheelDelta;
  var deltaY = wheelEvent.deltaY;
  // CHROME WIN/MAC | SAFARI 7 MAC | OPERA WIN/MAC | EDGE
  if (wheelDelta) {
    delta = -wheelDelta / 120; 
  }
  // FIREFOX WIN / MAC | IE
  if(deltaY) {
    deltaY > 0 ? delta = 1 : delta = -1;
  }
  return delta;
}
  • That's sweet, thanks. I've already implemented something similar, but I as I pointed to @j08691 that doesent quite covers all posible variants. May be there are some specifications out there... I don't know. I'll update the question pointing that out. One more question: In which plattform did you test it? Thanks again. – lalengua Apr 26 '18 at 01:57
  • I tried this on Windows and Linux on both Chrome and Firefox browsers. – Julián Esteban Salomón Torres Apr 26 '18 at 02:19
  • 1
    Thanks for the Reference that I did't look earlier. Very interesting answers there! – lalengua Apr 26 '18 at 03:49