5

Allright, I know what machine precision is, but this, I can't understand...

Code:

 console.log("meanX",meanX);
 meanX2 = meanX * meanX;           //squared
 console.log("meanX2",meanX2);

Console output:

meanX 300.3
meanX2 28493.4400000000002

In case you are wondering, the correct value for meanX2 would be 90180.09 And this is only one of the many examples visible in the screenshot..

See with your own eyes

.toFixed(6) seems to fix this... But I have no idea why it doesn't work without it.

Edit

Ok, I don't want to post the whole program code here because in first place I'm not the only author, and second, I also wouldn't like this to be copied without our permission. But I'll gladly explain how I get this error and will post the whole method/function code here.

This code belongs, as you may have guessed from the window title, to a lane detection algorithm. We use Three.js/webgl to run some pre processing shaders on each frame of a video and then we analyze the resulting image. The method/function you see on the screenshot is a perpendicular line fitting algorithm and is part of the whole thing. I can see the algorithm running nicely because I have the lane being drawn on top of the video, and It is well placed. Until suddenly the lane turns into an horizontal bar. This unexpected behavior happens exactly because of the phenomenon I described here, since it's from that moment that I start to see wrong math in the console.

Also, because the video and algorithm run at slightly different fps everytime, the problem doesn't always happen in the same moment of the video, and sometimes It doesn't happen at all.

Here is the code (it has some alterations because I was trying to isolate the issue):

this.perpendicularLineFit = function (points, slopeSign) {
    var count = points.length;
    var sumX = 0,
        sumY = 0;
    var sumX2 = 0,
        sumY2 = 0,
        sumXY = 0;
    var meanX, meanY;
    var i, lowp = {}, highp = {};
    var B;
    var slope;
    var originY;

    for (i = 0; i < count; i++) {
        sumX += points[i].x;
        sumY += points[i].y;
        sumX2 += points[i].x * points[i].x;
        sumY2 += points[i].y * points[i].y;
        sumXY += points[i].y * points[i].x;
    }

    meanX = sumX / count;
    meanY = sumY / count;

    //If you uncoment this, problem reappears: 
    //var numeratorLeft = meanY * meanY;
    console.log("meanX",meanX);
    var meanX2 = meanX*meanX;
    console.log("meanX2",meanX2);

    var numerator = (sumY2 - count * (meanY * meanY)) - (sumX2 - count * meanX2);
    var denominator = (count * meanX * meanY - sumXY);  
    B = 0.5 * (numerator / denominator);


    slope = -B + slopeSign * Math.sqrt(B * B + 1);
    originY = meanY - slope * meanX;

    slope = isNaN(slope) ? slopeSign : slope;
    originY = isNaN(originY) ? originY : originY;

    lowp.y = this.lowY;
    lowp.x = (this.lowY - originY) / slope;
    highp.y = this.highY;
    highp.x = (this.highY - originY) / slope;

    return {
        low: lowp,
        high: highp
    };
};

Now, I was trying to understand what was causing this, and the most bizarre thing is that it seems that when I place a statement of this form

var x = ... meanY * meanY ...;

before the meanX2 attribution, the issue happens. Otherwise it doesn't.

Also, I tried to catch this anomaly in the debugger but just when I enter the debugging tab, the problem disapears. And the values turn correct again.

I certainly don't believe in black magic, and I know that you are probably skeptic to this. I would be too. But here is a link to a video showing it happening: The video

Edit2:

I managed to reproduce this issue in another computer.. Both having ubuntu and using firefox (versions 20 and 21).

Edit3:

I'm sorry it took so much time! Here is a zip containing the issue. Just run it in any webserver. The code mentioned is in LaneDetection.js. Search for "HERE" in the file to find it.

https://docs.google.com/file/d/0B7y9wWiGlcYnYlo1S2pBelR1cHM/edit?usp=sharing

The problem might not happen in the first attempts. If that's the case refresh the page and try again. When the lines get horizontal you know it's there. As I said, I saw this problem happening in firefox versions 20 and 21 on ubuntu. In chrome it never happened.

By the way, I noticed that changing javascript.options.typeinference flag in firefox seems to stop the problem... I don't know exactly what that flag does, but maybe this optimization is not correctly implemented in firefox?

jmacedo
  • 773
  • 1
  • 13
  • 24
  • This should give you an idea of how precision works in javascript [https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toFixed?redirectlocale=en-US&redirectslug=JavaScript%2FReference%2FGlobal_Objects%2FNumber%2FtoFixed][1] – karthikr Jun 03 '13 at 02:49
  • possible duplicate of [Elegant workaround for JavaScript floating point number problem](http://stackoverflow.com/questions/1458633/elegant-workaround-for-javascript-floating-point-number-problem) – karthikr Jun 03 '13 at 02:49
  • 11
    I don't get anything like the same results when I try this. No "floating point is weird" issue would be responsible for such an anomaly. The computation works perfectly fine in any browser I try. – Pointy Jun 03 '13 at 02:51
  • 3
    @Pointy Exactly! I don't get it people. This is not about a minor floating point precision error..! – jmacedo Jun 03 '13 at 02:52
  • 3
    @karthikr Please read the question before assuming something when you see decimal points/places – Ian Jun 03 '13 at 02:54
  • 11
    How about a functional demo of your problem? Something is going on that you're not showing us. – mu is too short Jun 03 '13 at 02:54
  • What happens if you `console.log('meanX2 ' + meanX2)` instead? Is anything below the `console.log` call changing `meanX2`? – mu is too short Jun 03 '13 at 02:58
  • My error, bad eyes missed the 2 – Fallexe Jun 03 '13 at 03:22
  • 1
    @Fallexe: And that's a good illustration of why posting screen shots of code isn't the best idea ever :) – mu is too short Jun 03 '13 at 03:31
  • Instead of the `console.log()` calls, put in a `debugger;` call after you calculate `meanX2`. When it stops there, look around in the debugger and see if you get more clues. – Michael Geary Jun 03 '13 at 05:35
  • 1
    @MichaelGeary See the edited part of the question. The moment I press debugger tab, the algorithm corrects itself and console starts to give right values. I SWEAR. I putted this code: `if(meanX > 200 && meanX2 < 30000) console.log("WTF");` after the calculations and sometimes WTF is printed in the console, which is mathematically impossible. I have to record a video of this else noone will believe me... – jmacedo Jun 03 '13 at 13:57
  • @muistooshort added more info to the question. – jmacedo Jun 03 '13 at 14:00
  • Seems to work fine: http://jsfiddle.net/ambiguous/V6C8S/ . Is there some special data that triggers the problem? – mu is too short Jun 03 '13 at 16:49
  • @muistooshort See the link in the end of the question. It starts to happen more frequently in some parts of the video then in others.. So it may have something to do with the data, but how so? The data doesn't matter anymore when I print the values ... :/ – jmacedo Jun 03 '13 at 17:14
  • 1
    Sorry, a video of code is worth about as much as a screenshot of code (i.e. not worth very much at all). Either you have a functional demo of the problem or you don't have an answerable question. – mu is too short Jun 03 '13 at 17:40
  • @muistooshort As I explained I can't post all the code and I don't have time to create a demo which simulates a similar situation just for this. Besides, my partners haven't been able to reproduce this in their browsers too, so it would probably be for nothing. Maybe someone in the world can, but when I asked this question I was expecting a straight, obvious, answer... Not something serious. – jmacedo Jun 03 '13 at 18:19
  • 1. Your results don't seem to be correlated to meanX at all. When meanX goes down, sometimes meanX2 goes up and sometimes it goes down. 2. It does seem that the results are the square of something - the sqrt(28493.4400000000002) is exactly 168.8. – Fallexe Jun 03 '13 at 18:48
  • 3
    @joxnas - I don't think anyone doubts that this is actually happening when you run your code. But it's incredibly difficult to try to troubleshoot a problem like this if you can't actually see it in action yourself. This is why bug reports get closed as "cannot reproduce". You have a bunch of people curious about this, but none of us have any way to really help unless we can see it in action and poke at it to see what is going on. If there is any way you can make a test page that duplicates the problem, I'll bet someone can figure it out right away. – Michael Geary Jun 03 '13 at 20:30
  • 2
    `originY = isNaN(originY) ? originY : originY`? What? – Pang Jun 04 '13 at 01:19
  • @Pang leftover code. After so much debugging sometimes one ends up with these pearls – jmacedo Jun 04 '13 at 06:24
  • @MichaelGeary I'll see what I can do. If not sooner, in the end of the semester I can put this online. – jmacedo Jun 04 '13 at 22:17
  • @joxnas add a line like this: if(meanX2 != meanX * meanX) console.log(meanX, meanX2, points) and see if you can catch it in action – SheetJS Jun 07 '13 at 17:22
  • 1
    have you considered damaged FPU? – Maku Jun 11 '13 at 14:53
  • @Maku I managed to reproduce the problem in another computer... Both times with firefox, ubuntu, my computer had version 20, the other one had version 21. – jmacedo Jun 11 '13 at 15:35
  • @joxnas could it be a memory related issue since it obviously only happens after a certain time? Does `perpendicularLineFit` run asynchronous? And how frequent is that function called? – basilikum Jun 11 '13 at 20:11
  • Please read the answers to this question:http://stackoverflow.com/questions/588004/is-javascripts-floating-point-math-broken. That might be the cause. – Loek Bergman Jun 11 '13 at 22:46
  • Did you ever figure out what was going on here? Just curious. – Reinstate Monica -- notmaynard Jul 16 '13 at 20:11
  • @iamnotmaynard If you're still curious, check the new edition I've made to the question – jmacedo Aug 14 '13 at 00:15
  • @MichaelGeary I edited the question. Now with a working sample ;) – jmacedo Aug 14 '13 at 00:19
  • are you meaning to revisit an issue from two years ago by posting new information? – Claies Nov 03 '15 at 02:45
  • @Claies No, I just noticed I had misspelled some words so I corrected them.... – jmacedo Nov 03 '15 at 02:52

1 Answers1

3

I can't say for sure that I actually have an answer but I think that I have confirmed that basilikum was correct to suggest a memory problem. Here's what I did: I took the first ten entries from your screenshot and calculated the correct answer. I then converted the correct answer and the wrong answer into the hexidecimal representation of the double-precision float. What I ended up with was the following:

292.416^2 = 85507.506 = 40F4E0381C71C71E
 changed to 27583.373 = 40DAEFEB1C71C722

293.166^2 = 85946.694 = 40F4FBAB1C71C72A
 changed to 27583.373 = 40DAEFEB1C71C722

295.818^2 = 87508.396 = 40F55D4658DC0876
 changed to 28041.024 = 40DB62419637021F

294.500^2 = 86730.250 = 40F52CA400000000
 changed to 27583.373 = 40DAEFEB1C71C722

297.000^2 = 88290.000 = 40F58E2000000000
 changed to 28041.024 = 40DB62419637021F

221.750^2 = 49173.062 = 40E802A200000000
 changed to 24964.000 = 40D8610000000000

300.300^2 = 90180.090 = 40F6044170A3D70A
 changed to 28493.440 = 40DBD35C28F5C290

220.200^2 = 48488.040 = 40E7AD0147AE147B
 changed to 25408.360 = 40D8D0170A3D70A4

300.600^2 = 90360.360 = 40F60F85C28F5C29
 changed to 28493.440 = 40DBD35C28F5C290

213.000^2 = 45369.000 = 40E6272000000000
 changed to 28032.326 = 40DB6014E5E0A72E

There's no persistent pattern to the change but there are a couple instances that are very telling of a memory issue. In the first two entries you can see that bytes 1, 2 and 3 were unchanged. In the 9th entry there's something even more odd. It would appear that bytes 0 - 3 were shifted left by exactly 4 bits! Upon considering the statement that the problem doesn't arise until after some time has passed and in light of these two anomalies, I'm fairly confident that you're encountering some sort of memory issue. Could it be, dare I say, a stack overflow?

Travis R
  • 349
  • 4
  • 6
  • I managed to reproduce this on another computer, so I guess that makes it unlikely to be a memory problem? – jmacedo Jun 16 '13 at 03:10
  • @joxnas *minimal jsfiddle or no issue* - include information about the environments (browser/OS) with the observed issue and environments without the issue (if any are known). Here is [my test case](http://jsfiddle.net/2qVez/) – user2246674 Jun 16 '13 at 04:10
  • I'm not suggesting a physical memory problem but rather a memory management issue. The observations point to unintended data manipulations, which could occur with errant pointers or stack or heap corruption. Your reproduction occurred with essentially the same underlying software - a Linux version of Firefox. If this is a memory problem then that is most likely where the issue lies - I would lean toward the idea that your program is exposing something in Firefox, or abusing it's capabilities somehow. If you can (or cannot) reproduce with Chrome or another browser, then that would be telling. – Travis R Jun 17 '13 at 16:29
  • @TravisR Sorry for taking so long! I've made a new edition to the post, maybe you're interested. – jmacedo Aug 14 '13 at 00:16
  • 1
    @joxnas I've skimmed your updates and this is all consistent with the hypothesis that you've exposed some memory management flaw in Firefox for Linux - the unpredictable timing, the fact that small changes can make it appear or disappear and the restriction to a particular platform all fit the paradigm. I suggest you start using [memory tools](https://wiki.mozilla.org/Performance:Leak_Tools) to get a better grip on the problem. Then once you have strong evidence of a bug in Firefox, submit it to Mozilla and see if they can take it from there. – Travis R Aug 15 '13 at 04:08