0

I'm trying to reuse some old Flask web application from 2-3 years ago that seems to have broken in the interim.

I'm trying to figure out what the problem is but I'm having trouble debugging systematically.

Sorry I can't provide more specific code, the program is way too large and the problem is basically unknown. If I had to guess, the problem was happening in this segment:

elif resource=="shelfquery":
    print ("Shelf")
    if request.method=="PUT":
        userInput=request.form['userInput']
        print ("Shelf " + userInput)
        cur.execute ("SELECT * FROM picDB")
        for i in range (cur.rowcount):
            value=cur.fetchone ()
            if value is not None:
                searchTerm=value [0]
                print ("searchDB|")
                print (searchTerm)

        validInput=True
        try:
            # searchFind= ("SELECT searchString,pureJSON FROM nytDB where searchString LIKE %s")
            # searchVal= user
            if ("searchString" not in userInput and "searchDB" not in userInput and "DROP TABLE" not in userInput
                and "DELETE " not in userInput and "pureJSON" not in userInput and "*" not in userInput):
                cur.execute ("SELECT searchString,pureJSON FROM searchDB where searchString LIKE %s", [userInput])
            else:
                validInput=False
        except (MySQLdb.Error) as e:
            print ("Error1")
            print (e)

        print ("Row count ")
        # print (cur.rowcount)
        if cur.rowcount<=0:
            searchUrl=shelffURL + userInput + shelffURL2
            response = urllib2.urlopen(searchUrl)
            content=response.read ()

            jsonBalls= json.loads (content.decode ())
            if ("searchString" not in userInput and "searchDB" not in userInput and "DROP TABLE" not in userInput
                and "DELETE " not in userInput and "pureJSON" not in userInput and "*" not in userInput):
                cur.execute ("INSERT IGNORE INTO searchDB (searchString,pureJSON,readStatus) VALUES (%s,%s,%s)" ,(userInput,content,"unread"))
                db.commit ()
            else:
                print ("Invalid syntax, blocked to avoid SQL Injection")
                validInput=False




            return jsonify (jsonBalls)
        elif cur.rowcount>0 and validInput==True:
            value=cur.fetchone ()
            searchStringValue=value [0]
            jsonValue= value[1]
            # print ("Getting from searchDB")
            # print (searchStringValue)
            jsonValue2=json.loads (jsonValue.decode ())
            print (jsonValue)
            return jsonify (jsonValue2)
        else:
            return None

Since that is often where the segfault happens (though not the only place) . Corresponding javascript/jquery code:

function addToBookShelf(methodType){
console.log ("Adding");

var userInputValue;

console.log (userInputValue);
if(methodType == "PUT" || methodType == "POST"){
    userInputValue = $('#userInput').val();
}else {
    userInputValue=methodType;
    methodType="PUT";
}
/*
    jQuery ajax does not send undefined data, so GET and DEL
    are sent without any params when userInputValue is not
    initialized.
*/
console.log ("Shelf query");
var picURL;
$.ajax({
    method: methodType,
    url: "/shelfquery",
    data: { userInput: encodeURI(userInputValue) }
})
.done(function( msg ) {
    console.log (msg);
    console.log ("Time to get photos");
    //$('#gallery').empty ();
    // for (i = 0; i<10 ; i++) {
    $.ajax({
        method: methodType,
        url: "/picquery",
        data: { userInput: encodeURI(userInputValue) }
        })
        .done(function( msg2 ) {
            console.log ("Save");
            console.log (msg2);
        id=msg2.items[0].volumeInfo.industryIdentifiers[0].identifier;
        volumeID=msg2.items[0].id;
        desc=msg2.items[0].volumeInfo.description;


        picURL=JSON.stringify(msg2['items'][0]['volumeInfo']['imageLinks']['thumbnail']);
        divStart="<div class=\"panel panel-primary col-lg-3 col-md-4 col-xs-6 thumb\" id=\"" + id +".." + volumeID +"\">"
        divMiddle="<div class=\"panel-body\">"
        divImge= "<img src=" + picURL + "class=\"img-responsive center-block\" />"
        divMiddleEnd="<div class=\"panel-image\">" +divImge + "<label for=\"toggle-4\"></label></div>"
        category=JSON.stringify(msg2['items'][0]['volumeInfo']['industryIdentifiers']['categories']);
        title=msg2.items[0].volumeInfo.title;
        author =msg2.items[0].volumeInfo.authors[0];
        categoryLine= "<p>" + category +"</p>"
        previewLink=JSON.stringify(msg2['items'][0]['volumeInfo']['previewLink']);
        previewLinkLine="<p><a href="+ previewLink +"> Preview Link</a></p>"
        heading= "<div class=\"panel-heading\"><h3 class=\"panel-title\">" + title + " by " + author + "</h3></div>"
        buttonFormat="<button class=\"next btn btn-success\">View More Info</button>"

        buttonFormat2="<button class=\"endd btn btn-danger\">Delete from DB</button>"
        midButtonFormat="<div class=\"btn-group\"><a class=\"reads btn btn-xs btn-success\" href=\"#\">Read</a><a class=\"readngs btn btn-xs btn-warning\" href=\"#\">Reading</a><a class=\"unreads btn btn-xs btn-danger\" href=\"#\">Unread</a></div></div>"


        reviewCount=JSON.stringify (msg['books'][0]['average_rating']); 
        goodReadsRating= "<p> Average GoodReads Rating: " +reviewCount +"</p>"
        finalLine=divStart+ heading + divMiddle +divMiddleEnd +   "<div class=\"panel-body\" id=\"" + id + ".." + volumeID + ".." + title + ".." + author + ".." + desc + "\">"   + goodReadsRating + previewLinkLine + buttonFormat  +buttonFormat2+midButtonFormat; 
         $('#gallery').append (finalLine);




        })
         .fail(function() {
            console.log ("Failed, totally");
      });


})
 .fail(function() {
    console.log ("Failed before photos");

});

Basically, the structure is that I have a python file for the flask and javascript for the scripts running in the static webpages (and a MySQL database -using MySQLDB, from a few years ago).

However, I keep getting segfaults and "double free or corruption (!prev)" errors and, rarely, errors like "GC object already tracked".

I can't seem to figure out which side the error is coming from because it seems to be one of those floating errors (or the problem is simply systemic). I cut this out, the program segfaults on a different area.

I was wondering if anyone else had a way of debugging these sorts of programs? I tried GDB and it wasn't particularly useful.

Generally speaking, can I assume that a segfault would be coming from the javascript side? Would that be a place to start?

I do get some database hangs but I've changed databases a few times (from AWS to Google to local) just to check and I still keep getting segfaults when the errors seem to have been cut down on.

  • Segfaults in python either happen because of a bug in a C module (which doesn't seem to be the case here) or because you run out of memory (see [this question](https://stackoverflow.com/questions/19127777/a-python-segmentation-fault)). Are you maybe running out of memory? If not some imported library probably has bug, have you upgraded all dependencies? – syntonym May 29 '19 at 16:24
  • I don't see how I could be running out of memory. I barely write anything (if I do its to a SQL table) . The program is basically simply select and insert statements.Could the problem be in the Javascript file? EDIT: And I had to reinstall things like MYSQLDB and Flask all over to even get to this point so they should be fresh. – Gillard Dei May 29 '19 at 16:28
  • What does your RAM usage (e.g. per top) say when it crashes? Is it very high and then drops? Your program doesn't need to use lots of RAM if other programs use a lot. The error cannot be in the javascript if it is python which is segfaulting. It is python which is segfaulting? We need some more information what exactly is segfaulting. [This answer](https://stackoverflow.com/questions/10035541/what-causes-a-python-segmentation-fault) gives instructions to generate a C stacktrace on a python segfault. – syntonym May 29 '19 at 16:35
  • I ran it on GDB. It gave me: OperationalError: (2013, 'Lost connection to MySQL server during query') Thread 39 "python" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffffa510700 (LWP 193)] 0x00007ffffc0327d1 in ?? () from /usr/lib/x86_64-linux-gnu/libmysqlclient.so.20. I ran it on Windows 10's Ubuntu program though. Now I've switched and am running it straight from Windows 10 command line and I can't seem to get a segfault. So perhaps it was the memory in the Windows 10 Ubuntu environment? – Gillard Dei May 29 '19 at 17:03
  • When I run it in pure Windows command line I still get errors but no segfault, the error being:[2019-05-29 12:59:50,473] ERROR in app: Exception on /counter [PUT] Traceback (most recent call last): File "C:\Python27\lib\site-packages\flask\app.py", line 2311, in wsgi_app response = self.full_dispatch_request() ... The' TypeError: The view function did not return a valid response. The function either returned None or ended without a return statement. – Gillard Dei May 29 '19 at 17:03
  • It looks like your mysql connection somehow gets closed, and the SIGSEGV may be so incompatibility with the linux subsystem (and might be either a red hering or unrelated error). In the code you posted you are returning `None`. `None` is a not a valid response and flask doesn't know what to do with it. Return a valid response (like an error page). – syntonym May 30 '19 at 16:18

0 Answers0