How do I display full string in a CodeQL exported result?

Question

Converting a.getChild(1) to a string using toString()

test.ql

/**
 * @id custom
 * @kind problem
 * @problem.severity warning
 *
 */
import javascript

from ObjectExpr oe, Property p1, int i, AstNode a
where p1 = oe.getProperty(i) and
    p1.getName() = "fragment" and
    a = p1.getAChild().getAChild() and
    a.toString().indexOf("name") > -1
select a, a.getChild(1).toString()

Here is the codeql command used for generating a csv result file:

codeql database analyze ~/test.com ./test.ql --format=csv --output=results.csv

For example: a.getChild(1).toString() = PageLoadablePageWrapperQuery will be saved in the csv file like PageLo ... rQuery instead of the full string.

How do I have the full string in the exported csv result?

Could you add a fragment of the JavaScript code that contains such an object expression with the correct properties and child structure? That would make it easier to reproduce and try out some approaches. — Marijn, May 25 '23 at 18:44

Marijn · Accepted Answer · 2023-05-26T06:47:59.203

For the JavaScript library in CodeQL the shortening of long names in .toString() is hardcoded*. See https://github.com/github/codeql/blob/7361ad977a5dd5252d21f5fd23de47d75b763651/javascript/extractor/src/com/semmle/js/extractor/TextualExtractor.java#L121:

public static String sanitiseToString(String str) {
    if (str.length() > 20) str = str.substring(0, 7) + " ... " + str.substring(str.length() - 7);

However, it is not very difficult to modify the source code such that codeql database analyze skips this part of the string sanitation. This can be done using the following steps:

download TextualExtractor.java from the url above and comment or remove the mentioned if-statement
find the location of extractor-javascript.jar in the installation of CodeQL, which is something like codeql/javascript/tools/
compile the modified file using the following line:

javac -cp ".:/path/to/codeql/javascript/tools/extractor-javascript.jar" TextualExtractor.java`

extract the jar file, replace TextualExtractor.class in the directory com/semmle/js/extractor/, and re-compress the jar file
re-install the javascript pack with codeql pack install --force from the directory that contains qlpack.yml, i.e., the project root
regenerate your CodeQL project database with

codeql database create dbname --overwrite --language=javascript --source-root=/path/to/project

run the query again.

Tested with the following JavaScript file:

var PageLoadablePageWrapperQuery = "some query";

var p = {  // object literal containing five property definitions
  longtobetruncatedfragment: PageLoadablePageWrapperQuery,
  y: 1,
  diag: function() { return this.x - this.y; },
  get area() { return this.x * this.y; },
  set area(a) { this.x = Math.sqrt(a); this.y = Math.sqrt(a); }
};

and the following query:

/**
 * @id custom
 * @kind problem
 * @problem.severity warning
 *
 */
import javascript

from ObjectExpr oe, Property p1, int i, AstNode a
where p1 = oe.getProperty(i) and
    p1.getName() = "longtobetruncatedfragment" and
    a = p1.getAChild()
select a, a.toString()

Resulting csv:

,,"warning","longtobetruncatedfragment","/testfile.js","4","3","4","27"
,,"warning","PageLoadablePageWrapperQuery","/testfile.js","4","30","4","57"

*Interestingly for other languages, for example the Python library, the sanitation step is not implemented.

Thanks for the detailed answer. But may I ask in which path do we execute `codeql pack install --force`? — Benassin, May 26 '23 at 00:50
It worked! By the way, do you have any idea why there is a sanitation step in the javascript library? — Benassin, May 26 '23 at 01:13
@Benassin you probably figured it out but `pack install` should be run to install `qlpack.yml`, from the directory that contains that file, which is the project root. About the reason for sanitation: I don't know, it might be that the messages are primarily intended to be displayed in an IDE like Visual Studio Code, and there the space in the window is limited, so in order to get a good overview long names are truncated. Also in the IDE you can click through to the source so you can find out easily what the full string is. — Marijn, May 26 '23 at 06:52

How do I display full string in a CodeQL exported result?

1 Answers1