Problems with lua filter to iterate over table rows

Question

I'm trying to write a simply lua filter for pandoc in order to do some macro expansion for the elements in a ReST Table.

filter.lua

function tablelength(T)
  local count = 0
  for _ in pairs(T) do count = count + 1 end
  return count
end

function Table(table)

    elems=pandoc.Table(table)["rows"]

    print(tablelength(table))
    for v in pairs(elems) do
        print(v) -- Prints nothings
    end
    return table
end

test.rst

======= =========
A       B 
======= =========
{{x}}   {{y}}
======= =========

Now, if I run pandoc.exe -s --lua-filter filter.lua test.rst -t rst the program says that there are 5 elements in elems, but the for loop is just skipped and I really don't know what I'm doing wrong here.

I'm very new to Lua and also know pandoc very litte. How can I iterate over the elements in elems?

Answer 1

Pandoc lua-filters provide the handy walk_block helper, that recursively walks the document tree down and applies a function to the elements that match the key.

In the example below, we give walk_block a lua table (what's a map or dict in other languages) with only one key (the key Str), and the value of the table is the function to apply. The function checks for the braces, strips them and prepends foo.

function Table(table)
  return pandoc.walk_block(table, {
    Str = function(el)
      if el.text:sub(1,2) == '{{' then
        txt = 'foo' .. el.text:sub(3, -3)
      else
        txt = el.text
      end
      return pandoc.Str(txt)
    end
  })
end

Answer 2

There are a couple of areas of misunderstanding in your code. First you need to remember that everything in lua is a table (implemented as an associative array or dictionary), and arrays are just a special case of a table where the keys are integers. To avoid confusion, for the rest of this answer I will use Table when I'm referring to the pandoc document element, and table when I'm referring to the lua data structure.

Your tablelength function is just counting the number of elements in the pandoc table that represents a Table. If you look in https://www.pandoc.org/lua-filters.html#type-ref-Block you will see that a Table has 5 properties- caption, aligns, widths, headers and rows. The return value of this function is 5 because of this. If you print out the values in the loop inside tablelength then you will confirm this. If you want to count the rows then you will need to pass the rows array into the function, not the table.

The second problem is that you are creating a new table rather than using the one passed in by pandoc. Instead of using elems=pandoc.Table(table)["rows"] just use elems=table["rows"] or elems=table.rows which is equivalent. The function pandoc.Table() is used to create a new element.

Furthermore, to loop over the elements in a table that are in the form of an array, you can use the ipairs function- it will return the numerically indexed values as described here What is the difference of pairs() vs. ipairs() in Lua?.

The rows table is, as might be expected, an array of rows where each row is in turn an array of elements. So to access the elements of the table you will need to have two loops.

Finally there is the issue of the pandoc object model. Because a table can contain other things (images, links, bold text etc) the final cell value is actually a list of blocks. Now depending on what you want to do with the table you can deal with this in different ways. You can use the walk_block function that mb21 referred to, but looping over just the blocks in a single cell. If your Table only contains (unformatted) text then you can simplify things by using the stringify function, which collapses the list of blocks into a single string.

Putting all of this together gives the following modified version of your code.

local stringify=pandoc.utils.stringify

-- This function is no longer needed
function tablelength(T)
  local count = 0
  for e in pairs(T) do 
    count = count + 1 
    print(e) -- this shows the key not the value
  end
  return count
end

function Table(table)

    rows=table["rows"]

    print("TableLength="..#rows)
    for rownum,row in ipairs(rows) do
        for colnum, elem in ipairs(row) do
            print(stringify(elem)) -- Prints cell text
        end
    end
    return table
end

As to you followup question, if you want to modify things then you just need to replace the cell values, while respecting pandoc's object model. You can construct the various types used by pandoc with the constructors in module pandoc (such as the aforementioned pandoc.Table). The simplest table cell will be an array with a single Plain block, which in turn contains a single Str element (blocks usually contain a list of Inline elements).

The following code shows how you can modify a table using the existing content or the row/column number. Note that I changed the parameter to the Table function from table to tableElem because table is a common type used in lua and overriding it gives hard to track errors.

local stringify=pandoc.utils.stringify

function makeCell(s)
    return {pandoc.Plain({pandoc.Str(s)})}
end

function Table(tableElem)

    rows=tableElem["rows"]
    for rownum,row in ipairs(rows) do
        for colnum, elem in ipairs(row) do
            local elemText=stringify(elem)
            if elemText=="{{x}}" then
                row[colnum]=makeCell(elemText:gsub("x","newVal"))
            end
            if rownum==1 and colnum==2 then
                row[colnum]=makeCell("Single cell")
            end
        end
    end
    local newRow={ makeCell("New A"), makeCell("New B")}
    table.insert(rows,newRow)
    return tableElem
end