How do I serialize a variable in VimScript?

Question

I wish to save a random Vim dictionnary, let's say:

let dico = {'a' : [[1,2], [3]], 'b' : {'in': "str", 'out' : 51}}

to a file. Is there a clever way to do this? Something I could use like:

call SaveVariable(dico, "safe.vimData")
let recover = ReadVariable("safe.vimData")

Or should I build something myself with only textfiles?

Answer 1

You can put to good use the :string() function. Test these:

let g:dico = {'a' : [[1,2], [3]], 'b' : {'in': "str", 'out' : 51}}
let str_dico = 'let g:dico_copy = ' . string(dico)
echo str_dico
execute str_dico
echo g:dico_copy

... so you can save the str_dico string as a line of a vimscript file (e.g. using writefile()), and then source the vim file directly.

Answer 2

Thanks to VanLaser (cheers), I've been able to implement these functions using string, writefile and readfile. This is not binary serialization but it works well :)

function! SaveVariable(var, file)
    " turn the var to a string that vimscript understands
    let serialized = string(a:var)
    " dump this string to a file
    call writefile([serialized], a:file)
endfun

function! ReadVariable(file)
    " retrieve string from the file
    let serialized = readfile(a:file)[0]
    " turn it back to a vimscript variable
    execute "let result = " . serialized
    return result
endfun

Use them this way:

call SaveVariable(anyvar, "safe.vimData")
let restore = ReadVariable("safe.vimData")

Enjoy!

Answer 3

Vim has built-in json serialization:

let dico = { "a" : 10, "b" : 20, }
call writefile(json_encode(dico)->split(), 'saved.json')
let recover = json_decode(readfile('saved.json')->join())

Using a data-only format for storing your dictionary lets you avoid code injection.

To support old versions of vim, see this answer.

Answer 4

I used @iago-lito's answer in a script I wrote a few years ago. Yesterday I spent some time improving on it. The vim dictionary is very similar to a JSON object, but:

1. when I open the file and set filetype=json, the linter complains about the single quotes around the strings, and
2. the JSON formatter splits the text into multiple lines, and indents them to make a pretty file. As a result, reading only the 0'th line of text doesn't give a complete dictionary object.

Here are my modifications to fix both issues.

function! SaveVariable(var, file)
    " Change all single quotes to double quotes.
    " {'x':'O''Toole','y':['A',2,'d']} -> {"x":"O""Toole","y":["A",2,"d"]}
    let serialized = substitute(string(a:var),"'",'"','g')
    " Change all escaped double quotes back to apostrophes.
    " {"x":"O""Toole","y":["A",2,"d"]} -> {"x":"O'Toole","y":["A",2,"d"]}
    let serialized = substitute(serialized,'""', "'",'g')
    call writefile([serialized], a:file)
endfunction

function! ReadVariable(file)
    execute 'let result = [' . join(readfile(a:file),'') . ']'
    return result[0]
endfunction

This seems to work well for all kinds of data. I tested it with an object, a list, and number and string scalar values.

STRANGER, DANGER!

Here is a word of warning that goes along with this and any other dynamically-generated code. Both @iago-lito's and my solution are vulnerable to code injection, and if you are reading files that are out of your control, bad things can happen to your machine. For example, if someone sneaks this into the file:

42|call system('rmdir /s /q c:\')|call system('rm -rf /')

calling @iago-lito's ReadVariable() will return 42, but your computer will be toast, whether it's a Windows, Mac, or Linux machine. My version also fails, albeit with a more complex version of the statement:

42]|call system('rmdir /s /q c:\')|call system('rm -rf /')|let x=[

A proper solution would be to parse the text, looking for the end of the actual data, and dumping everything after it. This means you lose the simplicity of this approach. Vim and Neovim have come a long way in recent years. lua and python are, from what I've read, easier than ever to integrate into vimscript. I wouldn't be surprised if either of those languages has a built-in answer to this question.