I am working on a Javascript app with i18n support. Translation tokens are spread all over the source code using the following format: __('some.token') or __('other.token.with.vars', {name: 'Julia', otherName: 'Nadine'})
Any ideas on how to extract all the tokens from the files? Is there any tool for such a thing? or do I need to write my own parser?
From my personal experience where I had to deal with a similar situation, I wrote my own parser in order to do the job. I hadn't found any tool which could make that extraction. I made my own tool in Java which provides a strong handling in files, strings and chars.
You can follow 2 different approaches: either you make a syntax analysis (you can find some info about that here) which provides you more accuracy at the handling of the tokens, or you can use regular expressions and then retrieve the found values (in Java you can do it as described here). In the former approach you can handle with more accuracy the process (you will be able to trace the errors exactly where they happen), while in the latter you might accomplish your goal more easily.
