Mail merge in ruby on rails, but in big numbers

Question

I am trying to implement sort of mail merge for printed documents in Ruby on Rails 3.2. I have about 8000 recipients and template origin in Microsoft Word. Template includes images (photos) and contains about 10-20 pages.

Actual situation is, that I rewritten original template to Textile (redcloth) and pictures are inserted from internet (http address). I did all personalisation etc. So I generate HTML file and must divide it to many small files each for 1000 pages. Total I need print about 8000 x 20 pages = 160.000 pages.

Anyone know how to print it to PDF from HTML? Or how to insert commands for changing paper tray (for first and last page) or for binding after each 20 pages etc?

Thank you for any idea :-)

When you say "mail merge" you really mean "template substitution", right? — tadman, Sep 06 '12 at 16:22
Can you use [wkhtmltopdf](http://code.google.com/p/wkhtmltopdf/)? — PinnyM, Sep 06 '12 at 16:26
Each page in wkhtmltopdf is about 500kb => 160.000 pages x 500 kb = 74.5 GB :-( — dominikform, Sep 06 '12 at 16:29
You're splitting that 74 GB over 8000 users, which is only about 10 MB each. Subtracting for compression(zip, gzip, etc.), this doesn't sound like a big deal. — jdl, Sep 06 '12 at 16:54
Do you think for each user make one PDF? Is possible to add commands for changing paper tray and binding in PDF? — dominikform, Sep 06 '12 at 17:22
Problem is, that we generate mailmerge on heroku and store files on S3 amazon. For local printing we will must download all PDFs that will be 75 GB :-( — dominikform, Sep 06 '12 at 17:30

score 0 · Answer 1 · answered Oct 10 '12 at 20:28

Here's one idea: in your rails app, set it up to return one html per user. Also, have a nice /users/ index method that returns a list of users in something convenient, maybe json format.

Now, you want a local script, written in ruby, bash, whatever is convenient, to:

fetch a list of users from that /users/ method, probably save it to a file
loop over the list of users (from the file, so they're not all in memory), and fetch the HTML of the email
generate pdf from each HTML downloaded, either inside the loop, or loop over files in a directory where you saved the HTML. Use wkhtmltopdf or similar.
send each pdf to the printer, again either inside the same loop, or loop over the saved pdf's.

If you wanted to get fancy, and a little more efficient, you could use a queueing system like resque, and make each of those bullet points into a queue, and run one worker per queue. That would let you start printing some pdfs while others were still being downloaded and converted, so it should be less time overall. But if you're not already familiar with a queuing system like that, a simple script should get it done as well.

Mail merge in ruby on rails, but in big numbers

1 Answers1