Skip messages via Jinja template

[kapgit]

The context: Using a DB as primary datasource

I'm using mailmerge, using a MySQL database as datasource.
This DB contains data and a number of custom views. Each view queries the data and returns a row for each mail to be sent with mailmerge.

As mailmerge is accepting only CSV file as input (as far as I know), and not a DB view, a little utility bash script

send_to <view> <mail_template>

is executing the DB <view> and generates a transient CSV file containing the selected rows.

The script then calls mailmerge using the path to this CSV file and a Jinja2 <mail_template> as input parameters.

Mailmerge does its job and sends a mail for each line in the CSV file. Perfect!

Issue:

Sometimes, I need to quickly send a mail to a subset of the mysql <view>. One shot. No need to reuse the query.
I could of course create/alter the DB view, but that's not as agile as I would like and could quickly lead to proliferation of views, all with their own little variations on some 'master' view.

Question:

Is is possible to use some Jinja2 tagging in the mail template, that would have as an effect to skip the current record?

Something like:

{% if condition_based_on_current_record -%}
TO: {{email}}
SUBJECT: Interesting mail
FROM: My Self <[email protected]>

Dear {{name}},

You are the lucky recipient of this email.

Best regards,

Me
{% endif -%}

If condition_based_on_current_record is true then the mail is sent.
Otherwise, the sender, recipient and message are all empty, and mailmerge would skip the record.

Thank you.

[kapgit]

Maybe there is a better method, but at the moment, I'm applying a quick and dirty solution that seems to work for me:

In template_message.py (commented the 3 assertions):

    def render(self, context):
        """Return rendered message object."""
        try:
            raw_message = self.template.render(context)
        except jinja2.exceptions.TemplateError as err:
            raise MailmergeError(
                "{}: {}".format(self.template_path, err)
            )
        self._message = email.message_from_string(raw_message)
        self._transform_encoding(raw_message)
        self._transform_recipients()
        self._transform_markdown()
        self._transform_attachments()
        self._message.__setitem__('Date', email.utils.formatdate())
        #assert self._sender # COMMENTED
        #assert self._recipients # COMMENTED
        #assert self._message # COMMENTED
        return self._sender, self._recipients, self._message

In __main__.py (enclosed the 'sendmail' in an 'if' statement):

def main(sample, dry_run, limit, no_limit, resume,
         template_path, database_path, config_path,
         output_format):
...
   try:
        template_message = TemplateMessage(template_path)
        csv_database = read_csv_database(database_path)
        sendmail_client = SendmailClient(config_path, dry_run)
        for _, row in enumerate_range(csv_database, start, stop):
            sender, recipients, message = template_message.render(row)
            if sender and recipients and message: # ADD THIS LINE + INDENT FOLLOWING
                sendmail_client.sendmail(sender, recipients, message)
                print_bright_white_on_cyan(
                    ">>> message {message_num}"
                    .format(message_num=message_num),
                    output_format,
                )
                print_message(message, output_format)
                print_bright_white_on_cyan(
                    ">>> message {message_num} sent"
                    .format(message_num=message_num),
                    output_format,
                )
                message_num += 1

    except MailmergeError as error:
...

[awdeorio]

One way to do it is to make a copy of the CSV file and delete some of the rows with a text editor. Another way to do it is to filter the CSV file using command line tools like grep or csvkit.

Finally, here's an example of how to use the command line options to skip a message, e.g., message 11.

$ mailmerge --limit 10
$ mailmerge --resume 12

[bexelbie]

It'd be great to see a flag or similar for this if we can do it in a UI friendly way. I have a routine email that I send that needs to come from one of two email addresses. Today I have to bifurcate my data set and reset the conf for each run. Not having to bifurcate the data set would be a small win.

[awdeorio]

Here's a brain dump of some different ways this could work. Feedback welcome.

1. Include row numbers, similar to the cut -f option. From man cut:

   $ mailmerge --include-rows 1-10,13-20
   $ man cut
   ...
   Use one, and only one of -b, -c or -f.  Each LIST is  made  up  of  one
   range,  or  many ranges separated by commas.  Selected input is written
   in the same order that it is read, and is written exactly  once.   Each
   range is one of:
   
   N      N'th byte, character or field, counted from 1
   
   N-     from N'th byte, character or field, to end of line
   
   N-M    from N'th to M'th (included) byte, character or field
   
   -M     from first to M'th (included) byte, character or field

2. Exclude row numbers, e.g.:

   $ mailmerge --exclude-rows 11-12

3. Pattern match against a column, similar to csvgrep.

   $ mailmerge --column email --match umich.edu
   $ mailmerge --column email --match umich.edu --invert-match

4. Skip message if entire rendered template is blank. This is similar to the suggestion of @kapgit above. I'd perhaps couple it with a command line flag. Perhaps check for blank messages and produce a helpful error message, suggesting this flag.

   $ mailmerge --skip-blank

   And the corresponding mailmerge_template.txt:

   {% if condition_based_on_current_record -%}
   TO: {{email}}
   FROM: [email protected]
   
   Hello world
   {% endif -%}
   

5. Skip message if TO field is blank. EDIT: possible bug could be if there's a CC or BCC field. TO, CC and BCC all end up in the senders list.

   $ mailmerge --skip-blank-to

   And the corresponding mailmerge_template.txt:

   {% if condition_based_on_current_record -%}
   TO: {{email}}
   {% endif -%}
   
   FROM: [email protected]
   
   Hello world
   

I'm leaning towards option 4, similar to @kapgit 's suggestion above.

[bexelbie]

@awdeorio I like what you have. FWIW, I'd like to express my preference for number 5 because it is smaller and keeps the if small to prevent weird nesting typos later.

[captn3m0]

+1 for (5). Empty to currently results in a assertion error, so its anyway not a supported usecase.

[awdeorio]

I can get behind (5). Any volunteers? :)

I think this would involve:

• Code to add command line flag
• Code to add "skip" functionality
• Unit test: basic, with both skipped and not-skipped messages. Include CC, BCC.
• Unit test: jinja logic results in blank "TO". User forgets the command line flag. Probably should error with a helpful message in this case.
• Documentation. Maybe an "advanced usage" subsection?