Using Django's urlize with CommonMark

Question

I'd like to use Django's urlize function together with django-markwhat's CommonMark filter.

I'm imagining something like this:

{{ "http://example.com"|urlize|commonmark }}

Which outputs <p><a href="http://example.com">http://example.com</a></p>. The issue here is that URLs already marked up in commonmark, like <http://example.com>, will get rendered with angled brackets around them.

If I apply urlize after commonmark, like this:

{{ "http://example.com"|commonmark|urlize }}

The urlize function doesn't detect the url in <p>http://example.com</p> (and it's mentioned in the urlize docs that it won't work as expected for html input).

I haven't come up with a solution yet, so let me know if you have any ideas.

I'm confused. Doesn't CommonMark convert the [autolinks](http://spec.commonmark.org/0.18/#autolinks) (links with angle brackets around them) to proper links? The urlize filter shouldn't be needed for them. Seems to me like your first example should work just fine. What output is it giving? — Waylan, Apr 08 '15 at 13:25
CommonMark's version of autolinks are working fine, that's correct. In my application, I want even links that aren't surrounded by `<` and `>` to be converted to `` tags. I realize this isn't in the CommonMark spec, but it's common for websites to auto-markup recognized links, an example implementation is https://github.com/jch/html-pipeline — nnyby, Apr 08 '15 at 13:32
My first example works on non-marked-up links, like `http://example.com`. It breaks for markdown autolinks, like ``, putting extraneous brackets around the link, like `<
http://example.com
>` — nnyby, Apr 08 '15 at 13:36
Actually, CommonMark does handle autolinks (according to the spec as I linked in my previous comment). In any event, I see your problem now; urlize is breaking the autolinks. I believe that you can't use urlize for this. Might want to look at some third party filters that do a better job of this. Perhaps a urlize filter that works on HTML which can be run on the output of CommonMark. — Waylan, Apr 08 '15 at 13:51
Another option is to loose CommonMark and use a Markdown parser (CommonMark is not Markdown IMO). In fact, various [extensions](https://github.com/waylan/Python-Markdown/wiki/Third-Party-Extensions#url-conversion) exist which build your desired behavior right into Python-Markdown. — Waylan, Apr 08 '15 at 13:55
I realize it handles autolinks, I guess there's some confusing terminology. When I say "autolink" I think, converting a plain link like `http://example.com` to html, not a link surrounded by angle brackets. I was using Python-Markdown previously, it works well, but it implements a slightly different version of Markdown than what I'm using on the front-end: Remarkable.js, which caused confusing behavior. I'm thinking that disambiguating the syntax with CommonMark is the right direction forward. — nnyby, Apr 08 '15 at 14:13

score 4 · Accepted Answer · answered Apr 10 '15 at 19:40

For completeness, the easy answer is to switch to a Markdown parser which provides the behavior you want out of the box. Multiple extensions exist which give Python-Markdown the desired behavior.

However, if you want this behavior with CommonMark, which does not support this behavior itself, then you will need to create a custom urlize filter which operates on HTML. Fortunately, The Bleach library provides a linkify function which will handle that correctly.

Of course, you then need to create a template filter which wraps that:

from django import template
import bleach
from html5lib.tokenizer import HTMLTokenizer

register = template.Library()

@register.filter(is_safe=True)
def linkify(value):
    return bleach.linkify(value, skip_pre=True, parse_email=True, tokenizer=HTMLTokenizer)

Note that I'm assuming you do want to parse email addresses as well as URLS, you do not want to have URLs in code blocks parsed, and you do not want CommonMark's output sanitized. Feel free to review Bleach's docs and adjust accordingly.

To use your custom filter, save the above code to a file named linkify.py within a templatetags directory in your app (see the Django docs for a full explanation). Then within a template, do the following:

{% load linkify %}

{{ "http://example.com"|commonmark|linkify }}

Using Django's urlize with CommonMark

1 Answers1