[Feature request] Permit custom extras

[JeffAbrahamson]

It would be quite helpful to have an interface to add custom extras (not necessarily all extras) as needed.

For example, we use markdown2 with django, and we'd like to permit editors to insert certain types of buttons into their text. With static templates, we use the django template tags feature.

It would be great to be able to define a syntax for ourselves, such as [[button:text]]((button link)) that we could expand to some rather complicated bit of html.

A related use is internal links, where we'd like to do [[internal:text]]((link)) for an internal link that works with django's url system.

These are use cases that I don't think merit library distribution, but if there were a hook system so that we could declare a custom filter, this would be quite useful.

Our work-around is to post-process the entire string coming out of markdown2, which isn't hideous but could be better if markdown2 supported some sort of filters.

[JeffAbrahamson]

For what it's worth, and partly just for info to far future browsers of this issue, what I eventually did was to write a small FSM that implemented the syntax I wanted, then wrote this django templatetag:

@register.filter(name='markdown')
def safe_markdown(value):
    parser = TNLinkParser(verbose=True)
    return mark_safe(markdown(parser.transform(escape(value))))

[einenlum]

I agree with @JeffAbrahamson.

Would be nice to decouple the core object from the extras. This way, it would make it easier for the users to create and use their own extras on their project.

@nicholasserra Would you be open to a refactoring (if someone makes it) to make the extras decoupled and easily injected?

I'm currently using markdown2 but I have some specific needs and it's not that easy to know where to start to make a custom extra.

[nicholasserra]

Yeah I don't like how closely tied in the extras are to the main code. That being said, I don't think it's really possible to pull them out in any meaningful way, it's a big regex mess. If it can be done i'd be all for it. I just don't see it happening without redoing the core way this library works, which at that point might not be worth it. Maybe take a closer look and see where you get?

[Crozzers]

I did a bit of nosing about the code and I think that we could definitely decouple some of the extras. Things like admonitions or the numbering extra aren't too tightly integrated. Others like wavedrom depend on some of the regex from fenced-code-blocks.

I think as long as there's some way to piggyback off other extras (by class inheritance most likely) and a mechanism to control the execution order then it's doable.
I have something roughly like this in mind:

class Extra:
    def run(self, text):
        ...

class FencedCodeBlocks(Extra):
    something_regex = r'something'

class Wavedrom(FencedCodeBlocks):
    before = FencedCodeBlocks

There would also need to be a way to mark extras as block level extras or span level extras, as well as what stage in the pipeline they occur. From a quick look, the stages, minus any extras, are:

• preprocess
• hash html spans/blocks
• strip link definitions
• run block gamut
  • headers
  • lists
  • code blocks
  • block quotes
  • hash html part 2
  • paragraphs
    • span gamut
      • code spans
      • escape special chars
      • links + autolinks
      • encode &<>
      • italics and bold
• postprocess
• unhash html

If we had these in an Enum class or something and could do

class Something(SpanExtra):
    before = OtherSpanExtra
    stage = Stage.LINKS

This is all in theory with none of it doable right now but I think the main things would be

• Having a standard Extra base class
• Each extra can define execution order somehow
• Each stage will execute associated extras in an order that makes sense

[nicholasserra]

Thank you for the nice write up. Sounds like a good plan, if we ever decide to go for it.