Introducing Builderoo
My MovableType plugin contest entry: Builderoo.
Builderoo is a text formatting system that removes some of the constraints normally placed on text filters. With Builderoo, you can switch filters in the middle of an entry, disable filters for a section of an entry, or apply multiple filters in any order to a section or the entire entry. Builderoo can cache the output of a filter for a section so the filter is not run again the next time the section is rendered (if the contents have not changed). Builderoo also includes a pre-filter macro facility, so macros can evaluate to text that is subsequently passed to filters, and can even change filters themselves. Builderoo has its own plugin API, so filters can interface with the Builderoo Console for web-adjustable configuration, or interface with the caching mechanism for cleaning up unused resources (e.g. generated images).
So text filters can do things like this:
That image was created by a MovableType plugin (included with Builderoo) from LaTeX mark-up embedded directly in the entry, like this:
<% Roo:latex %>
\begin{displaymath}
\int H(x,x')\psi(x')dx' = -\frac{\hbar^2}{2m}\frac{d^2}{dx^2}
\psi(x)+V(x)\psi(x)
\end{displaymath}
</% Roo %>Why switch text filters in the middle of an entry?
It is common to want to apply special transformations to just a section of an entry. However, the existing text filter mechanism only lets you apply one filter to an entire entry. If the selected text filter is to perform the transformation, one text filter plugin has to do everything. A common MovableType idiom to apply a transformation to a section of text is to implement the plugin doing the work as an MT template tag, then use Brad Choate's MTMacro plugin to scan text for special tags that you define, and replace those tags with MT template tags and re-build the results using the template engine. Sean Voisen's MTCodeBeautifier plugin is implemented this way.
Unfortunately, this requires that the delimiter tags and the special section must pass through the selected text filter before MTMacro can apply transformations, potentially causing compatibility problems with powerful, syntax-sensitive text filters, like Brad's MTTextile. One workaround is to explicitly code compatibility between the conflicting plugins, which produces the desired result, but restricts authors to using plugins that cooperate. (MTTextile detects the presence of MTCodeBeautifier to beautify sections described as source code using Textile markup.)
As seen above, Builderoo can change text filters in the middle of an entry using "Roo" tags. The Roo section only has the specified filters applied, and is isolated from the filters applied to the surrounding text. Only the text in the Roo section is passed to the LaTeX text filter, and the text is never touched by the filter applied to the rest of the entry, so LaTeX sees the mark-up as I typed it.
Similarly, the code sample below the image was rendered by copying and pasting the Roo section, and surrounding it with Roo tags specifying the Verbatim filter (an example filter included with Builderoo). (I also made a slight change to tell Builderoo that the Roo tags in the example are to be printed as typed, and not treated as real tags.) Roo tags can even specify that no filters be applied to the section.
Builderoo includes a sample text filter that simply applies MTCodeBeautifier to text. Using this filter and Builderoo, MTCodeBeautifier can be used for a section of code without concern for conflicts with other filters.
Why apply multiple text filters?
It is sometimes desirable to apply multiple transformations to the same section of text. Because the existing text filter mechanism only allows one filter to be used at a time, once again, the selected filter must either perform every transformation, or the task must be handed down to MT template tags surrounding the entry. And again, with template tags, the text must first pass through the selected filter before reaching the tags, forcing the order in which transformations are applied.
John Gruber's SmartyPants plugin converts ASCII representations of single-quotes, double-quotes, apostrophes, dashes and ellipses to the HTML code for their typographical characters. Presumably because users are likely to want to use it in conjunction with another text filter, it was implemented as a template tag. Explicit compatibility with SmartyPants was added to several popular filters, including MTTextile, and John's own Markdown filter.
With Builderoo, SmartyPants could be implemented as a text filter, then applied to text in addition to other filters. The filters can be applied in any order. Builderoo includes a sample text filter that simply applies SmartyPants transformations to text.
Why cache the output of a text filter?
MovableType expects text filters to be very fast. With a basic installation of MT 3.0D, an entry published for the first time is run through its text filter as many as seven times as the site is built. A text filter that takes longer than a split second to process an entry could seriously slow down a site re-build, and use up a great deal of resources on a web hosting system. As such, text filters are not expected to do more than convert text into other text.
By caching the results of building a section the first time it is published, Builderoo allows for text filters that take several seconds to perform a task. A text filter can generate an image or an applet, or access a web service. The next time the entry is rendered (in the same build, or in subsequent builds, or the same section appearing in multiple entries), the filter is not run, but instead the cached version is used.
Builderoo assigns a "cache ID" to a cached section, and passes this ID to the filter. Filters that generate images, such as the LaTeX filter used above, can use the ID to uniquely identify the image. Furthermore, Builderoo uses MT 3.0's new plugin API to detect when a section is no longer used on the weblog (the last instance has been removed, or the only instance has changed), to remove the unused section from the cache. A plugin can tell Builderoo to call it when a cached section is purged, so unused external resources (like images) can be cleaned up automatically.
Why macros?
Now that filters are potentially more powerful with Builderoo, macros are useful to define common or complex content to quickly insert into entries. Builderoo applies macros before doing any other processing, so macros can contain Roo tags (or parts of Roo tags), expand to text that is then passed to other filters, or expand to text that is isolated from text filters.
Why Builderoo?
My target use case was to embed Lilypond mark-up in an entry, and have it transformed into musical score. A Lilypond text filter is included with Builderoo. See the Builderoo documentation for a demo. (I'd demo it here, but I do not yet have Lilypond installed on my web hosting—admittedly a distinct hurdle for those wishing to do major content processing with Builderoo.)
My hopes for fabulous contest winnings are mixed, I'm afraid, as I found several small but nasty bugs after I had submitted my entry. They went undetected because I didn't do enough testing in multiple web hosting environments. Here's hoping the judges are all using Mac OS X and Apache. :)
braindan,
I have no fucking idea what you just said but I'm sure it was effing brilliant cause you are one effing brilliant individual.
I, on the other hand, am one soused individual! Woop!