Racket / Pollen First Impressions

I’ve had my eye on Pollen for awhile now. Decided I'm going to give it a try, partly because I've been thinking about what to do when my current CMS is no longer supported, partly because I'm curious to mess with a LISP, and partly because enlightenment comes from doing.

Installation of Racket is simple on Mac but there's missing info for newcomers.

• The Racket download page defaults to the 32-bit build for OS X, but there's a 64-bit build available. Why not the 64-bit build? I went for the latter, hoping it won't come back to bite me later.
• The "easy steps" on Pollen's main page advise to 1) install Racket then 2) do raco pkg install pollen on the command line, I guess I have to manually add Racket's stuff to my PATH though.

I have my reservations about Racket's Markdown-parsing package, simply because every one of these inevitably supports a different set of features and produces different results for the same input. Digging into how it handles things and how well-maintained it is will be its own exercise, one I don't exactly relish.

[update: these reservations are shared by Pollen's creator, more below.]

First shots at the included Markdown parser:

• Out of the box, it supports "smartypants" style converting of quotes, ellipses, and em-dashes to the typographically correct glyphs.
• Also supports footnotes.
• Images placed on a line by their own are placed inside a <div class="figure"> with the alt text also placed within a nested <p class="caption">. This is nice, would be better to use <figure> and <figcaption>
• If there is any documentation on Racket's Markdown, I couldn't find it in 15 minutes of digging.

So far I think of Pollen like a Lego set. It has a picture of a nice website on the box (http://practicaltypography.com), and all the pieces. But unlike a normal Lego set, the instructions tell you A) how great Legos are because you can build anything you want, and B) how to connect two or three individual legos.

I'm not saying Pollen should include complete instructions to replicate the Practical Typography website, and the documentation does do an admirable job of explaining the very basics. But ideally it would include complete instructions to replicate something approaching the functionality of a modern CMS. The step-by-step instructions included with Legos don't limit you from building other things, they don't lessen the flexibility of the system; they just make it easier to learn by doing.

Sidebar on Pollen's appeal as a 3rd Way CMS

Right now with CMSs you typically see two approaches:

1. Use a database engine to store, retrieve and manipulate blocks of content and associated metadata
2. Get rid of the database and make the file system do the work of the database.

Option 2 can be faster and simpler to administer, because the queries most commonly needed by a CMS can actually be handled by file systems. Need the ten most recent posts in reverse chronological order? Just store the posts in filenames that use a yyyymmdd prefix. Need categories for your posts? Store all those posts under a folder named categoryname.

But as soon as you need anything more complex than a blog or a simple static site, though, Option 2 gets to be a real pain, because you're trying to get database results without using a database. And this results in even more complexity and friction than you would have by simply using a database.

What if you want a post to appear under more than one category? What if you want a keyword-based index? How about a comment system that where you actually possess the comments yourself instead of using Disqus? How about including new comments along with posts in chronological order? If your CMS depends on the filesystem to handle all its queries then you're looking at a lot of hacks to get what you want.

(Faced with the limitations of the database-less CMS, the common response is Yay Simplicity We Don't Need That Anyway. "Disable" comments. Get rid of categories. Get rid of search. Obviously this is a valid approach. But the ability to add, display, and link the same content in multiple organizational contexts is one of the core benefits of the web as a medium. There are experiences I want to offer readers that depend on this. Cutting yourself out of those possibilities in the name of simplicity seems kind of sad to me.)

So what's the 3rd way? It's treating the site (or the book, or whatever) as a program.

• You stop relying on formatting languages like Markdown and YAML for formatting and metadata: these are defined and implemented by other people, so you're always at their mercy when trying to manipulate your content
• You stop relying on the file system for any metadata. The file system still houses your content, sure, but you don't depend on paths and file names to store information needed to select and organize it.
• Your content is code (or structured data, depending on how you look at it). Each page of your site, being a program, loads and runs this code differently depending on the context.

I could keep going about whether transforming all one's writing into a programming exercise is a good idea, and for whom, but I'm trying to stay focused here.

Suffice to say A) I'm already aware that for the kinds of projects I'm working on, all my writing is going to involve programming one way or another, and B) at this point I just want to see whether, over the long term, Pollen is going to be more or less cumbersome than simply maintaining my current system (Markdown + Textpattern + Python kludges + some XML manipulations + etc).

(end sidebar)

Pollen's current design definitely does not contemplate generating multiple different formats from the same source content.

For example, I'm interested in taking a single set of files and converting it to both HTML and PDF. Not allowed: Pollen requires you to specify a single target format in the filename of the source file.

(Update: Multiple target formats are supported now, see here)

There's a discussion on Github that shows this is something MB is thinking about, though.

Well, there goes "3rd way" for now. I was [initially]disappointed to finally find this in the documentation:

Individual pages in a Pollen project don’t know anything about how they’re connected to other pages. In theory, you could maintain this information within the source files.

Pollen's idea for organizing multiple pages of content is to dictate that each page is its own isolated kingdom, and to use a static list of pages stored as a separate file, to organize them.

Apparently, Pollen contemplates no way for files to provide information about themselves for use in other files, no way for blocks of content to discover and relate to each other except through their relative positions in a rigid hierarchy (or a flat list).

(update: It turns out that the excerpt from the docs above isn't strictly true, and there are mechanisms for Pollen pages to parse data out of other pages; see below.)

I emailed some questions to MB and he was kind enough to respond.

There are ways of gathering metadata from other files. E.g. https://github.com/mbutterick/pollen/issues/57. It's just that Pollen per so provides no pre-cooked high-level mechanisms for doing so. It's expected that you'll learn the basic paradigms of Pollen, and the Racket language itself, in order to build what you need.

The more I look into it, the more appealing Pollen's approach becomes. It's pretty agnostic about the structure of your documents, which makes it very flexible.

Most of my difficulty at this point has to do with grokking LISP and learning the functional vocabulary for basic operations.

As of yesterday multiple targets are supported. So for any tag that I design in my custom markup, I can specify what output it will produce for HTML targets, and LaTeX, and plain text, etc. I blogged about it.