For almost a year I was looking to get confortable enough with a tool kit for having my own website, and I tried several static site generators, and CMS’s but for some reason I never felt in peace with them, they were things I didn’t love about them, on several levels, surrounding dumb things like oh, this one is written in Ruby or this one doesn’t allow easy extensions.
In the meanwhile I disliked on other static site generators, I discovered pandoc, (which I’ll talk about in another entry), and my world changed, because pandoc is a tool that allows you to convert almost from any document format to any another document format, like to
html or markdown to
docx, etc… It has a long list of formats that can handle, and the best part about it, is that it’s extensible, without any pain, because It follows a very unixy way of thinking.
Not very long after discovering pandoc, I started trying to contribute (with no success), the first problem was that it’s writen in haskell which it’s a totaly differnt way of programming, due the fact that it’s a functional language, but that didn’t stop me to trying to hack it, because I found out about a project called
panflute, which is a python module that allows you to write filters for pandoc very easily.
In a way using pandoc felt alot like what I’ve read about compilers, specially when you understand the concept of an AST and middle representations. I used
panflute to alter the AST in with python, and the code for a filter looks something like this.
""" Set all headers to level 1 """ from panflute import * def action(elem, doc): if isinstance(elem, Header): elem.level = 1 def main(doc=None): return run_filter(action, doc=doc) if __name__ == '__main__': main()
This piece of code it’s a basic
panflut filter that will find all the headers in the document and it will turn them in to a header of level 1, so every section in the document after this filter is applied have the same level of importance you can say.
Now the idea of the filters I’ve put together to make this little engine work in my favore are:
- A filter that will take all
MathInlineelements, and it will catch them with the filter and process them with a
katexthat renders (to my taste) better than using
mathjaxbesides, I’m only using
- A filter that will take all
CodeBlocksand apply highlighting with a
pygmentsbecause for some reason I like a bit more how it outputs than the default that pandoc’s offers, also this filter is put in place because in the future I want to add support for the
rstformat which is renderd with a python module called
docutilsand I figure that for that moment I could try to use the same library for code highlighting in both types of files, making it me ship less
cssto the user.
When I talk about shipping no-
js to the users I’m lying alittle bit, because we’ll some things If you think about them could be done and make no harm to the user so as it is right now the index for the
blog section of the page, pulls all the data for it’s render (on the user side) from a
json file I generated and it uses
The so called engine is really simple, It only uses a
Makefile which you can extend or reduce as you want to include anything you like. This make file has some
PHONY rules to help you
debug some parts of the page, and others just to make the typical
clean. But the secrete sause here are the rules to process diferently the files you want to work with.
The engine works with the following structure:
At it’s root, you have
static you have what ever
css project, but it has to meet the to contain its own
Makefile such that the
all target calls for a full build of the project and stores it under
static/build because this file will be coppied to
resources. (Right now I’m using just
template directory you’ll be storing all the templates that you need to build with
jinja2. My suggested structure (and the one I use), is:
pandoc are exactly the same under template, but they just apply and insert the pandoc values in to their specific blocks. If you write inteligently the templates for pandoc you’ll need to make no change on them when you want to do a redesign.
scripts directory, you’ll find all the
misc scripts needed to build this (your) page, which could be a variaty of functions or scripts such as: (few of them I have in place but this are thougts for the future.)
pandocfitlers that you are using.
- A python script that I’m using to work with jinja and
- A script that allows me to query metadata from the html documents.
- A script that builds for me a sitemap and robots file.
And the most important directory of all
content directory which is a directory where you’ll be adding your content. The whole idea of this directory is to work with the same directory structure that you desire for the website but the files are actually in the format you desired such as raw
md files or what ever you like as long the rule exists on the Makefile.
All files that output
html will be subjected to the
jinja script that will take them and apply
template inheritance to them, so all the blocks will be put in place where they belong, and if file doesn’t have any sort of block specified on it, well it will not apply anything (as jinja works) allowing you to write whole html pages from scratch with totally different templates, but the whole idea of templates is to usethem consistently.
When you call for a build it’s done to for a page, and all pages are stored under the public directory, which will hold the finished and publishable site, and the make file will do what it needs to build the page in question.
There are more scripts I’m working on right now, to enhance this, becuase until now I didn’t understand the power for the
Makefile but if it’s allowing me to do this, simple static page generator, think that the
make utilitie is highly undervalued.
This project of the site will soon be on github, and I’ll be having proper documentation on all the scripts I’m using and how it’s orchestrated together, and the purpouse of it, is to help me build my personal website, and if this can express to someone the magic of
simple scripts I’ll be gald.