ekiim's blog

05-05-2019

For almost a year I was looking to get confortable enough with a tool kit for having my own website, and I tried several static site generators, and CMS’s but for some reason I never felt in peace with them, they were things I didn’t love about them, on several levels, surrounding dumb things like oh, this one is written in Ruby or this one doesn’t allow easy extensions.

In the meanwhile I disliked on other static site generators, I discovered pandoc, (which I’ll talk about in another entry), and my world changed, because pandoc is a tool that allows you to convert almost from any document format to any another document format, like $\LaTeX$ to html or markdown to docx, etc… It has a long list of formats that can handle, and the best part about it, is that it’s extensible, without any pain, because It follows a very unixy way of thinking.

Not very long after discovering pandoc, I started trying to contribute (with no success), the first problem was that it’s writen in haskell which it’s a totaly differnt way of programming, due the fact that it’s a functional language, but that didn’t stop me to trying to hack it, because I found out about a project called panflute, which is a python module that allows you to write filters for pandoc very easily.

In a way using pandoc felt alot like what I’ve read about compilers, specially when you understand the concept of an AST and middle representations. I used panflute to alter the AST in with python, and the code for a filter looks something like this.

"""
Set all headers to level 1
"""

from panflute import *

def action(elem, doc):
elem.level = 1

def main(doc=None):
return run_filter(action, doc=doc)

if __name__ == '__main__':
main()


This piece of code it’s a basic panflut filter that will find all the headers in the document and it will turn them in to a header of level 1, so every section in the document after this filter is applied have the same level of importance you can say.

Now the idea of the filters I’ve put together to make this little engine work in my favore are:

• A filter that will take all MathBlocks or MathInline elements, and it will catch them with the filter and process them with a javascript library called katex that renders (to my taste) better than using mathjax besides, I’m only using javascript for the build using node.js and no javascript code will get shipped to the user (Regarding the math rendering).
• A filter that will take all CodeBlocks and apply highlighting with a python module called pygments because for some reason I like a bit more how it outputs than the default that pandoc’s offers, also this filter is put in place because in the future I want to add support for the rst format which is renderd with a python module called docutils and I figure that for that moment I could try to use the same library for code highlighting in both types of files, making it me ship less css to the user.

When I talk about shipping no-js to the users I’m lying alittle bit, because we’ll some things If you think about them could be done and make no harm to the user so as it is right now the index for the blog section of the page, pulls all the data for it’s render (on the user side) from a json file I generated and it uses pure-javascript to render all the cards of summary for all the posts.

The so called engine is really simple, It only uses a Makefile which you can extend or reduce as you want to include anything you like. This make file has some PHONY rules to help you debug some parts of the page, and others just to make the typical build and clean. But the secrete sause here are the rules to process diferently the files you want to work with.

The engine works with the following structure:

At it’s root, you have

• content
• public
• static
• scripts
• template
• Makefile

In static you have what ever js or css project, but it has to meet the to contain its own Makefile such that the all target calls for a full build of the project and stores it under static/build because this file will be coppied to resources. (Right now I’m using just scss compiler)

In your template directory you’ll be storing all the templates that you need to build with jinja2. My suggested structure (and the one I use), is:

• template
• pandoc
• base.htm
• blog.htm
• promo.htm
• sections
• footer.htm
• header.htm
• nav.htm
• base.htm
• blog.htm
• promo.htm

So the template under pandoc are exactly the same under template, but they just apply and insert the pandoc values in to their specific blocks. If you write inteligently the templates for pandoc you’ll need to make no change on them when you want to do a redesign.

Under the scripts directory, you’ll find all the misc scripts needed to build this (your) page, which could be a variaty of functions or scripts such as: (few of them I have in place but this are thougts for the future.)

• pandoc fitlers that you are using.
• A python script that I’m using to work with jinja and stdin.
• A script that allows me to query metadata from the html documents.
• A script that builds for me a sitemap and robots file.

And the most important directory of all content directory which is a directory where you’ll be adding your content. The whole idea of this directory is to work with the same directory structure that you desire for the website but the files are actually in the format you desired such as raw html, md files or what ever you like as long the rule exists on the Makefile.

All files that output html will be subjected to the jinja script that will take them and apply template inheritance to them, so all the blocks will be put in place where they belong, and if file doesn’t have any sort of block specified on it, well it will not apply anything (as jinja works) allowing you to write whole html pages from scratch with totally different templates, but the whole idea of templates is to usethem consistently.

When you call for a build it’s done to for a page, and all pages are stored under the public directory, which will hold the finished and publishable site, and the make file will do what it needs to build the page in question.

There are more scripts I’m working on right now, to enhance this, becuase until now I didn’t understand the power for the Makefile but if it’s allowing me to do this, simple static page generator, think that the make utilitie is highly undervalued.

This project of the site will soon be on github, and I’ll be having proper documentation on all the scripts I’m using and how it’s orchestrated together, and the purpouse of it, is to help me build my personal website, and if this can express to someone the magic of make and simple scripts I’ll be gald.