Essays

Zettelkasten, Emacs, and Creative Thinking

An in-depth description of a Zettelkasten note taking practice that helps me improve clarity of thinking and creativity.

Creativity is connections. The more things you can associate together the more creatively you can solve problems. Writing is thinking. The more clear your writing, the more clear your thinking. It's the combination of creativity and clear thinking that leads to good results and I'm contantly looking for ways to invest in both.

Several months ago I started a Zettelkasten note taking practice. If you are not familiar, there are two key concepts 1) each note contains a single idea and 2) notes link to other related notes frequently. Over time this produces a tapestry of interlinked ideas that can lead to new connections and ideas.

The form factor of writing each note to contain a single idea has a few immediate benefits. First, by constraining the content to a single idea or concept, the note tends to be short. There's less pressure to write a note about a single thing compared to an essay about an entire topic and so you are more likely to do it. Second, writing it down in your own voice forces you to understand better than copying and pasting verbatim from the source or highlight it in a book. Clarity in writing is clarity in thought.

Much has been said about the value of bi-directional linking, but associations between ideas and concepts is where creativity lives. The more things you know, the more connections you can make between them, but it helps to make deliberate effort to find connections especially when it is top of mind. The more connections you make the more you can draw from to creatively solve new problems you encounter.

What follows describes how I implement my note taking practice using Emacs, org-roam, hugo, Working Copy, and GitHub Actions.

Zettelkasten Setup

In order to get this new habit started and sustain it, I like to keep in mind the framework from Atomic Habits—make it obvious, make it attractive, make it easy, make it satisfying.

Every day, first thing in the morning, I open up Emacs and start a journal entry using org-roam.

It's not hard to remember or even conjure up the necessary will power—I wake up early enough that there is nothing else competing for my attention and it's always the first thing I do.

I start off thinking I don't have much to say, but invariably, I end up writing a fair amount. I don't have any set prompts or template, but I usually touch on how I'm doing, reflections from the day before, and interesting things I've come across or learned.

More often than not, journaling leads to a few notes and connections being added to the collection. Next I review a list of ideas/thoughts/facts I've captured previously (more on this later). I find a few to turn into a note or two and look for related notes in my collection to link to.

Each time I add a note to the collection I anticipate how I might use it again and the potential to apply it in some way. I often find myself citing something from my notes during conversations daily which feels validating.

Throughout the day, when I come across some interesting idea or fact, I capture it using Beorg (although any bare bones note taking app will do). These quick notes are things I want to come back to because they are interesting and I want to remember or investigate further. Separating out writing the note from quickly capturing the idea makes it very easy to do, which is important because any delay and I'll likely forget.

Using org-roam and my familiar Emacs writing environment is simple and easy. Notes are text files in a single directory. No hierarchy, no pre-planning, just C-c n c to add a new note or C-c n ito insert a link, C-c n f to find a note, and C-c n l to see backlinks.

I made it prettier and removed distrations while writing notes by using writeroom-mode only when using org-roam. It looks nearly as nice as the super minimalist writing apps like Ulysses or iA Writer.

All files are version controlled using git and hosted in a GitHub repository which let's me sync across devices. Using the amazing Working Copy iOS app, I can write and publish notes from any of my devices. A Scriptable script makes it even easier by generating all the boilerplate that would be a pain to type out with my thumbs (e.g. file names, frontmatter).

Using this process I add anywhere between 2 - 5 notes per day. However, the most satisfying part, the stuff that keeps me going, is rendering those notes into a website and browsing around.

Publishing

Browsing notes and backlinks with org-roam is all well and good, but I get more enjoyment by perusing my notes on a snappy static website. I can browse notes from anywhere and it's a publc, visible reminder that knowledge is accumulating every day.

As knock-on effect, I find that working out in the open forces me to improve my understanding of what I'm writing. Maybe for the fear of putting something out in the public that isn't objectively good.

I wrote a pre-processor to ox-hugo which outputs org-roam notes exactly how I want. I use a private tag to exclude notes from being published (like journal entries). On export, I query the org-roam database to include a list of backlinks along with a short preview at the end of each note.

Because of the extra hop of exporting to hugo flavored markdown, I ended up with two repos—one for the org-mode notes and one for the exported markdown notes which gets automatically deployed by Netlify. This was painful because I always needed to deploy from my main Emacs workstation and had to commit and push to two different repos anytime I wanted to pubish.

To automate it, I wrote a GitHub Actions job which pulls down my Emacs config, exports to markdown, and commits the results to the hugo repo hooked up to Netlify. The result is publishing notes requires a push to one repo, no other steps required. Using Working Copy means I can also run my full workflow from any iOS device.

You can see the results for yourself at notes.alexkehayias.com.


Appendix

Scriptable script to reduce note boilerplate

Note: you need to create a folder 'bookmark' by going to Scriptable settings and creating a new bookmark. If the file path to your notes is different you will need to change the part that says fs.bookmarkedPath.

const ui = new Alert();
ui.addTextField("title", "");
await ui.present(true);
var raw_title = ui.textFieldValue(0);

if (raw_title===null || raw_title==="") {
  QuickLook.present("Note title can not be blank", false);
  Script.complete();
  return
}

const title = raw_title.toLowerCase().trim();
const d = new Date();

const date_title_formatter = new DateFormatter();
date_title_formatter.dateFormat = "yyyy-MM-dd--HH-mm-ss";
const datestring = date_title_formatter.string(d) + "Z";
const note_title = datestring + "--" + title.replace(/ /g, "_") + ".org";

const date_formatter = new DateFormatter();
date_formatter.dateFormat = "yyyy-MM-dd";
const date_str = date_formatter.string(d);

var fs = FileManager.iCloud();
const dir = fs.bookmarkedPath("notes");
const path = fs.joinPath(dir, "/" + note_title);
const preamble = `#+TITLE: ${title}\n#+DATE: ${date_str}\n#+ROAM_ALIAS:\n#+ROAM_TAGS:\n\n`
await fs.writeString(path, preamble);

// Make sure its actually there otherwise there will be a race condition
var ready = false;
while (!ready) {
  // There is no setTimeout in Scriptable so this will spin
  ready = await fs.read(path) !== {};
}

// Open the note in Working Copy
var cb = new CallbackURL("working-copy://open");
cb.addParameter("repo", "notes");
cb.addParameter("key", "SECRET_KEY_HERE");
cb.addParameter("path", note_title);
cb.addParameter("mode", "org-mode");
cb.open();

Script.complete();

GitHub Actions powered publishing

name: Publish

on:
  push:
    branches:
      - main

jobs:
  publish:
    runs-on: ubuntu-latest

    steps:
    - uses: purcell/setup-emacs@master
      with:
        version: 27.1

    - name: Checkout notes
      uses: actions/checkout@v2
      with:
        path: notes

    - name: Checkout hugo template
      uses: actions/checkout@v2
      with:
        repository: alexkehayias/zettel
        path: zettel
        token: ${{ secrets.ZETTEL_REPO_PAT }}

    - name: Checkout emacs config
      uses: actions/checkout@v2
      with:
        repository: alexkehayias/emacs.d
        path: .emacs.d

    - name: Install emacs packages
      run: |
        echo "Attempting to install packages..."
        ${EMACS:=emacs} -nw --batch \
          --eval '(let ((debug-on-error t)
                        (url-show-status nil)
                        (user-emacs-directory default-directory)
                        (user-init-file "'${GITHUB_WORKSPACE}'/.emacs.d/init-export.el")
                        (load-path (delq default-directory load-path)))
                    (load-file user-init-file)
                    (setq org-roam-directory "'${GITHUB_WORKSPACE}'/notes")
                    (run-hooks (quote after-init-hook)))'
        echo "Install successful"
    - name: Export to hugo compatible markdown
      run: |
        echo "Attempting export..."
        ${EMACS:=emacs} -nw --batch \
          --eval '(let ((debug-on-error t)
                        (url-show-status nil)
                        (user-emacs-directory default-directory)
                        (user-init-file "'${GITHUB_WORKSPACE}'/.emacs.d/init-export.el")
                        (load-path (delq default-directory load-path)))
                    (load-file user-init-file)
                    ;; Override the note and publish path to the workspace
                    (setq org-roam-directory "'${GITHUB_WORKSPACE}'/notes")
                    (setq org-roam-publish-path "'${GITHUB_WORKSPACE}'/zettel")
                    (run-hooks (quote after-init-hook))
                    (require (quote org-roam))
                    (org-roam-db-build-cache)
                    (org-roam-to-hugo-md))'
        echo "Export successful"
    - name: Install hugo
      uses: peaceiris/actions-hugo@v2
      with:
        hugo-version: '0.81.0'
        extended: true

    - name: Install stork-search
      run: |
        wget https://github.com/jameslittle230/stork/releases/download/v1.2.0/stork-ubuntu-latest
        chmod +x stork-ubuntu-latest
    - name: Generate search index
      run: |
        cd ${GITHUB_WORKSPACE}/zettel
        hugo && ${GITHUB_WORKSPACE}/stork-ubuntu-latest build -i public/search_config/config.toml -o static/search-index/index.st
        rm -r public
    - name: Push to static site repo if there are any changes
      run: |
        cd ${GITHUB_WORKSPACE}/zettel
        if [[ -z $(git status -s) ]]
        then
          echo 'No new public notes found. Exiting...'
          exit
        else
          echo "New notes detected. Committing..."
        fi
        git config --global user.name 'Alex Kehayias'
        git config --global user.email 'alexkehayias@users.noreply.github.com'
        git add --all
        git commit -am 'Auto commit from alexkehayias/notes GitHub Actions'
        git push