Parsing Time

Interactive Relative Priorities in Org Agenda

agenda

The Problem

Priority buckets have never really worked for me. Jira, for example, lets you prioritise things by assigning them one of the priorities "Highest", "High", "Medium", "Low", "Lowest", etc. Emacs subscribes to something similar - you can introduce more, but typically you have the letters "A", "B", "C", etc. to specify priority.

If I manage to keep on top of assigning priorities, typically I end up with a bucket full of "A"-level tasks, all of which are "high priority" but of which I can only do one at a time realistically. It's not really clear which of these I should work on first at a glance. Additionally, assigning the priority "B" or lower to a task feels like I'm condemning it for eternity, to be marked as cancelled with a sigh in a few months.

I would prefer to manage priorities as I do in Jira - rank everything from highest to lowest priority, and then work on the top thing first.

It's been on my wishlist for ages to get this functionality into my Emacs agenda. It's almost already supported - org-agenda-drag-line-backward and org-agenda-drag-line-forward, invoked by M-<up> and M-<down> in the org agenda, move lines in a todo-tags block up and down, regardless of what default ordering you've set up in org-agenda-sorting-strategy.

However, these functions don't actually persist the order - as soon as you refresh the agenda, everything reverts to your default ordering.

Thankfully, this is Emacs! We can do whatever we want!

The Solution

The goal is to persist the ordering set by org-agenda-drag-line-backward and forward. Instead of trying to change those functions, I instead decided to think about how I'd implement custom ordering without those functions. I will come back to them later though.

We need a few things:

Custom agenda sorting strategy

I started my journey by checking out the documentation for org-agenda-sorting-strategy (using C-h v to invoke describe-variable). Although it took me a little while to realise it was there amongst the wealth of options, sure enough, it's possible to add custom ordering:

user-defined-up    Sort according to ‘org-agenda-cmp-user-defined’, high last.
user-defined-down  Sort according to ‘org-agenda-cmp-user-defined’, high first.

So, as long as I have org-agenda-cmp-user-defined set to my own comparison function that encorporates the new :RANK: property, I should be able to get the ordering I want by setting org-agenda-sorting-strategy to user-defined-down or user-defined-up, depending on the value of :RANK:. I am considering higher numerical values of :RANK: to be higher priorities for reasons I'll go into later, so I want user-defined-down.

I then checked the documentation for org-agenda-cmp-user-defined:

A function to define the comparison ‘user-defined’. This function must receive two arguments, agenda entry a and b. If a>b, return +1. If a

It works like a lot of first-order comparison functions do - negative value for less-than, positive value for greater-than, 0 (nil) for equal.

My comparison function just needs to dive inside the agenda entry and grab the value of the new property, then compare that numerically. If the property isn't set on either entry, the other one is considered 'greater'.

There is most likely a better way of implementing the actual numerical comparison part here, but I just went ahead and wrote it out explicitly.

    (defun rostre/org-agenda-rank-cmp (a b)
      (let* ((apos (get-text-property 0 'org-marker a)) ; get marker for a
             (bpos (get-text-property 0 'org-marker b)) ; get marker for b
             (pa (org-entry-get apos "RANK")) ; get RANK value for a
             (pb (org-entry-get bpos "RANK"))) ; get RANK value for b
        (cond ((not pa) -1) ; RANK is not set for a
              ((not pb) 1) ; RANK is not set for b
              ((> (string-to-number pa) (string-to-number pb)) 1)
              ((< (string-to-number pa) (string-to-number pb)) -1)
              (t nil))))

Now in my agenda, I add the org-agenda-cmp-user-defined and org-agenda-sorting-strategy variable settings:

    ...
    (tags-todo "+TODO=\"TODO\""
               ((org-agenda-overriding-header "Todo")
                (org-agenda-cmp-user-defined 'rostre/org-agenda-rank-cmp)
                (org-agenda-sorting-strategy '(user-defined-down))
                ...

Swapping entries

Now to tackle these points:

Both commands can be implemented by swapping the rank property of one entry with the rank property of another entry in the general case. The only difference is which entries are being swapped.

Swapping the values given two markers that point to the org headings isn't too complicated. We save the current value of the rank for the second entry, set the value of the rank for the second entry to the current value of the first entry, and finally set the rank of the first entry to the saved value.

    (defun rostre/org-agenda-swap-ranks (curr-org-marker target-org-marker)
      (let ((target-old-rank (org-entry-get target-org-marker "RANK")))
        (progn
          (org-entry-put target-org-marker "RANK" (org-entry-get curr-org-marker "RANK"))
          (org-entry-put curr-org-marker "RANK" target-old-rank))))

To specialise this for M-p, we need to find the markers for the entries at point and one line up from point.

I looked at the code for org-agenda-switch-to to find out how to get the marker at point. This is the function that's invoked when you hit <RET> on an org agenda entry, which opens up the original heading in its original file. I only knew this because keycast told me, which is a package that reports the last keybinding invoked in the modeline, plus what function that is bound to. It's super useful when trying to understand how existing things work and stealing their ideas!

Anyway, org-get-at-bol can be used to get the marker for the entry at point, and we can temporarily move point up a line to get the marker for that entry as well. Then we just call the function I defined to swap the rank values:

    (defun rostre/org-agenda-swap-ranks-up ()
      (interactive)
      (let ((curr-org-marker ; org-marker for heading at current line
             (or (org-get-at-bol 'org-marker) (org-agenda-error)))
            (target-org-marker ; org-marker for heading on previous line
             (save-excursion
               (org-agenda-previous-line)
               (or (org-get-at-bol 'org-marker) (org-agenda-error)))))
        ; swap the rank of the heading at the current line with that of the line above
        (rostre/org-agenda-swap-ranks curr-org-marker target-org-marker)))

The code is basically the same for M-n - we just have to navigate down a line rather than up a line.

    (defun rostre/org-agenda-swap-ranks-down ()
      (interactive)
      (let ((curr-org-marker ; org-marker for heading at current line
             (or (org-get-at-bol 'org-marker) (org-agenda-error)))
            (target-org-marker ; org-marker for heading on next line
             (save-excursion
               (org-agenda-next-line)
               (or (org-get-at-bol 'org-marker) (org-agenda-error)))))
        (progn
          ; swap the rank of the heading at the current line with that of the line below
          (rostre/org-agenda-swap-ranks curr-org-marker target-org-marker))))

To actually bind these functions to M-n and M-p, I'll add a hook that sets up a local keybinding in org-agenda-mode only:

    (add-hook 'org-agenda-mode-hook
            (lambda ()
              (progn
                (local-set-key (kbd "M-n") 'rostre/org-agenda-swap-ranks-down)
                (local-set-key (kbd "M-p") 'rostre/org-agenda-swap-ranks-up))))

This works! But you have to manually refresh the agenda each time to see the changes…

Speedy interactive swapping

To recall, one of my requirements was:

The simplest brute force thing to do here would be to refresh the whole agenda each time I hit M-n or M-p. Pressing g in the org agenda and checking keycast in the modeline tells me that the function to do this would be org-agenda-redo-all. This is kind of slow though - it doesn't take a long time to refresh my agenda by any means, but it's very noticeable when trying to move one entry interactively up the list, one key press at a time.

Thankfully, we already have functions that will do just the updating that is needed! I already mentioned them! They are org-agenda-drag-line-backward and forward! All I need to do is invoke the appropriate one of these functions at the end of my custom swapper functions, and the agenda view will update without me needing to refresh the whole thing.

For example, rostre/org-agenda-swap-ranks-down now looks like this. We can use org-agenda-drag-line-backward in the rostre/org-agenda-swap-ranks-up function in much the same way.

    (defun rostre/org-agenda-swap-ranks-down ()
      (interactive)
      (let ((curr-org-marker ; org-marker for heading at current line
             (or (org-get-at-bol 'org-marker) (org-agenda-error)))
            (target-org-marker ; org-marker for heading on next line
             (save-excursion
               (org-agenda-next-line)
               (or (org-get-at-bol 'org-marker) (org-agenda-error)))))
        (progn
          ; swap the rank of the heading at the current line with that of the line below
          (rostre/org-agenda-swap-ranks curr-org-marker target-org-marker)
          ; update the agenda - quicker to use drag line function to avoid full refresh
          (org-agenda-drag-line-forward 1))))

Setting rank for new entries

The final requirement I set myself is this:

The tricky bit here is that the :RANK: value ought to be different for the new entry to the entries that have come before to make sure it is distinct in the ordering. In order to do that, I need to iterate through all of the headings that could possibly be populating my org agenda and find the maximum rank that's already been set, and then make sure I add 1 to that before using it.

I could do this every time I add a new TODO item from a capture template, but that seems like a lot of effort. We can take a sneaky shortcut here and store that calculation in a variable on startup, then just add 1 to the variable each time we invoke the capture template.

org-map-entries makes the first calculation really easy - you give it a function you want to run and a symbol that specifies what entries to map over (the 'agenda setting iterates over every heading in org-agenda-files, incredibly useful), then it'll set point to the location of each entry in turn and call your function. The function in my case just dives in the properties of the entry and grabs the numerical value of :RANK:. Then I just run seq-max to find the maximum rank value from all the results. Emacs is great :)

    (setq rostre/curr-max-rank
      (seq-max (org-map-entries
                (lambda ()
                  (let ((heading-marker (org-get-at-bol 'org-marker)))
                    (string-to-number (or (org-entry-get heading-marker "RANK") "0"))))
                "+TODO=\"TODO\""
                'agenda)))

I still need a function that I can call from the capture template that will increment this variable and return the new value. That's not overly complex:

    (defun rostre/incr-and-emit-rank ()
      (progn
        (setq rostre/curr-max-rank (+ rostre/curr-max-rank 1))
        (number-to-string rostre/curr-max-rank)))

Finally, I can add something like this to my org-capture-templates, so when I run org-capture and add a new task, it'll automatically increment the rank and insert it into the new heading. In the template itself, you can use %(expr) to resolve a lisp expression, which I'm using to run my rostre/incr-and-emit-rank function.

    ("t" "Task" entry
     (file+headline "~/notes/journal.org" "journal")
     "\n* TODO [#%^{Priority: |A|B|C|D|E}] %?\n:RANK: %(rostre/incr-and-emit-rank)\n:END:\n\n" :empty-lines-before 1)

And that's it! I can now move entries up and down in my agenda interactively, and the resulting order will persist between agenda refreshes. Hoorah!

The value of :RANK: will keep increasing forever as I add new tasks, but that feels like a future Rob problem. If I get to the point where I'm overflowing the integer or something, I would be somewhat surprised.