I needed to remove the categories block from the footer (across all pages).

The way to do this is

menu:Modules > Positions

Find the hook (aka section) called 'displayFooter'

Find the module (one of the items within the hook) called 'Categories Block'

To the right there is a drop down menu. Click on 'Unhook', and you're done.