I needed to remove the categories block from the footer (across all pages).
The way to do this is
menu:Modules > Positions
Find the hook (aka section) called 'displayFooter'
Find the module (one of the items within the hook) called 'Categories Block'
To the right there is a drop down menu. Click on 'Unhook', and you're done.