Just Hackin'

Ever have services suddenly break and realize after running df -h that you're out of disk space?

Fixes to common problems:

# this one can clear **gigabytes** of space
sudo purge-old-kernels -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" --allow-downgrades

# this will remove packages not in use
sudo apt-get autoremove -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" --allow-downgrades

# this will remove snapd package cache
sudo rm -rf /var/lib/snapd/cache/*

Packages

In general, a good recommendation is to set up a script that runs some commands to cleanup packages every week or so:

export DEBIAN_FRONTEND=noninteractive
sudo apt update

# This long command will automatically choose to keep existing files and install all upgrades on your machine
# NOTE: This is the most risky command as it'll keep you updated and on the bleeding edge if you run it on schedule - make sure you know what you're doing (and the risks) if you use this in production environments
sudo apt-get upgrade -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" --allow-downgrades

purge-old-kernels -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" --allow-downgrades

# This will remove dependencies not installed/in use
sudo apt-get autoremove -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" --allow-downgrades

# Delete unnecessary cache for snapd packages
sudo rm -rf /var/lib/snapd/cache/*

Services

One place that lots of logs can accumulate – journald logs. You can clean them up pretty easily by using the CLI arguments to remove old logs:

sudo journalctl --vacuum-size=5M

Don't this manually in the future, instead open up the journald config and set a limit on how much space can be used.

Normally, this line in /etc/systemd/journald.conf is commented out, but just remove that comment and set a reasonable log filesize limit:

SystemMaxUse=5M

Then do sudo systemctl restart systemd-journald.service, and now it should stay under 5 megabytes in size.

Misc

General advice: look for logs, file uploads, things that endlessly grow and might never be cleaned up or removed.

One command to show the 30 largest files and folders just in the current directory:

du -cks * | sort -rn | head -30

A command to show the 30 largest individual files under the current directory:

find . -printf '%s %p\n'| sort -nr | head -30

(Courtesy of TutorialsPoint)

 

Docker & Docker-Compose

If you use docker, make sure to create/edit a file called /etc/docker/daemon.json and make sure it has this attribute in it:

{
    "log-driver": "none"
}

If you use docker-compose, set your configs to turn off logging by setting the driver to none for all docker-compose files' services:

services:
  website:
    image: nginx
    logging:            # <--- add this
      driver: none      # <---

Alternatively, set a limit:

services:
  website:
    image: nginx
    logging:
      driver: "json-file"
      options:
        max-size: "512m"

UFW

If you use UFW, you'll see it generate a lot of useless logs. Run ufw logging off to completely turn off logging.