I ran into some issues with my unraid server recently. The docker image usage started skyrocketing at an alarming rate. I started digging into multiple ways it could be getting filled up and ran into a very odd file while doing so (sadly it didn’t contribute to the docker image usage, but good to have it removed anyway). I ran into a 20GB log file being generated by docker.
Continue reading “Docker Log File Usage”Docker Compose on Unraid
Recently I restarted my journey towards a self hosted photos system again. This time I was planning on trying it immich. Immich provides a docker compose file to startup their entire stack of 6+ containers, which I wanted to run on unraid that lacks compose in the default configuration. Queue journey to get docker compose capabilities on unraid.
Continue reading “Docker Compose on Unraid”Docker Hub Login on Unraid
I’ve had a few times now where I can’t grab a new container image to startup due to their rates limits on docker hub. This has been quite annoying, and there is a free plan for double the image pulls. Here is how you can login on unraid and setup the go file so it persists between reboots.
Continue reading “Docker Hub Login on Unraid”Cleaning up the docker registry
Working on my docker swarm recently I noticed that my storage was running low. After investigation, I found I had 17GB of usage by my docker registry. I have no need to have that many versions of my custom built containers on hand, so I went through the process of cleaning it up.
Continue reading “Cleaning up the docker registry”Installing Docker on a Raspberry Pi
On Raspbian
Install Docker
sudo curl -sSL https://get.docker.com | sh Add Permissions to run Docker Commands
sudo usermod -aG docker pi Test Docker Install
docker run hello-world Install Docker-Compose
sudo apt-get install libffi-dev libssl-dev
sudo apt-get install -y python python-pip
sudo pip install docker-compose Fixing the Docker Swarm tasks.db
The Issue: The docker swarm manager node becomes useless after the tasks.db file explodes in size. This can be seen by worker nodes not being able to connect to the swarm, or manager nodes not seeing the other manager.
The Fix: Stop the docker service (service docker stop), delete or move the tasks.db file, start the docker service (service docker start). This seems too simple to be true, but it isn’t, the tasks.db file can be safely removed and regenerated by the docker swarm manager.
Disabling the Aggressive Inspircd Health Check
One of the first application stacks I went to install and setup on my new Raspberry Pi docker cluster was Inspircd/qwebirc/anope. This stack was running originally on a raspberry pi 1b (256MB RAM version). I wanted to move this off the pi1 since it was out of date and would need a complete reinstall to be back to full patch status. However shortly after getting it running in my swarm, I ran into issues.
The IRC server would restart every few hours, sometimes it would restart every 10 minutes or so. I deemed that as unacceptable on my basic setup even for just using it in development of IRC bots.
Continue reading “Disabling the Aggressive Inspircd Health Check”Connecting Inspircd and Anope on Docker-Swarm
I encountered some issues when connecting Anope and Inspircd on the docker swarm. When running them on a single node using docker-compose, the services were able to connect just fine to Inspircd, however running in a docker swarm, there were issues in how Inspircd was filtering IPs in the default services XML block.
In order for Inspircd and Anope to talk I had to comment out some of the link block for the services. The allowmask and expected IP address didn’t quite work as intended inside the docker swarm cluster. Since the containers use a local network available to only them, I’m not worried about a rogue services server getting in. This IRC server is also primarily used for developing and testing the IRC bots I have written over the years, so its not a core part of my infrastructure. Below shows the link block that I modified from the default included in the inspircd docker container setup.
Continue reading “Connecting Inspircd and Anope on Docker-Swarm”Jenkins Docker Revisit
After my initial jenkins setup, I thought my system would be good to go for a long time, however I encountered a problem with permissions after my docker cluster reboot. After all my nodes were back up, and jenkins was running, it could no longer access the docker.sock that it used to handle building and pushing containers. I tried a few things, rebuilding the container, updating it, changing some groups, and found quite a few threads on the topic. Some people had chmod’d the docker.sock to 777 (BAD) or had given jenkins root (ALSO BAD). I ended up finding the solution in using a specific entrypoint script that would determine the group to add to the jenkins user, then launch jenkins using the jenkins user from root.
Most of my additions are from sudo-bmitch’s jenkins-docker repository on GitHub. These include the dockerfile changes and the entrypoint.sh script (as well as the healthcheck mentioned later on).
Continue reading “Jenkins Docker Revisit”Docker Cluster Reboot
Due to unforeseen events, I ended up having to shutdown all of my servers. Due to this, my docker VM cluster ended up getting its first chance to reboot the entire stack of all nodes. This ended up showing some problems in my configuration sadly, as well as a docker issue I had previously encountered in my raspberry pi cluster.
Continue reading “Docker Cluster Reboot”