Docker persistent MAC address generation problem
When I was try start 150+ docker containers with docker-compose I found strange problem: "Could not generate persistent MAC address"
After long googling and research I found lot of simillar issues..., same problem became in many virtualization systems like Docker, KVM... Than I found root cause of problem, too fast actions in linux network stack. So, lot of workarounds, like sleep on network subsystem action (Not help)
pre-generate MAC addresses in docker-compose.yml. (Not help and not work at all...)
or rebuild linux modules or docker with patches... (I don't think is a good idea.)
From my point of view much better implement slow-start for docker compose service and roll back after official fix.
Before
After (Experimental solution)
1 | Could not generate persistent MAC address for veth: No such file or directory |
1 2 3 | # cat /etc/udev/rules.d/01-net-setup-link.rules SUBSYSTEM== "net" , ACTION== "add|change" , ENV{INTERFACE}== "br-*" , PROGRAM= "/bin/sleep 10" SUBSYSTEM== "net" , ACTION== "add|change" , ENV{INTERFACE}== "docker[0-9]*" , PROGRAM= "/bin/sleep 10" |
1 2 3 4 5 6 7 | services: test : image: busybox command : sleep 3600 mac_address: 00:18:8B:0D:4F:0B networks: - hello |
From my point of view much better implement slow-start for docker compose service and roll back after official fix.
Before
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | # cat /etc/systemd/system/docker-compose.service [Unit] Description=Docker compose service Requires=docker.service network-online.target After=docker.service network-online.target [Service] WorkingDirectory= /srv/docker-compose ExecStartPre= /usr/local/bin/docker-compose down - v ExecStartPre= /usr/local/bin/docker-compose rm -fv ExecStart= /usr/local/bin/docker-compose up ExecStop= /usr/local/bin/docker-compose down - v [Install] WantedBy=multi-user.target |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # cat /etc/systemd/system/docker-compose.service [Unit] Description=Docker compose slow start service Requires=docker.service network-online.target After=docker.service network-online.target [Service] PIDFile= /var/run/docker-compose .pid WorkingDirectory= /srv/docker-compose ExecStartPre= /usr/local/bin/docker-compose down - v ExecStartPre= /usr/local/bin/docker-compose rm -fv ExecStart= /bin/bash /srv/docker-compose/docker-compose-daemon .sh ExecStop= /usr/local/bin/docker-compose down - v [Install] WantedBy=multi-user.target |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | # cat /srv/docker-compose/docker-compose-daemon.sh #!/bin/bash umask 022 echo $$ > /var/run/docker-compose .pid bg () { cd /srv/docker-compose /usr/local/bin/docker-compose config --services | awk '{print "/usr/local/bin/docker-compose up --no-recreate -d "$1}' | sh | systemd- cat } bg & while true ; do echo "ping - $$ - `date`" | systemd- cat sleep 600; done |
Comments
Post a Comment