Docker persistent MAC address generation problem

When I was try start 150+ docker containers with docker-compose I found strange problem: "Could not generate persistent MAC address"
1
Could not generate persistent MAC address for veth: No such file or directory
After long googling and research I found lot of simillar issues..., same problem became in many virtualization systems like Docker, KVM... Than I found root cause of problem, too fast actions in linux network stack. So, lot of workarounds, like sleep on network subsystem action (Not help)
1
2
3
# cat /etc/udev/rules.d/01-net-setup-link.rules
SUBSYSTEM=="net", ACTION=="add|change", ENV{INTERFACE}=="br-*", PROGRAM="/bin/sleep 10"
SUBSYSTEM=="net", ACTION=="add|change", ENV{INTERFACE}=="docker[0-9]*", PROGRAM="/bin/sleep 10"
pre-generate MAC addresses in docker-compose.yml. (Not help and not work at all...)
1
2
3
4
5
6
7
services:
 test:
  image: busybox
  command: sleep 3600
  mac_address: 00:18:8B:0D:4F:0B
  networks:
  - hello
or rebuild linux modules or docker with patches... (I don't think is a good idea.)

From my point of view much better implement slow-start for docker compose service and roll back after official fix.

Before
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# cat /etc/systemd/system/docker-compose.service
[Unit]
Description=Docker compose service
Requires=docker.service network-online.target
After=docker.service network-online.target
 
[Service]
WorkingDirectory=/srv/docker-compose
 
ExecStartPre=/usr/local/bin/docker-compose down -v
ExecStartPre=/usr/local/bin/docker-compose rm -fv
 
ExecStart=/usr/local/bin/docker-compose up
 
ExecStop=/usr/local/bin/docker-compose down -v
 
[Install]
WantedBy=multi-user.target
After (Experimental solution)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# cat /etc/systemd/system/docker-compose.service
[Unit]
Description=Docker compose slow start service
Requires=docker.service network-online.target
After=docker.service network-online.target
 
[Service]
 
PIDFile=/var/run/docker-compose.pid
 
WorkingDirectory=/srv/docker-compose
 
ExecStartPre=/usr/local/bin/docker-compose down -v
ExecStartPre=/usr/local/bin/docker-compose rm -fv
 
ExecStart=/bin/bash /srv/docker-compose/docker-compose-daemon.sh
 
ExecStop=/usr/local/bin/docker-compose down -v
 
[Install]
WantedBy=multi-user.target
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# cat /srv/docker-compose/docker-compose-daemon.sh
#!/bin/bash
 
umask 022
 
echo $$ > /var/run/docker-compose.pid
 
bg() {
  cd /srv/docker-compose
  /usr/local/bin/docker-compose config --services | awk '{print "/usr/local/bin/docker-compose up --no-recreate -d "$1}' | sh | systemd-cat
}
 
bg &
 
while true; do
  echo "ping - $$ - `date`" | systemd-cat
  sleep 600;
done

Comments

Popular posts from this blog

FreeRadius and Google Workspace LDAP

pssh (parallel-ssh) problems on Debian 10 with Python 3.7