Docker Fails To Start When Using A FirewallD Zone That Has A Mis-Configured forward-port Line

Submitted by david.reagan on Sun, 12/27/2020 - 14:04

If you use FirewallD to manage your firewall rules, and you include a forward-port tag in your zone file that is missing the to-port or to-addr parameters, Docker will refuse to start.

For example:

/etc/firewalld/zones/public.xml

  1. <?xml version="1.0" encoding="utf-8"?>
  2. <zone target="DROP">
  3.  <description>public zone.</description>
  4.     <service name="ssh" />
  5. <forward-port port="51820" protocol="udp" />
  6. </zone>

Will cause this output when you try to start Docker:

From journalctl -l -u docker

  1. Dec 27 21:43:38 test systemd[1]: Starting Docker Application Container Engine...
  2. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.939029977Z" level=info msg="Starting up"
  3. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.941435919Z" level=info msg="detected 127.0.0.53 nameserver, assuming systemd-resolved, so using resolv.conf: /run/systemd/resolve/resolv.conf"
  4. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.943581501Z" level=info msg="parsed scheme: \"unix\"" module=grpc
  5. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.943633881Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
  6. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.943665051Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
  7. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.943696801Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
  8. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.945747572Z" level=info msg="parsed scheme: \"unix\"" module=grpc
  9. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.945763802Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
  10. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.945781072Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
  11. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.945794012Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
  12. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.961351463Z" level=info msg="[graphdriver] using prior storage driver: overlay2"
  13. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.964738675Z" level=warning msg="Your kernel does not support swap memory limit"
  14. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.964752445Z" level=warning msg="Your kernel does not support CPU realtime scheduler"
  15. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.964758455Z" level=warning msg="Your kernel does not support cgroup blkio weight"
  16. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.964763475Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
  17. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.964896935Z" level=info msg="Loading containers: start."
  18. Dec 27 21:43:38 test dockerd[7539]: time="2020-12-27T21:43:38.971881740Z" level=info msg="Firewalld: docker zone already exists, returning"
  19. Dec 27 21:43:39 test dockerd[7539]: time="2020-12-27T21:43:39.111947329Z" level=warning msg="could not create bridge network for id acf7f168cd7aca8a45945338e5b60aa281b6c9f175c38adcd0bf854b25f4e01c bridge name docker0 while booting up from persistent state: Failed to program NAT chain: COMMAND_FAILED: '/usr/sbin/ip6tables-restore -w -n' failed: ip6tables-restore v1.8.4 (legacy): goto 'PRE_docker' is not a chain\n\nError occurred at line: 2\nTry <code>ip6tables-restore -h&#039; or &#039;ip6tables-restore --help&#039; for more information.\n&quot;&#10;Dec 27 21:43:39 test dockerd[7539]: time=&quot;2020-12-27T21:43:39.118342533Z&quot; level=info msg=&quot;Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address&quot;&#10;Dec 27 21:43:39 test dockerd[7539]: time=&quot;2020-12-27T21:43:39.144437152Z&quot; level=info msg=&quot;stopping event stream following graceful shutdown&quot; error=&quot;&lt;nil&gt;&quot; module=libcontainerd namespace=moby&#10;Dec 27 21:43:39 test dockerd[7539]: failed to start daemon: Error initializing network controller: Error creating default &quot;bridge&quot; network: Failed to program NAT chain: COMMAND_FAILED: &#039;/usr/sbin/ip6tables-restore -w -n&#039; failed: ip6tables-restore v1.8.4 (legacy): goto &#039;PRE_docker&#039; is not a chain&#10;Dec 27 21:43:39 test dockerd[7539]: Error occurred at line: 2&#10;Dec 27 21:43:39 test dockerd[7539]: Try</code>ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
  20. Dec 27 21:43:39 test systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
  21. Dec 27 21:43:39 test systemd[1]: docker.service: Failed with result 'exit-code'.
  22. Dec 27 21:43:39 test systemd[1]: Failed to start Docker Application Container Engine.
  23. Dec 27 21:43:41 test systemd[1]: docker.service: Scheduled restart job, restart counter is at 1.
  24. Dec 27 21:43:41 test systemd[1]: Stopped Docker Application Container Engine.

If you set your forward-port like:

/etc/firewalld/zones/public.xml

  1. <?xml version="1.0" encoding="utf-8"?>
  2. <zone target="DROP">
  3.  <description>public zone.</description>
  4.     <service name="ssh" />
  5. <forward-port port="51820" protocol="udp" to-port="51820" />
  6. </zone>

Docker will start normally.

I ran into this because FirewallD's site says that to-port and to-addr are optional.

I also would have figured this out more quickly if I had thought to look at the FirewallD logs sooner.

Anyway, here's hoping someone, someday, has some time saved by this post.