Docker: the false promise of "run anywhere"

We all know and love Docker, right? The ability to build and deploy your software in a completely platform-agnostic manner is a boon for developers, and modern deployments that aren't containerized have become very rare indeed.

Well, I'm here today to present you with conclusive proof that Docker builds are not platform-agnostic. That's right: the same version of Docker building the same image has different behaviour depending on the system it's running on! Fortunately, in the present case, the bug only affects users of Chrome for Android. Unfortunately, Chrome for Android is by far the most commonly used browser in the world.

Context

My latest project is a website built with Flask on the back end and React on the front end. The basic idea is to display a world map on which users can pin travel blog posts, or find blog posts about places they're interested in visiting. The site is online at blogtheworld.co, if you want to check it out, while the source code is on my GitHub.

The site is deployed with separate containers for the back and front ends, orchestrated with docker-compose. I have separate configs and compose files for dev and production builds, so I can run dev or prod on any machine.

Now, those of you who visited my site using Chrome on an Android device may have noticed something strange: when you tap on one of the pins on the map, nothing happens. Surely it should open a link, or show some details of the blog post, right? Indeed, when you visit the site using any other browser and hover over (on desktop) or tap (on mobile) one of the pins, you should see a 'popover' looking something like this:

This in itself isn't particularly strange: web development is hard when you have to take into account the myriad devices and applications which can be used to access your site, and it's common for some features not to work on some browsers. But now I want to show you something really weird. Follow along with me:

1. Click on the three dots in the top-right corner of Chrome to open the menu.

2. Check the option to open the desktop version of the browser, and wait for it to load. The page should look something like this:

   

3. Now switch back to the mobile version, and tap on one of the pins. Believe it or not, the popover functionality now works! That is to say, switching from mobile to desktop and back to mobile has 'tricked' Chrome into enabling functionality that wasn't enabled before!

   

This is extremely weird and surprising behaviour, but so far it has nothing to do with Docker. The most we can say is that we've identified a potential bug in Chrome for Android. Now I want to show you where things get to off-the-charts levels of weird.

Docker Jekyll and Mr. Hyde

As I mentioned before, I can run dev or production builds on any machine thanks to my various config and Docker files. Most of the time I'm developing on my local machine and deploying on my remote server (a DigitalOcean droplet, or VPS). So what happens when I access the development build running on localhost with my phone? Thanks to USB debugging and Chrome's port-forwarding feature (visit chrome://inspect to set it up), I can easily do this.

Well, it turns out that the development version of the site works on Chrome on my Android phone, no problem!

What gives? Using Chrome's dev tools, I can set a breakpoint on pointerDown events and see that in the working version, on localhost, there are #document.pointerDown, BODY.pointerDown and DIV.pointerDown events fired when clicking on a pin. On the non-working version running on my DigitalOcean droplet, we still have #document and DIV events, but the BODY events are never fired.

The only differences between my dev and prod builds concern the nginx config and the use of SSL certificates, so there's no reason for any difference in the behaviour of purely front-end functionality like this. Nonetheless, let's be thorough and try running the dev build on the remote server. By doing this, we get to the core of the problem: I'm using identical Docker configs, and the site works on my machine but not on the remote host!

We can also try things the other way round, and run the production build on my local machine. The browser will complain about the SSL certificate being registered to the wrong domain, but we can bypass that and see that...it works!

Now, the versions of Docker running on my local machine and the remote host aren't completely identical:

• On local I have Docker version 20.10.22, build 3a2c30b and Docker Compose version v2.15.1 (system is Ubuntu 20.04.4 LTS running in WSL 2 on Windows 11).

• On remote I have Docker version 20.10.16, build 20.10.16-0ubuntu1 and docker-compose version 1.29.2. (system is Ubuntu 22.10).

I would say though that if there are significant differences in behaviour between different minor versions of Docker, it loses most of its value as a platform-agnostic deployment tool. Sure, I could work around the apt packaging system and get identical versions of Docker on each machine, but this really shouldn't be necessary. The packages are already equivalent versions for the respective operating systems:

<local> $ apt-cache policy docker
docker:
  Installed: 1.5-2
  Candidate: 1.5-2
  Version table:
 *** 1.5-2 500
        500 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages
        100 /var/lib/dpkg/status

<remote> $ apt-cache policy docker
docker:
  Installed: 1.5-2
  Candidate: 1.5-2
  Version table:
 *** 1.5-2 500
        500 http://mirrors.digitalocean.com/ubuntu kinetic/universe amd64 Packages
        100 /var/lib/dpkg/status

Just to make absolutely sure this insanity was for real, I tried one more thing: the nuclear solution of building my Docker images locally and copying them to my remote server (docker save, scp and docker load) before running them there. Sure enough, these images running in containers on my local machine worked fine while the exact same images running in containers on the remote host exhibited the UI problem.

Conclusion

So, I have conclusive proof that applications built with the same Docker version but running on different operating systems and different hardware behave differently. I've already posted about this on StackOverflow, but for the time being there's not much I can do.

If anyone reading this has a tale of similar woes with Docker deployments behaving differently on different systems, I'd love to hear from you. Please comment below and let me know if you ever found a solution!