Click here to Skip to main content
15,885,084 members
Articles / Containers / Docker
Technical Blog

When Containers Become Trashcans

Rate me:
Please Sign up or sign in to vote.
5.00/5 (1 vote)
4 Mar 2020CPOL2 min read 8.7K   1   1
How containers can quickly become trashcans if unused ones are not deleted
Containers are really awesome, powerful, and useful to the modern developer, but if you aren't mindful, those unused ones can quickly turn from containers to trashcans that can make your development environment unwieldy

When Containers Become Trashcans

Containers are so awesome.

Prior to containers, if you wanted to experiment with some new technology, you had to go through the ringer to configure and install all of the appropriate dependencies, set up the proper infrastructure, and clutter your machine with tons of trash that you might not ever care to use in the future. Thankfully, Docker and other containerized technologies came along and empowered developers to just throw a one-liner into the command line and like magic, the entire world appeared in a magical little box.

Several days ago, I had an idea in my head so I scrounged around through Docker Hub looking for the perfect container that had everything that I wanted in it. I'd pull down one, tinker a bit, and throw it away. Pull another down, try something else, and then eventually throw it away. Pull yet another one down and ... crap!

docker: write /var/lib/docker/tmp/GetImageBlob785934359: no space left on device.  

No space left on my device? That's odd, maybe it's the hours upon hours of videos that I've downloaded for my kids to watch on road trips, or all those computer games that I've installed but never can find the time to play (some day)? Let me check out the hard-drive and see how that looks:

When Containers Become Trashcans

Okay, so either this image that I'm pulling down is really big, or something else is going on. After a bit of mulling it over, I realized that like all good humans that don't want the machines taking over, delegated a specific amount of space and resources that Docker could take advantage and sure enough, I was right at the threshold.

It was time to take out the trash, however since docker dump trash isn't a legitimate command (at least out of the box), I just had to do a quick prune:

docker system prune  

The prune command in a nutshell does the following:

  • Removes all non-running / stopped containers
  • Removes all volumes that aren't being used (by at least one active container)
  • Removes all networks that aren't being used (by at least one active container)
  • Removes all dangling images within the system.

Since I mentioned above that I really enjoy playing with containers, I figured that we'd just do it live, so here's what that looked like on my local machine:

When Containers Become Trashcans

With three little words, I managed to free up nearly 126GB of unused space across 22 containers, 5 networks, and 45 images). That's quite impressive, but more importantly, I was then able to immediately install the image that I was trying to in the first place.

So the moral of the story is this - containers are really awesome, powerful, and useful to the modern developer, but if you aren't mindful, those unused ones can quickly turn from containers to trashcans that can make your development environment a stinky place to work in.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer (Senior)
United States United States
An experienced Software Developer and Graphic Designer with an extensive knowledge of object-oriented programming, software architecture, design methodologies and database design principles. Specializing in Microsoft Technologies and focused on leveraging a strong technical background and a creative skill-set to create meaningful and successful applications.

Well versed in all aspects of the software development life-cycle and passionate about embracing emerging development technologies and standards, building intuitive interfaces and providing clean, maintainable solutions for even the most complex of problems.

Comments and Discussions

 
Questiondocker system prune should be run nightly Pin
kalberts5-Mar-20 1:41
kalberts5-Mar-20 1:41 
Why not set up a nightly job to run docker system prune?

For those who develop their own images: Every image rebuild leaves the old one behind as garbare (unless you have tagged it as something else than 'latest'). Often, during the development phase, only a few of the modules are actually updated. Yes I see lots of people rebuilding everything from the bottom base layer (say, ubuntu). If not a single bit has changed in the sources, the lower layers of the new image might be exactly identical to the old image (same SHA), but for anything downloaded from the network (apt, pypi, ...) updates are very frequent. So unless you explicitly specify the exact version for every single package (the count easily runs up to several hundreds!), anything from that layer and upwards the layer pile is discarded.

If you build a local image with all the reasonably stable modules - e.g. compilers/interpreters, libraries, database software, network software - but omitting the software you are actively modifying, you may use this image as the base for all you rebuilds, just adding a few top layers with the dynamically changing software. Those top layers may still end up as garbage when a new version is built, but they are usually thin (small). The rebuild will be significantly faster, maybe by a factor of 10. The dockerfile will (usually) be significantly smaller and simpler, as only the top layers are specified.

You won't have the daily...weekly updates of the system software - which may be an advantage: Sometimes, new versions of apt or python packages may introduce tiny changes in behaviour affecting your application. As long as you use the "composite" base layer you created for yourself, you are unaffected by version updates made in the background. Every now and then you may want to rebuild your composite base with all the updated package versions, but as a controlled operation, and with the option to go back to your previous base if the new one gives problems.

Note that even though your composite base contains e.g. a number of python packages, and you need a newer version of one of them (or an additional one), in your top layers you can use pip install to modify the configuration; you don't need to make a new composite base layer just to add/update one or a few packages.

You still should prune the system every now and then, but unless you are creating dozens of new versions of your image every day, you can probably reduce the frequency to, say, weekly pruning.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.