This post might have multiple titles. “How to speed up rails project CI build times”, “How to reduce docker images size”, “How to save planet by reducing carbon print with smaller docker containers” and so on.
Optimizing Continous Delivery process is somewhat of my passion. I think it’s the strong influence one book by Jez Humble’s put on me.
When I started helping VAS (first key OSO’s client) with going cloud I wrote TBH very inefficient process of release. The complete build of docker images took place inside 3 phase Dockerfile which at best took 40 minutes to complete. Downloading gems, downloading nodejs packages, precompiling assets ended up with almost 2 GB docker image which was shipped to kubernetes cluster with the help of fluxcd. The second part of the inefficiency was our git flow process which we had. Standard develop, feature, release, and main git branches each triggered docker image re-build individually. After feature branch passed code review and got merged to develop the build process started to prepare image to deploy to dev environment. When preparing a batch of commit to ship to prod we first created release- branch which also triggered another build process (this time of release- branch) to deploy to uat env. And finally after merge to main branch prod build was triggered. 3x 40 minutes each, building docker image from scratch, downloading everything from web without any middle tier package registry. On purpose I won’t write how much time unit tests took which were running in separate github workflow and which also had dedicated dockerfile with postgres and chrome. We worked with this setup quite a long time. There were some attempts to cache and re-use final docker images which result in big ECR bills due to high data transfers.
The first thought that helps with moving towards better process is “promote currently built images”. We annotate each docker image with tags like dev-x.x.x which is semi semver style with environment prefix. This is one of the default flux strategies we took from the shelf. So basically annotating docker image with uat- or prod- prefixes is a way to tell flux listener to deploy accordingly. This single step helped to save 2x 40 minutes of build times of uat- and prod- images.
Another step is to get rid of middle layer docker images for assets precompilation and github cache utilization. Our final docker images are alpine linux distro and we are building app on ubuntu-latest. Both are x86 architecture both are linuxes. However bundler complains about some native packages for alpine and requires additional bundle install invocation but that action moved to the very end of the process helps a lot. The biggest gains are due to caches of gems, node packages and precompiled assets. Github actions caches name is created basing on hashFiles function. So until no changes are applied to any of TS, JS, TSX or yarn.lock files the assets are taken from cache. Additional benefit is there are no node_modules on final docker image. Only public/ directory. Modified dockerfile contains 2 phases and first one runs “bundle clean” which will remove development and test gems used only for build,test time.
The complete build process went down to 5-6minutes and I see another optimization which would take previous docker image and only copy /app directory to it if no frontend changes. Final docker image is now 446 MB size. And we track that number in ever build summary.
Tests execution was parallelized and whole suite was split into 15 groups with matrix github workflow directive and –only-group flag for parallel_rspec. Rspec examples in groups are not well balanced in terms of execution times but right now the whole tests suit takes as long as the longest group which is usually up to 10 minutes.
This app contains a few popular gems like device, sidekiq, draper, aasm, rolify, activeadmin, react-rails, turbo-rails. I find this very satisfying when seeing cost savings, shorter image transfer times and overall quicker build times.