Immutable tags

infrahead
Aug 15, 2024
3 min read

Updated: Nov 12, 2024

In GitOps, the CD on Kubernetes is ensured with the deployment operator that monitors the CR or the Git repository containing the service helm charts or manifests. When the Git repository changes, the manifests are reapplied after some period. The same operator also monitors the CR for the new tags, which can be configured to update the manifests in Git and trigger the application process. ArgoCD and FluxCD are examples of such operators.

A meticulous approach to managing release candidates is crucial in software development and deployment. Occasionally, teams face the dilemma of a release candidate being rejected due to a bug. A common but ill-advised practice emerges: deleting the original git tag and recreating it with updated code. Unfortunately, this method distorts historical data and context, undermining the integrity of the project’s version history.

Understanding the gravity of this issue, AWS Elastic Container Registry (ECR) offers a solution through immutable container repositories (CRs). Immutable tags in AWS ECR ensure that once a tag is assigned to an image, it cannot be reassigned to another image. Attempt to do so results in an error, preserving the sanctity of your project’s history.

The problem with mutable tags

In Git, tags are mutable. This means that tags can be deleted and recreated with different commits, allowing for the manipulation of historical data. Many Continuous Integration (CI) pipelines are designed to trigger builds and deployments based on tag push events. When these pipelines push images to a container registry, mutable tags can lead to confusion and inconsistency, especially if the same tag is reused for different images.

AWS ECR’s immutable tags come as a safeguard against such practices. By preventing the reassignment of tags, ECR makes it impossible to overwrite an existing image with a new one under the same tag. This immutability acts as an additional layer of security, thwarting attempts to alter repository data.

The “latest” tag dilemma

Using the “latest” tag in Docker and other container technologies poses challenges. While it may seem convenient, relying on the latest tag can complicate deployment. Tools like ArgoCD and FluxCD, used for Continuous Deployment (CD), may struggle to detect changes if tags are mutable and the “latest” tag is used. Without clear, immutable tags, these tools must rely on additional information, such as Git repository and container registry hashes, to detect updates.

Moreover, the “latest” tag lacks semantic information. It doesn’t convey what changes have been made, be they bug fixes, new features, or feature removals. This ambiguity is not ideal for maintaining precise and reliable software deployment practices.

Embracing tag immutability

Fortunately, Docker and other container technologies allow multiple tags to be assigned to an image. This flexibility enables teams to provide more meaningful tags that indicate specific versions while maintaining the same hash for identical Docker images. This practice enhances clarity and reduces dependency on mutable tags.

However, when relying on third-party images, it’s crucial not to trust tags blindly. Due to the potential for tag mutation, a security threat, verifying the hash of an image before using it is recommended. This step ensures that you’re using the exact version you intend without falling prey to the risks associated with mutable tags.

Conclusion

In conclusion, AWS ECR’s immutable tags offer a significant security and reliability advantage for DevOps practices. By enforcing immutability, teams can maintain a clear and accurate history of their deployments, avoid the pitfalls of mutable tags, and enhance the overall security of their software delivery process. Clarity and consistency in software deployment are critical to a successful and secure DevOps pipeline. DevOps agency infraheads recommend using admission controllers in Kubernetes to restrict the usage of the latest tags.