EX.CO is a video technology platform that enables publishers to monetize video content on websites.
EX.CO is a Disney-backed publisher-first video technology platform. It is used by publishers to monetize video content on websites and to add interactive and media elements intended for a particular user base. Empowering publishers to own their video strategy for an optimal page experience and maximum revenue growth.
Customer Challenge
Ex.Co Infrastructure did not have a unify IaC for all the workloads, and some of them did not have any IaaC. Infrastructural changes were made manually.
- Some minor IaaC efforts were made, authored by third-party companies
- IaaC parts were unconnected, subproject-based
- No naming conventions or tags policy existed
- The workload scales made manually
- The workload monitored with DataDog, with few alerts
- No drift detection was possible
Partner Solution
The IaC project allows the customer to deploy improved workload while keeping the best practices of naming convention, high availability and scalability. Drift detectors for changes became available, and reuse of modules helps the company to have a standardized usage for AWS resources.
The IaaC method that was chosen is Terraform, elaborating the ‘modules’ concepts in order to keep the naming convention, tagging policies, cost optimization strategy and the auto-scalable capabilities of the AWS services.
Most of the workload run over EKS and spot fleet allowed the project to keep the cost optimization in high standards.
The Deployment method of each environment is Terragrunt which enable the customer full control over the variables.
IaC
Reusable Terraform modules were created alongside a generic configuration approach. For example, a new project can start by copying the content of the default configuration configuration/us-east-1/prd/ops into a new location and modification of a single values.yaml file with ~100 lines in it.
Fine-grained access
IAM OIDC provider is enforced, so in case of required access to any AWS service from EKS, there is a role created by Terraform/Terragrunt and a Kubernetes ServiceAccount linked to it.
Autoscaling
By default, EKS is created in a scale to fit required software, including Karpenter, which is used to scale dynamically according to workload. Fig. 4
High Availability
By default, all the configurations utilize at least two Availability Zones in a region. Fig. 3
Disaster recovery
It’s out of the code scope here, but most services running inside EKS use backups and automatic restoration of data when it’s possible.
Continuous deployment
Handled by FluxCD integrated into the EKS cluster. Fig. 4
GitOps
Versioning, testing, and code reviews are enforced on the repository level, both IaaC and application releases.
IPAM
It’s out of the code scope, but Netbox was configured to track IP block usage and physical servers out of AWS.
Services
- EKS – Utilizing the official modules of EKS
- IAM roles – practicing the least privileged concepts
- Parameter store – keep secrets protected in AWS
- KMS – encrypt all data
- Autoscaling – by using the node group auto scaler
- Route53
- VPC peering
- EFS – storage that scale and accessible from the clusters
Solution Diagram
Repository layout
Configuration dependencies
AWS layout
Kubernetes layout
Results and Benefits
The implementation of Infrastructure as Code (IaC) at EX.CO has significantly transformed their AWS workloads, bringing about standardization, efficiency, and adherence to best practices. The transition from manual infrastructural changes to a comprehensive IaC approach, utilizing tools such as Terragrunt and Terraform, has not only streamlined the deployment process but also enhanced the overall reliability and scalability of the environment. Ex.Co is handling 15 EKS clusters and changing them manually consumes 1500% more time. The incorporation of fine-grained IAM role-based access, autoscaling mechanisms, and high availability across multiple Availability Zones reflects a commitment to robust security, efficient resource utilization, and resilient operations. Additionally, the integration of GitOps, continuous deployment with FluxCD, and versioning practices ensures a disciplined approach to development and release management. The project’s success lies not only in the deployment of improved workloads but also in the establishment of a standardized and modular foundation for future endeavors, showcasing EX.CO’s dedication to embracing cutting-edge technologies and best-in-class practices for optimal cloud infrastructure management.
The money saved by implementing EKS autoscaler and spot fleet is $15,000 per month.
About the Partner
GlobalDots helps IT, DevOps & security leaders navigate in today’s quickly-evolving innovation & threat landscape. We explore web & cloud innovation and implement cutting-edge solutions to enhance your performance and security posture, for efficient, growth-ready infrastructures.
