After migrating their data applications to the Azure cloud environment, the client faced an unexpected surge in cloud costs due to inaccurate service provisioning and inefficiencies in their data engineering processes.
Ingrity performed a thorough analysis of the client’s cloud setup and identified that the increased cloud expenses were primarily caused by inaccurate service provisioning and inefficiencies in their data engineering processes. We delved deeper into the infrastructure to pinpoint specific areas for cost savings and advised the client to take prompt action.
For example, we discovered that certain SQL queries used for ETL operations had a structure that increased cluster usage, resulting in higher cloud expenditure. Additionally, one of the client’s data providers was streaming motor insurance data using a message queue service on the AWS platform, which was received via an Azure Event Hub service. The cost of the Azure service was driven by the number of folder scans required to read the messages. By modifying the folder structure, optimizing message groupings, and adjusting the scan settings to a higher periodicity, we achieved a significant cost reduction of 60%.
As a result of our cloud cost optimization efforts, the client managed to reduce their monthly cloud expenses by over 70%. This substantial cost reduction allowed them to allocate their cloud resources more efficiently and effectively, resulting in improved financial management and overall operational efficiency.
