FinOps: Reduce Cloud Costs by Removing "Dead Branches"!

 

FinOps : Reduce Cloud Costs by Removing "Dead Branches"!

Data lineage and analysis of the uses of information to calculate the cost of "living branches" and "dead branches". For simple and cheaper cloud systems.

Intro  Cloud costs, a real subject!  ____________________________________

OWe are not going to dwell on the adoption of the Cloud, but we will note that 50% of companies have 100% of their infrastructure in the Cloud (Rackspace Technology 2022, in association with Google Cloud), and the others are thinking about it seriously: end of infrastructure to manage, infinite scalability, often lower costs, etc. The strengths of the Cloud are also its limits: the systems are ultra-scalable, and the teams have a field day for the greatest benefit of the business. It is also unfortunately the realm of informational inflation that is not always justified, with strong impacts: 7 out of 10 companies do not know exactly what they are spending their cloud budget on (Fortinet 2021). 131 IT professionals say undue cloud spend could account for up to 47% of a cloud budget (Stormforge 2022). ... Especially since invoicing in the cloud is generally very abstract. And few CIOs really see it clearly.   The Google Cloud Platform (GCP) model, which we know well, essentially uses a pay-as-you-go model, which means that customers only pay for the resources they actually use ( data processing). This prompted us to develop an original, efficient, turnkey FinOps model.

How to reduce your Cloud bill? ______________________________

FinOps (monitoring and optimization of costs in Cloud computing) mainly consists of using the tools made available by Cloud providers to try to optimize your bill. Overall, we are more on a "task force" model than on a more sustainable "software" model. We believe that the most obvious, most effective lever for reducing billing consists first and foremost in evicting the countless "dead branches" from the system which are processed unnecessarily.   The positive effects induced are potentially numerous: system intelligibility, system security, green IT, etc.

Methodology (in GCP) ______________________________

Define the cost of the “living branches” of the system ____________________      {openAudit}, our software, will make it possible to identify the tables/files consumed by people (queries) or by machines, through a continuous analysis of the logs: these are the "uses" of the information. A strong depth of history will allow you to have a very objective view of things.

Starting from the data consumed (tables / files), therefore "uses", the upstream flows are deconstructed to the sources thanks to the data lineage of {openAudit}, which operates simultaneously on different technologies: ETL/ELT, scripts. These are the "living branches" of the system.   Then {openAudit} identifies the processing costs of each transformation available in the Stackdriver, and adds them for the entire flow.   By multiplying this mechanism with all the flows, we obtain the full cost of the useful ("consumed") part of the system, ie the "living branches".

Define the cost of the "dead branches" of the system _________________________   The "dead branches" of the system are simply all those... which are not "living"! They are identified by deduction.    {openAudit} establishes their costs in the same way: addition of the processing costs of each transformation identified thanks to the data lineage, and via an analysis of certain Stackdriver logs. Costs are summed for complete streams, then for the entire system.

All that remains is to decommission these branches to significantly lower billing! Then do it continuously.

Conclusion Automate FinOps !  ________________________________

It is possible to do FinOps "paper / pencil", but the automation, and the continuous implementation of the process of detection of "dead branches", then the calculation of their costs has the merit of allowing action with objectivity and efficiency. The system can thus develop harmoniously in a FinOps, GreenOps logic, while minimizing security risks. #finops #greenops #datalineage