Treat the "IT debt" in 2 steps

Treat the "IT debt" in 2 stages

“ IT debt” –  why ?

"Debt" through approximations Speed ​​often borders on haste. Developers are being asked to work faster and faster to construct new solutions. The accumulation of approximations creates technical problems that are difficult to solve in the medium to long term. This is technical "debt".

"Debt" through the complexity of systems  The number of publishers and technical solutions is growing at a phenomenal rate. Cloud computing and SaaS have been tremendous factors in modernization, but also in increasing complexity. We didn't have time to "close" one technological door before five others opened. We have to chase after increasingly rare skills to manage the oldest technologies. And we're not even talking about decommissioning them  .  So, we're banking on upward technological compatibility to continue thickening the "layer cake" without too much risk of seeing it collapse. Once again, we're creating "technical debt."

It is estimated that around 40% of developers' time is spent maintaining systems! This figure can go up to 60%...   Software Engineering Stack Exchange - Radixweb​ “Maintenance” vs “Development”,  general trend

But who is willing to miss the technological train to quietly devote themselves to purging systems of all obsolescence and approximations, i.e. of all the "debt"? Not many.   However, there are solutions.

1- Remediation? A system mastered by all, because it is continuously mapped

We believe it is essential to make everyone aware of all the processes at work within a data system: where the data comes from, where it goes, who uses it, how it is processed, with which tools, how it is stored, and what management rules are applied to it, etc. And continuously.

Data lineage, based on the analysis of data processing, storage, scheduling and consumption (reporting and queries), makes it possible to draw up a comprehensive portrait of a system. Understanding the system can be shared by everyone, even when the technologies are old and the chains are complex.

Two search engines are available within {openAudit} to investigate complete flows, in the power supply layers and in data visualization technologies: one for legacy, "on-premise" technologies, the other for Cloud technologies.  Breaks in flows and all abstraction layers are processed, for a holistic view of the system.  Nothing is hidden from the teams' knowledge anymore. There is no longer a "black box," and technical debt becomes less of a problem.   Film: 3'

2- Remediation? A pivot model in SQL to standardize and modernize a system

Technology stacking is not inevitable. SQL, which had been marginalized with the advent of ETL/ELT, is making a strong comeback in the Cloud with numerous variations.  Why? Robustness, scalability, control by the greatest number, etc.

With this in mind, we have developed an agnostic SQL generation engine based on ETL/ELT technologies and dataviz technologies. This engine will enable an "As Is" migration of all intelligence from the source system to flat SQL, without subqueries, in the target corporate system.  The "disorder" will be largely reduced, the system will become mechanically infinitely more homogeneous. These projects are offered as a package.

This base will allow for factorization, simplification, and adaptations. In short, it becomes possible to break the ever-increasing number of barriers to systems composed of proprietary solutions.

Conclusion  To solve the technological challenges posed by the accumulation of technical "debt" and the increasing complexity of systems, it is essential to map and understand all the processes applied to the data. Furthermore, SQL standardization offers a robust and scalable solution for simplifying and standardizing these systems. By transforming complex processing chains into SQL, systems can be mastered and maintained more easily, thus reducing obsolescence and technical approximations.