Treat the "IT debt" in 2 steps

  

Treat  the "IT debt" in 2 steps

“  IT debt”  –  why ?

"Debt" through approximations Speed ​​often borders on haste. Developers are asked to move faster and faster to build new answers. The stacking of approximations creates technical problems that are difficult to solve in the medium/long term. This is technical "debt".

"Debt" through the complexity of systems  The number of publishers and the number of technical solutions are multiplying at a phenomenal speed. Cloudification and SaaS have been formidable factors of modernization, but also of complexity. We didn't have time to "close" one technological door before 5 others opened. We have to run after increasingly rare skills to manage the oldest technologies. And we're not even talking about decommissioning them  .  So, we are banking on upward technological compatibility to continue to thicken the "layer cake" without too much risk of seeing it collapse. Once again, we are creating "technical debt".

It is estimated that around 40% of developers' time is spent maintaining systems! This figure can go up to 60%...   Software Engineering Stack Exchange - Radixweb​ “Maintenance” vs “Development”,  general trend

But who is ready to miss the technological trains to quietly devote themselves to purging the systems of all obsolescence and approximations, i.e. of all the "debt"? Not many.   However, there are solutions.

1- Remediation? A system mastered by all, because it  is continuously mapped

We believe that it is essential to make everyone aware of all the processes at work within a data system:  where the data comes from, where it goes, who uses it, how it is processed, with which tools, how it is stored, and what management rules are applied to it, etc. And continuously.

Data lineage, based on the analysis of  data processing  , storage, scheduling and consumption (reporting and queries), makes it possible to draw up an exhaustive portrait of a system. Understanding the system can be shared by everyone, even when the technologies are old and the chains are complex.

2 search engines are made available to investigate complete flows, in the power supply layers and in the data visualization technologies: one for legacy technologies, "on premise", the other for Cloud technologies.  Breaks in flows and all abstraction layers are processed, for a holistic vision of the system.  Nothing is hidden from the teams' knowledge anymore. There is no more "black box", technical debt becomes less of a problem.   Movie: 3'

2- Remediation? A pivot model in  SQL  to standardize and modernize a system

Technology stacking is not inevitable. SQL, which had been marginalized with the advent of ETL/ELT, is making a strong comeback in the Cloud with numerous variations.  Why? Robustness, scalability, control by the greatest number, etc.

In this logic, we have developed an agnostic SQL generation engine based on ETL/ELT technologies and dataviz technologies. This engine will enable an "As Is" migration of all intelligence from the source system to flat SQL, without subqueries, in the target corporate system.  The "disorder" will be greatly reduced, the system will become mechanically infinitely more homogeneous.

This base will allow factorization, simplification, adaptations. In short, it becomes possible to break the ever-increasing number of locks of systems composed of proprietary solutions.

Conclusion  To solve the technological challenges posed by the accumulation of technical "debt" and the increasing complexity of systems, it is essential to map and understand the entire set of processes applied to data. Furthermore, standardization in SQL offers a robust and scalable solution to simplify and homogenize these systems. By transforming  complex processing chains  into SQL, it is possible to control the systems and facilitate their maintenance, thus reducing their obsolescence and technical approximations.