Data Lineage to Transform an Information System

 

Le data lineage to  Transform  an Information System:   Automated Simplifications  & Migrations

At a time of accelerating digital transformation, massive movements towards the Cloud, and the surge of "as a Service", it seems useful to be able to understand the way in which Information Systems work to give them a chance to transform them in depth.    When they are still largely "on premise", their retro documentation is tedious, due to lack of time or skills still in place.    These legacy systems are often impossible to migrate to the Cloud given the high risks of regression.  And when these Systems have been migrated to the Cloud, they tend to become exponentially more complex (and more and more expensive).     We propose to  break down its complexity and share its understanding  by dynamically delivering a map based on data lineage and the uses of information.   Above all, we offer answers to carry out  massive simplifications and automated migrations  to the Cloud , making it possible to get rid of "legacy" technologies that are often stuck in companies :  Oracle, Teradata, Cobol for databases for example, or SAP BO and SAS for data visualization technologies. _____

1  methodology

Automated and exhaustive reverse engineering  of the Information System with {openAudit} .    {openAudit} is the association of technical data lineage and the analysis of information uses.    Technical data lineage to know the origin and future of each piece of information, from its sources to its uses. This data lineage is based on granular and continuous analysis of flows (procedural code, ETL/ELT, ESB, transformations in the data visualization layer, etc.), Analysis of logs to know the real uses of each data: who accesses it, when, how.     This mapping reflects the exact reality of the Information System at time “t”.    The analyzes are carried out in continuous time, at the delta, and without mobilizing teams. Even when the systems are heterogeneous, hybrid.     Some precisions :    {openAudit} analyzes views; views of views, etc. ;  {openAudit} manages dynamic procedures by processing the run in parallel; {openAudit} fixes other generators of breaks in the lineage: FTP transfers, cursors, subqueries, etc. which are managed by various mechanisms (structural recognition, others);  {openAudit} processes SQL (or its derivatives) encapsulated in ELT/ETL jobs;  {openAudit} processes the data visualization layer to have a true end-to-end view, and to highlight all the management rules  often contained within the data visualization layer itself.

...for 3  major answers:   ________ Mapping Information Systems  to share understanding  Simplify systems by identifying my dead matter Automatically migrate Systems  (Cloud migrations, etc.)

Map  Data lineage allows you to know the path of each piece of information in the System in an ultra-granular manner.  This data lineage is produced end to end, from the operational system to the dashboard cell, with certain interest:  To share the understanding of a system to everyone.  To act in the event of a problem on a supply chain. To identify a problematic expression in a dashboard.  To identify the dissemination of information under the GDPR.     Simplify  The intersection of data lineage and the analysis of information uses makes it possible to detect “dead branches” of systems. On average, 50% can be eliminated via mass decommissioning: tables, views, procedural code, ELT/ETL jobs, dashboards, etc.  To reduce the complexity of a system and therefore maintenance.  To reduce costs in the Cloud by identifying unnecessary flows and the processing/storage associated with them.     Migrate  Cloud migrations are never simple. They are sometimes even impossible: prohibitive costs, too high a risk of regression. We end up stacking technologies and living with outdated legacy technologies.  {op enAudit} allows you to automatically migrate data storage technologies: Oracle, Teradata or Cobol... to GCP, Azure, or PostGre. {openAudit} also allows you to migrate data visualization technologies: SAP BO to Power BI or Looker for example.

Some features in video  :  Conclusion  The complexity of Information Systems is not inevitable, and should no longer be an obstacle.  We are convinced that by bringing out all the technical underlying systems of the systems in an intelligible way, these systems can very naturally harmonize over time.  This detailed understanding also makes it possible to automate what constitutes among the biggest challenges of the moment:  The massive simplification of systems to reduce maintenance and costs, in the Cloud in particular;  And to automate technical migrations, which for some become unattainable horizons, due to their costs and the fear of significant regressions. contact@ellipsys-lab.com