Interoperability 

of dataviz tools with SQL

 

In today's world of data management, information is often spread across different data visualization platforms depending on business needs, technical challenges, budgetary constraints, etc.

 

To optimize the management and use of this data, an SQL-based interoperability approach can help restore dataviz tools to what they were originally: "visualization" tools - not data prep tools, which they have tended to become, creating a damaging increase in opacity and inertia. 

 

It can be a relevant link in the context of a migration, for the sustainable modernization of a platform. 

 
 

SQL as a  pivot  model

The approach we propose is to use SQL as a pivot point to modernize data platforms.  This SQL will be the receptacle of the intelligence of the platform to be decommissioned.

 

The intelligence contained in the source platform(s) may become common to various current or future dataviz platforms. 

 

In this logic, we have developed an agnostic SQL generation engine based on different dataviz technologies, allowing an "As Is" migration of all intelligence to SQL.

 

Our engine works with the following technologies (others under study):  

 

But how do you migrate  to  SQL?

 

Introspection of the source technology

Our {openAudit} software retrieves structuring information such as the list of expressions and variables used, their nesting level, the functions used in these expressions, SQL queries, useful fields and tables, joins, custom SQL, contexts and prompts used in dashboards, etc.

 

Migrating an SQL

From this information, {openAudit} generates SQL to build the data pipelines in the target database. The SQL generated from the dataviz technology in source will be transposed into the target database, be it Azure SQL, BigQuery, Redshift, etc. 

 

Expected  benefits ​ 

 

Migrating intelligence to the target database

By creating a multi-platform business data layer in the target database, via new pipelines in simple "flat" SQL, the dataviz technology used will be able to make do with simple queries. This will improve performance and facilitate future migrations.

 

Dataviz, only dataviz

Teams working on data representation will be able to focus on creating reports and dashboards without worrying about data transformations, with simple, explicit models.

 

Shared maintenance for multiple tools

Centralizing SQL queries will simplify the maintenance of the dataviz layer. Updates and modifications can be made directly at the database level, without complex interventions on multiple platforms.

 

Vastly improved performance

By moving the complexity of the dataviz layer to simple SQL within databases, we will mechanically improve performance. Intelligence can be factored and data persisted, thus reducing the latency times of certain dataviz tools. Especially since hyperscaler databases are particularly efficient and resource-efficient.

 

Conclusion 

 

The SQL-based interoperability approach enables the creation of a flexible and scalable data architecture.

 

By centralizing data intelligence in simple SQL, companies can maximize the efficiency of their dataviz teams while ensuring data quality and consistency. Especially since this movement can be completely automated. Do not hesitate to contact us if you would like to know more. 

 

See also: 

 

Commentaires

Posts les plus consultés de ce blog

Migrer de SAP BO vers Power BI, Automatiquement, Au forfait !

La Data Observabilité, Buzzword ou nécessité ?

BCBS 239 : L'enjeu de la fréquence et de l'exactitude du reporting de risque