Interoperability of dataviz tools with SQL

 

Interoperability  of dataviz tools with SQL

In today's world of data management, information is often distributed across different dataviz platforms depending on business needs, technical issues, budgetary constraints, etc.   To optimize the management and use of this data, an SQL-based interoperability approach can help restore dataviz tools to what they initially are: "visualization" tools - not data prep tools ' that they have tended to become, creating a damaging increase in opacity and inertia.    It can be a relevant link in the context of a migration, for the sustainable modernization of a platform.

SQL  as a  pivotal  model

The approach we propose consists of using SQL as a pivot point to modernize data platforms.   This SQL will be the receptacle of the intelligence of the platform to be decommissioned.   The intelligence contained in the source platform(s) may become common to various current or future dataviz platforms.    In this logic, we have developed an agnostic SQL generation engine based on different dataviz technologies, allowing an "As Is" migration of all intelligence to SQL.   Our engine works for the following technologies (others under study):

But how to  migrate  to  SQL?

Source Technology Introspection Our {openAudit} software retrieves structuring information such as the list of expressions and variables used, their level of nesting, the functions used in these expressions, SQL queries, useful fields and tables, joins, custom SQL, contexts and prompts used in dashboards, etc.

Migration en SQL From this information, {openAudit} generates SQL to build the data pipelines in the target database. The SQL generated from the source dataviz technology will be transposed into the target database, whether Azure SQL, BigQuery, Redshift, etc.

The  expected  benefits

Migration of intelligence to the target database By creating a multi-platform business data layer in the target database, via new “flat” simple SQL pipelines, the dataviz technology used will be able to handle simple queries. This will improve performance and make future migrations easier.

Dataviz, only dataviz Teams working on data representation will be able to focus on creating reports and dashboards without worrying about data transformations, with simple, explicit models.

Shared maintenance for several tools Centralizing SQL queries will simplify the maintenance of the dataviz layer. Updates and modifications can be carried out directly at the database level, without complex interventions on several platforms.

Vastly improved performance By moving the complexity of the dataviz layer towards simple SQL within the databases, we will mechanically improve performance. The intelligence can be factorized and the data persisted, thus reducing the latency times of certain dataviz tools. Especially since the databases of hyperscalers are particularly efficient and not very resource-intensive.

Conclusion    The SQL-based interoperability approach creates a flexible and scalable data architecture.   By centralizing data intelligence in simple SQL, companies can maximize the efficiency of their dataviz teams while ensuring data quality and consistency. Especially since this movement can be completely automated.