TimeXtender democratizes access to corporate data enabling business users and liberating IT. The Data Warehouse Automation software TimeXtender combines with Qlik (or any other frontend tool) to create an unrivaled, comprehensive, end-to-end solution for Qlik users. TimeXtender helps companies tap into multiple data sources, extract relevant information, and place that information in a structured Modern Data Warehouse (MDW), allowing users to access information quickly and easily, and get one version of the truth. Plugged in TimeXtender gives companies the power to fully engage their organization. With more than 3,300 customers and various partners worldwide, TimeXtender is a clear leader in Data Warehouse Automation.
Using the Modern Data Warehouse (MDW) from TimeXtender allows integration and consolidation of the data from any application or data source into a single database. The TimeXtender (TX) solution comes ready with more than 250 ready-to-use connectors for most databases and applications including those that are common to managing the delivery of construction projects. This will enable TX to automatically write the codes required for extracting the needed data from all applications that are needed to be reported on.
Staging tables can be created to review the data extracted from the different data sources as well as the rejected data tables to save, but not use unwanted data. The staging tables are used to temporarily store data, quality checks the data, and perform mathematical operations (aggregation, averaging) as necessary before loading the data into the data warehouse.
The TX application allows the carrying out and automation of the data cleansing and data validation processes that need to be carried out when extracting data from different databases. Different tests can be added to cleanse the data and report on erroneous, invalid, and duplicate data. When this is done, the data from the staging tables will be loaded into the fact tables.
After selecting the needed tables, the transfer command is executed to bring the data from TX Operational Data Store (ODX) or Data Lake into the TX Modern Data Warehouse (MDW). TX uses multiple threads (Orange) to expedite the data transfer process (Light Blue). During the transfer process, TX will also do the data cleansing (Dark Blue) and data transfer error identification (Red) as per the defined rules.
The Modern Data Warehouse (MDW) is used to de-normalize the extracted data from the different data sources to enable associating and blending of multiple data fields to create multi-data source tables and views without the lengthy and costly data integration effort. The result is a unified structure that is automatically updated and easier to maintain in cases of changes and new versions of data sources.
TX provides the option for creating new database tables known as views. Those views would integrate data from the different applications as well as include calculated fields. Those could be dynamic views or static views. Dynamic views can contain data from one or two tables and automatically include all of the columns from the specified table or tables. Dynamic views are automatically updated when related objects or extended objects are created or changed.
Using the Modern Data Warehouse (MDW), the PMIS team can decouple the Data Hub from the sources and store and maintain their extracted data sources for as long as it will be needed. Multiple schedules can be predefined to automatically extract, load, and transfer (ETL) new data either incremental or full data from the linked data sources to the Modern Data Warehouse (MDW). It is highly recommended that the data update be scheduled to run automatically on daily basis during the night and after working hours.
The Modern Data Warehouse (MDW) is used to create semantic models that include all measures, whether standard, derived, or calculated. This allows the creation of what is known as Cubes. A cube consists of several Measures and Dimensions and could be a 3-dimensional, 4-dimensional, or 5-dimensional structure that stores facts organized by dimensions to allow quick retrieval of data and increased performance in processing basic mathematical functions.
Those cubes or data marts can then be used in any business intelligence visualization like MS Power BI or reporting tool like MS SQL Reporting Services (SSRS) to ensure a single version of the truth when it comes to reporting projects’ performance, status, and other needed information.
To ensure that access to the extracted, transferred and loaded data is aligned with the access permission rights, TX data security provides object-level permissions to the different database roles. It is possible to restrict access to specific tables, schemas, or views according to specific Roles (for example executive stakeholders, PCS managers and engineers, and other project team members). In this way, single security can be managed for front-end applications in place of all sources’ security. In addition, Data Level Security can be added in a way to filter data for different roles.
All security and access permission rights will be documented with the rest of the data warehouse automation (DWA) processes in the automatically created documentation PDF file that the Discovery Hub generates. This will save a big part of manual work and costly effort related to Traceability and Compliance documentation.
All the activities that were completed to extract, transfer, load, and present the data from the different data sources will be documented in a manual that will be automatically generated from what has been done. The manual will include details of all extracted tables, what fields are in each table, what fields were used, links between those tables, details of all created measures with a lineage chart to show details of each measure, and other documentation needed for the created data warehouse.
Similar to other implemented solutions which use separate instances for development and production, the created data warehouse will be carried out using the TX development instance for which it will be subjected to User Acceptance Testing (UAT) before the metadata can be transferred into the production environment. This optional process will be repeated whenever there is a new version of the created data warehouse. TX allows selecting previously created versions to make it the current production version whenever this might be needed.
