Article #094 Overcoming the Challenges of Reaping Benefits from Big Data in the Construction Industry

More than ever, industries of all types understand the value that data can bring to their business in terms of achieving their organizational objectives and delivering their business strategy. Businesses today are using data for better and faster-informed decisions, a better understanding of their customer’s needs and habits, providing smarter services and products, improving their internal processes and operations, and creating additional revenue (Marr, B., (2019, March). Why Every Company Needs A Data Strategy For 2019, Forbes, Retrieved from This explains the increasingly growing trend of data capturing that is estimated to exceed 44 zettabytes by 2020.

Challenges to Reap Benefits from Big Data in the Construction Industry

The construction industry is no different from any other industry when it comes to the great value that they will achieve from reaping the benefits of having the right data from their projects. Nevertheless, many challenges are limiting the construction industry to take the many advantages of having access to their everyday project data. The details of the top six challenges that are prohibiting the construction industry, and in particular in the Middle East region, today from benefiting from the Big Data available on their projects are listed below. In addition, the proposed solution for each challenge is also detailed.




Challenge #1 Massive Volume of Wasted Data

One of the key challenges that face the construction industry is that many of the project management-related processes continue to be manual and lack the needed automation that other industries have adopted. Surprisingly, the construction industry is using technology to plan and update their projects’ schedules and CAD/BIM to design the projects but yet avoids using the right technology to capture project and contract management processes information. Not only this affects the quality and credibility of captured data but it also means that much of the everyday data gets wasted and neither gets captured nor shared.

This challenge can be mitigated by using Project Management Information Systems (PMIS) like PMWeb to automate everyday project management processes. This will ensure that all originated projects’ data are not only captured in the desired structured format but also robust, reliable, auditable, attributable, traceable, transparent, and trustworthy.

Challenge #2 Multiple Applications and Data Silos

Another challenge is that the construction industry will continue to use multiple software applications to capture everyday data that each has its data source. For example, construction projects use BIM and other applications to create the project design documents, scheduling software to plan the project activities and resource requirements, Monte Carlo simulation software to quantify schedule and cost risks, project management information system to automate project management processes, sensors data using IoT, ERP applications to manage home office operations along with the many new cloud-based applications that are addressing different construction projects’ requirements. Not only this creates the issue of multiple data sources, but some of those applications could eventually be dropped by the organization or even discontinued by the software vendor.

This challenge can be strongly mitigated by using a data management platform like Discovery Hub® from TimeXtender which can be used to integrate and consolidate data from any application or data source into a single Data Hub aka operational data storage (ODX). In this way the organization will always have access to data from a single place for any kind of request from a business, analytics, or other applications, allowing it to easily extract the value from insight into data.

Challenge #3 Non-Unified Data Structures and Field Names

Another challenge that is also associated with the multiple applications is the fact that each of those application data sources comes with its own different data table structures and field names that can prove to be very costly and time-consuming to integrate so they can become of value to the organization. Some of the bespoke built integrations might even become obsolete when a new version is released for any of the integrated products.

Discovery Hub® from TimeXtender helps to normalize and de-normalize those data sources to enable associating and blending multiple data fields to create multi-data source tables and views without the lengthy and costly data integration effort. The result is a unified structure that is automatically updated and easier to maintain in cases of changes and new versions of data sources (see challenge #4).

Challenge #4 Discontinued Access to Applications

Another challenge has to do with the fact that projects will eventually come to an end where ownership of the captured data needs to be transferred to another business unit. This brings the high risk of misplacing and losing this valuable data when accountability for securing this data is disrupted. In addition, there is always the risk of the non-availability of the software applications originally used to populate the captured data or expired software subscription agreements for using those applications.

Using an application like Discovery Hub® from TimeXtender, the organization can decouple the Data Hub from the sources and store and maintain their extracted data sources for as long as they want. Multiple schedules can be predefined to automatically extract, load, and transfer new data (incremental) or full data from the linked data sources to the Discovery Hub®.

Challenge #5 Differing Data Visualization and Reporting Needs

There is also the challenge of project managers and senior business executives who could be users of different visualization and reporting tools and who need to be governed by semantic models with the same documented data sets to visualize or report on. Usually, project management team members need reports that can be printed or shared whereas executives tend to use interactive business intelligence and data visualization tools like MS Power BI, Qlik, and Tableau.

With Discovery Hub® it is possible to create semantic models, creating all metrics and measures, whether derived or calculated, so they can be used in any business intelligence visualization or reporting tool to ensure a single version of the truth when it comes to reporting projects’ performance, status, and other needed information. In this way, a single business language is used and can be easily shared through different data visualization tools.

Challenge #6 Restricted Data Access

Getting access to the data warehouse is the last challenge which also relates to the confidentiality requirement for this industry where very few senior executives would usually be permitted to have access to the complete captured data. This requires restricting access to certain parts of the captured data and being only limited to certain executives within the organization.

TimeXtender data security provides object-level permissions to the different database roles. It is possible to restrict access to specific tables, schemas, or views according to specific Roles (for example Executives, Managers, and Developers). In this way, single security can be managed for front-end applications in place of all sources’ security. In addition, Data Level Security can be added in a way to filter data for different roles. All security will be documented with the rest of the model in the automatically generated dynamic pdf generated by Discovery Hub, which can save a big part of manual work related to Traceability and Compliance.

The Conclusion

In conclusion, organizations in the construction industry must overcome the many challenges that are prohibiting them from realizing the many benefits that data can bring to their business in terms of achieving their organizational objectives and delivering their business strategy. Many of those challenges relate to the challenges in extracting, transferring, loading, associating, blending, storing, and maintaining the multi-data source data before they can benefit from having access to robust, reliable, auditable, attributable, traceable, transparent, and trustworthy data. Those organizations must use applications or platforms that will end the many manual-based processes for data capturing as well as ensure that there is a single data estate to store this priceless information.

About the Authorfounder

Bassam Samman, PMP, PSP, EVP, GPM is a Senior Project Management Consultant with more than 35-year service record providing project management and controls services to over 100 projects with a total value of over US $5 Billion. Those projects included Commercial, Residential, Education and Healthcare Buildings and Infrastructure, Entertainment and Shopping Malls, Oil and Gas Plants and Refineries, and Telecommunication and Information Technology projects. He is thoroughly experienced in complete project management including project management control systems, computerized project control software, claims analysis/prevention, risk analysis/management (contingency planning), design, supervision, training, and business development.

Bassam is a frequent speaker on topics relating to Project Management, Strategic Project Management, and Project Management Personal Skills. Over the past 35 years, he has lectured at more than 350 events and courses at different locations in the Middle East, North Africa, Europe, and South America. He has written more than 250 articles on project management and project management information systems that were featured in international and regional magazines and newspapers. He is a co-founder of the Project Management Institute- Arabian Gulf Chapter (PMI-AGC) and has served on its board of directors for more than 6 years. He is a certified Project Management Professional (PMP) from the Project Management Institute (PMI), a certified Planning and Scheduling Professional (PSP), and Earned Value Professional (EVP) from the American Association of Cost Engineers (AACE) and Green Project Management (GPM).

Bassam holds a Masters in Engineering Administration (Construction Management) with Faculty Commendation, George Washington University, Washington, D.C., USA, Bachelor in Civil Engineering – Kuwait University, Kuwait and has attended many executive management programs at Harvard Business School, Boston, USA and London Business School, London, UK.


Contact us