The Niagara Region encompasses 12 municipalities in southern Ontario. The region is responsible for treating water and managing the master infrastructure data plan. Municipalities are responsible for the local water distribution system and the wastewater collection system.
The water department team wanted to accelerate the integration of municipal water and sewer data with transformation, validation, and correction processes. They also wanted to work with other municipalities to improve data quality over time.
WHAT WE IMPLEMENTED
This project involved extracting, transforming, validating, and distributing data. Transaction reports, consulting, and training were also part of the project. More specifically, the Consortech team:
- Developed FME workspaces to extract, transform, validate, and correct data
- Automated transformation, validation, and remediation tasks using an FME workspace to orchestrate the process
- Ensured that FME workspaces can be configured using a configuration file to make it easier to operate the system
CHALLENGES
To succeed, it was necessary to find the right balance between having an easy-to-operate system and managing complex data. The challenge also revolved around developing a powerful solution that gives users fast feedback while being scalable to meet future needs.
The project was divided into several phases:
- Analysis of the client’s situation
- Technical meeting with stakeholders
- Presentation and acceptance of a detailed work plan
- Development of the solution in an agile environment with frequent deliveries
Staged delivery enabled the client to provide continuous feedback. As a result, the Consortech team could make adjustments as they went. To mitigate the learning curve, knowledge transfer and training were done after each delivery.
RESULTS
The client now has a system that enables them to integrate infrastructure data about distributing drinking water and managing wastewater from external suppliers in their territory. In addition, by leveraging the use of configuration files, the client can exploit the same system for transforming, validating, and correcting of other types of data—benefiting from greater flexibility!
“Consortech’s acute expertise in FME and ETL processes helped us save 50% on the time it takes to integrate water and wastewater data into our regional model by automating critical steps.”