As FME Server grows in usage and popularity – even, in many cases, expanding beyond the GIS department that first implemented it – keeping it organized and under control as well as making decisions regarding its architecture and usage can become challenging. Implementing an FME governance strategy in the early days can ensure that FME Server is being used efficiently, securely, and cost-effectively.
“’There is enough room in the sandbox for all the castles, but we have to agree on how to share the sand.”
The rise of FME Server
FME Server is increasingly being implemented by organizations to leverage data integration processes built on the FME platform. Across the board, the expectation for up-to-date, accurate and robust data is growing every day, so to are the technologies that deliver it.
“FME Server is the second piece of the FME platform that allows you to do more with data integration workflows created in FME Desktop. Use FME Server to run workspaces on a schedule, create Automations for real-time data processing, or build self-serve data portals. Stay updated by setting up notifications that alert you and your team when new information becomes available. Automate, manage, and control how data flows in a convenient web interface.”
– Safe Software’s FME Server Tour
At Consortech, we have been helping clients implement FME Server and Desktop technology for many years, in all kinds of organizations and contexts including government and private industry. One of the key things we have noticed is that the growth of FME in an organization is organic, from the ground up, one data process at a time. It often starts in a single department (in most cases the GIS department) as a tool to solve data problems, build data integrations, and automate data processes.
Over time, as the use and breadth of FME Server grows, it eventually takes on a more important role in the organization. Before this happens, it is important to consider the inherent challenges that come with this growth, and what strategies you can implement to manage it and ensure you are getting the most out of your FME Server.
A victim of its own success
The challenges that come with the rapid, organic growth of FME Server can vary among organizations, but most stem from a lack of planning, organization and knowing who is responsible for what. Here is a short, non-exhaustive list of some of the common topics we have seen become problems when FME Server is implemented without a governance strategy.
- Losing track of quality control for data processes: Did my processes run to completion? How do I know they worked? What happens if they don’t?
- Changing human resources and staff turnover: Shifting ownership of FME Server workflows, sharing and permissions. What do we do when this needs to happen? Who should own and have access to what?
- Inefficient job routing and engine usage: Not knowing what types of jobs to send to which engines, how to set it up, and who is responsible, anyway?
- Duplicate processes: Version control (or lack thereof), duplicate workflows on FME Server and confusion between FME Desktop and FME Server
- Permissions and security issues: What data can your FME Server access? Who oversees your database and web connections, and whose accounts are connected? Where are your connection files stored?
- Migrations to newer versions of FME: When and how often are you going to migrate? What components should be migrated? Who is responsible for this?
Key elements of an FME Governance strategy
FME Server is a powerful data integration tool that often grows organically and can quickly interface with many different departments, systems, and data sources in an organization. To make sure that it is running smoothly, efficiently, and securely, it is important to implement a governance strategy that is tuned and scaled to the business needs of the organization. When designing an FME governance strategy, here are a few key facets to consider.
1. People
When implementing technology, it is easy to forget one of the most important elements: humans. People drive the business needs, manage the technology, operate the tools and consume the products. It is crucial to consider the human element in any FME Governance strategy. Clearly outline roles and responsibilities of administrators, authors and users – this applies to both the FME Server and all of the data that it connects to. Consider how you will handle staff turnover: who owns the FME Server processes? What is the procedure when it needs to change?
Keep in mind that operating FME and FME Server requires skills and experience, and without proper training, it is possible to create problems in your IT environment. How will you train your staff, and what is the level of skill required to do the tasks related to FME and FME Server?
2. Technology
FME Server is an enterprise-level software that can connect with systems all across and even outside of an organization. Of course, technology is something to consider! When installing FME Server, it is important to ensure you have the right machine specifications, permissions and network setup. Ensure that FME Server is set up to meet organizational IT standards in terms of infrastructure, security, management, and disaster recovery. FME architecture choices can impact the level of service; consult with your stakeholders and make predictions for how the tool will be used in the short, medium, and long term, and build your FME Server architecture accordingly.
In version 2021.0, FME Server provides job information and server metrics which are important to take into consideration as well. As FME Server is inherently scalable (easy to add engines, for example, for more processing power), it is important to use all this information to configure it in a scalable way that runs efficiently and securely.
3. Standards
Set and enforce FME best practices, project and repository structure, and naming conventions. Understanding what is happening on your FME Server relies heavily on its administrators and Workspace authors following the same standards. Which best practices, structure and naming conventions you choose are up to you and can be tailored to the business needs and scale of your FME Server. These vary widely from organization to organization.
However, whatever you choose, make sure you stick to it and that someone is tasked with enforcing it. Planning FME Server standards in the early days creates the framework so that the tool can evolve in an organized and succinct manner which makes it easier to know what is happening and make good decisions accordingly.
Implementing an FME governance strategy sets an appropriate framework for the use of the tool while making space for evolution and scalability. A governance plan should be a living document, adapted to the changing needs of the business and IT requirements of the organization. While it is ideal to apply a governance strategy from the beginning of an FME Server implementation – it is never too late.
At any point, getting control and gaining a better understanding of how FME Server is being used will allow you to maximise your investment by running your data processes and automations in a secure, efficient, and cost-effective environment.