FME Workspace Optimization: Attribute de-cluttering in 4 easy steps

One of the major challenges to having the most efficient workspaces also happens to be a real-life challenge:  Clutter!  Nobody likes it, and it can be a pain to get rid of. 

In FME, one of the biggest sources of clutter is the accumulation of unnecessary attributes.  These attributes can be there from the start or can be sneakily created by transformers as your workspace progresses.  To gain efficiency, get rid of these as early as possible in the workspace because more attributes = more processing time and more chances you will make mistakes in development.  These 4 steps will help you clear the attribute clutter and improve the efficiency of your workspaces.

1. Figure out which attributes you really need

The first step to de-cluttering your attributes is knowing which attributes you actually need to work with.  Start with your destination dataset and work your way back, asking yourself these key questions:

• What attributes do I need on my destination dataset?
• Do I need to create or calculate new attributes?
• If yes, what attributes do I need to generate or calculate those?

Once you have an idea of what you need, it becomes a lot easier to get rid of the rest.  And don’t be afraid – you can always bring attributes back if you realize that you need them later on in development.

2. Un-expose the attributes you don’t need

When you are setting up your reader, make sure you are only reading and exposing the attributes you need.  On the Reader Feature Type, you can uncheck unnecessary attributes so they are not brought into the workspace.

3. Keep the attribute clutter at bay with these key transformers

There are two key transformers that are useful for getting rid of attributes as you move through your workspace:  AttributeKeeper and AttributeRemover.  They are opposites, so in the AttributeKeeper you define the attributes you want to keep, and in the AttributeRemover, you define the attributes you want to get rid of.  Use these at any time throughout your workspace to clear your attribute clutter.  As a bonus, when you need to remove a lot of attributes based on a common expression (for example, all transformers that begin with PR_), you can use the BulkAttributeRemover.

4. Get your Transformers under control

Some transformers are notorious for generating lots of attributes (and list attributes) that you might not need.  Keep an eye out for transformers that join datasets together, for example, the FeatureMerger, SpatialRelator and the Aggregator.  You can deal with the attributes generated by these transformers configuring the parameters, and then by cleaning up the attributes after with an AttributeKeeper or AttributeRemover.  Most of these joining transformers contain a parameter section called “Attribute Accumulation” with two sub-sections: Merge Attributes and Generate List.  You can use both, neither or one or the other.

Merge Attributes

Merge Attributes controls how attributes are copied from one dataset to another.  Note that you cannot select which attributes to merge, so you might want to consider removing unnecessary attributes before this transformer.

Generate List

Generates an FME list (or array) of attributes containing all the attributes from the “Supplier” dataset. Lists are great for managing one-to-many relationships in your data, but they can create a lot of attribute clutter very quickly if not handled properly.  If you are creating a list, keep only the attributes you need by identifying them in the “Select Attributes” section, and make sure you clean up the list when you are done with it using the AttributeRemover.

Following these four steps can help clear the attribute clutter in your workspaces and improve the efficiency of your FME processing and development.  You can apply them to existing workspaces, or keep them in mind as you develop your next one!