Filtering Resources in Maven

tl;dr - How do I configure Maven resource filtering?

Here at The Swoosh¹, we use an environment configuration tool that allows for properites in files to be substituted at deploy time with environment specific values. It works... ok? I mean it does its job. Finding information out about it is near impossible. Getting values added to the primary data store requires three or more levels of beauracratic hell. And... there is no facility for local deploy replacements.

This last deficiency has led to more than one team coming up with project specific hacks. My team kept two copies of config files, foo.properties and foo-dev.properties. Using Maven, and a "local" profile, one file was copied over the other.

In practice, this works. However, there is no guarantee that files are in sync, and our build artifact is published all the way out to prod with these vestigal dev files. Someone suggested we move the dev files out of our main source tree, but that did nothing to ensure the files stayed in sync.

My team is responsible for modernizing these apps and making them ready for things like delivering micro-services, auto-scaling in cloud environments, and continuous deployments. For me, the idea of modifying our build artifact between environments is a major roadblock to enabling the team to push code to production automatically. There is no guarantee that what goes through our QM process is what gets pushed. Sure, the stakes are low here - there is a "prod-ready" version of the file in the build, but if a dev makes changes to their local copy, they may neglect to migrate those changes. Hopefully, automated testing and QM will catch those mistakes, but I would rather try to address it.

Ideally, we could stuff local deployment values into our keystore but the overhead and the glacial speed at which changes are made make that a non-starter.

So, Maven filtering to the rescue.

• First, we created a "dev" directory that lived outside of the maven src/main hierarchy. This ensures that any files here won't get put into a build.

• Then, we created a "substitutions-source.properties" file that contained all the keys we were interested in replacing. The real system consults a substitution.txt file to determine which files should be parsed, so the name was chosen to complement that.

app.database.port=1521  

• With local values ready to go, it's time configure Maven. To our POM, we added

<profiles>  
    <profile>
      <id>local</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-resources-plugin</artifactId>
            <version>2.6</version>
            <configuration>

              <delimiters>
                <delimiter>${*}</delimiter>
                <delimiter>%%</delimiter>
              </delimiters>
            </configuration>
          </plugin>
        </plugins>
        <filters>
          <filter>${project.basedir}/dev/substitutions-source.properties</filter>
        </filters>
        <resources>
          <resource>
            <directory>${project.basedir}/src/main/resources</directory>
            <filtering>true</filtering>
          </resource>
        </resources>
      </build>
    </profile>

• What's going on here?
  • We created a profile called "local". We'll pass this profile on the command line to Maven to trigger this behavior. (eg. mvn clean install -P local)
  • For our needs, we needed to tell Maven to look for different delimiters. Our tool uses the syntax %%key.to.replace%% instead of the Maven standard ${key.to.replace}. We declare both of them so that normal Maven functionality continues to work.
  • We point to our file containing the values we want to use with the <filter> declaration.
  • And finally, we point to the files that we want to filter with the <resource> declaration.

So, what did we end up with? We ended up with a build artifact that is consistent between environments and does not shed dev files to prod and a process that ensures what we use locally mimics deployed environments as closely as possible.

Update

After some thought, I switched to using the Google Replacer plugin. It is a bit more powerful and allows more fine grained control. Here is the profile that replaces the above code:

<profile>  
      <id>local</id>
      <build>
        <plugins>
          <plugin>
            <groupId>com.google.code.maven-replacer-plugin</groupId>
            <artifactId>replacer</artifactId>
            <version>1.5.3</version>
            <executions>
              <execution>
                <phase>package</phase>
                <goals><goal>replace</goal></goals>
              </execution>
            </executions>
            <configuration>
              <regex>false</regex>
              <includes>
                <include>${project.basedir}/target/${project.build.finalName}/WEB-INF/classes/database.properties</include>
                <include>${project.basedir}/target/${project.build.finalName}/WEB-INF/classes/log4j.xml</include>
              </includes>
              <tokenValueMap>${project.basedir}/dev/substitutions-source.properties</tokenValueMap>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>

• Our source file remains the same, except now, we explicitly include the delimiters in the source file. This makes it easy to mix and match styles (%%foo%% and ${bar}) on the fly without having to declare them.
• Our list of files to filter becomes explicit. I would like to eventually be able to parse the same file our environment tool uses but for now we can duplicate them here.
• This is now an explicit token replacement step as opposed to extending Maven's. This just seems cleaner to me and ensures that the default build process is the same from environment to environment.

Further Reading:
Maven Filtering Examples
Google Code Replacer Plugin