SPI and the Shadow of Flyway

I got a simple web app with Java and Postgres, and Flyway for database migrations. Everything was woking fine until I deployed it to cloud. On the startup, I got this error:

Exception in thread "main" org.flywaydb.core.api.FlywayException: Unknown prefix for location (should be one of filesystem:, classpath:, gcs:, or s3:): classpath:db/callback        at org.flywaydb.core.internal.scanner.LocationParser.lambda$parseLocation$1(LocationParser.java:59)        
        at java.base/java.util.Optional.orElseThrow(Unknown Source)
        at org.flywaydb.core.internal.scanner.LocationParser.parseLocation(LocationParser.java:59)
        at org.flywaydb.core.internal.scanner.ClasspathClassScanner.scanForType(ClasspathClassScanner.java:44)
        at org.flywaydb.core.api.configuration.ClassicConfiguration.loadCallbackLocation(ClassicConfiguration.java:1639)
        at org.flywaydb.core.Flyway.<init>(Flyway.java:132)
        at org.flywaydb.core.api.configuration.FluentConfiguration.load(FluentConfiguration.java:64)

There are two problems with this error. First, it mentions some callback location that I never used. In fact, the Flyway config was as simple as

Flyway.configure()
  .dataSource(hikariDataSource)
  .locations("classpath:db/migration") // this line can also be omitted as it's a default location
  .load()

Second, it basically tells that the location prefix is incorrect and lists the supported prefixes but then it mentions the one it got (classpath:db/callback) that has in fact prefix that is supported.

If we look into LocationParser.java:59 from the error stack:

final ReadOnlyLocationHandler locationHandler = Flyway.configure()
    .getPluginRegister()
    .getInstancesOf(ReadOnlyLocationHandler.class)
    .stream()
    .filter(x -> x.canHandlePrefix(prefix))
    .findFirst()
    .orElseThrow(() -> new FlywayException(
        "Unknown prefix for location (should be one of filesystem:, classpath:, gcs:, or s3:): "
            + normalizedDescriptor));

we can see that the error message is misleading. The problem is not that the prefix is incorrect, it's that Flyway could not find a location handler that can handle this prefix.

Why is that? First, let's log what handlers are available.

Flyway.configure().getPluginRegister().getInstancesOf(ReadOnlyLocationHandler.class)

Locally, it prints

[org.flywaydb.core.internal.scanner.classpath.ClasspathLocationHandlerImpl@3c756e4d, org.flywaydb.core.internal.scanner.filesystem.FilesystemLocationHandler@7c0e2abd]

Two handlers are available, and the first one, ClasspathLocationHandlerImpl, is what we need.

Let's try on cloud:

[]

No handlers available at all.

What's going on? I use ShadowJar Gradle plugin to make Fat Jar and then run it on cloud. Actually, if I make Fat Jar locally and run it instead of running classes directly, it will fail in the same way. The problem has to do with Service Provider Interface.

SPI is a Java mechanism for runtime extensibility. It allows library users to extend the library in runtime and without making any modification to the library code itself. This is achieved by creating an interface that is expected to be extended in the library, and then anyone can implement it. In order for the library to discover that external implementation, or service, it needs to be registered in META-INF/services, in a file with name equal to the interface's full name. Then those services can be loaded with java.util.ServiceLoader.

Let's look at the example in flyway-core. It has one file in META-INF/services:

org.flywaydb.core.extensibility.Plugin

This Plugin is an interface that Flyway provides for extensibility. The content of the file is a list of services that are provided by the library itself. Among them, we can see ClasspathLocationHandlerImpl that we need.

To use Flyway with Postgres, I also imported flyway-database-postgresql library. It also has one file in its META-INF/services with the same name, but inside there are different classes, those that are used for Postgres support.

When running locally, all those META-INF/services files are available individually for every library. But when you produce a Fat Jar, there's just one resulting META-INF/services directory. When ShadowJar plugin meets a duplicate file, it just overrides the previous one. As a result, the list of services in my app only contained Postgres-specific Flyway services coming from flyway-database-postgresql library, but not those basic ones coming from flyway-core.

To fix this, we need to tell Shadow to merge service files:

mergeServiceFiles()

We also need to set duplicate strategy to make sure both lists are included:

duplicatesStrategy = DuplicatesStrategy.INCLUDE

So the final shadowJar block is

tasks.shadowJar {
    mergeServiceFiles()
    duplicatesStrategy = DuplicatesStrategy.INCLUDE
}