# Sharing outputs between projects

version 6.5-milestone-1

A common pattern, in multi-project builds, is that one project consumes the artifacts of another project. In general, the simplest consumption form in the Java ecosystem is that when `A` depends on `B`, then `A` would depend on the `jar` produced by project `B`. As previously described in this chapter, this is modeled by `A` depending on a variant of `B`, where the variant is selected based on the needs of `A`. For compilation, we need the API dependencies of `B`, provided by the `apiElements` variant. For runtime, we need the runtime dependencies of `B`, provided by the `runtimeElements` variant.

However, what if you need a different artifact than the main one? Gradle provides, for example, built-in support for depending on the test fixtures of another project, but sometimes the artifact you need to depend on simply isn’t exposed as a variant.

In order to be safe to share between projects and allow maximum performance (parallelism), such artifacts must be exposed via outgoing configurations.

| Don’t reference other project tasks directly A frequent anti-pattern to declare cross-project dependencies is:
    dependencies {
       // this is unsafe!
       implementation project(":other").tasks.someOtherJar
    }
This publication model is unsafe and can lead to non-reproducible and hard to parallelize builds. This section explains how to properly create cross-project boundaries by defining "exchanges" between projects by using variants.

There are two, complementary, options to share artifacts between projects. The simplified version is only suitable if what you need to share is a simple artifact that doesn’t depend on the consumer. The simple solution is also limited to cases where this artifact is not published to a repository. This also implies that the consumer does not publish a dependency to this artifact. In cases where the consumer resolves to different artifacts in different contexts (e.g., different target platforms) or that publication is required, you need to use the advanced version.

## Simple sharing of artifacts between projects

First, a producer needs to declare a configuration which is going to be exposed to consumers. As explained in the configurations chapter, this corresponds to a consumable configuration.

Let’s imagine that the consumer requires instrumented classes from the producer, but that this artifact is not the main one. The producer can expose its instrumented classes by creating a configuration that will "carry" this artifact:

Example 1. Declaring an outgoing variant
producer/build.gradle

    configurations {
        instrumentedJars {
            canBeConsumed = true
            canBeResolved = false
            // If you want this configuration to share the same dependencies, otherwise omit this line
            extendsFrom implementation, runtimeOnly
        }
    }

producer/build.gradle.kts

    val instrumentedJars by configurations.creating {
        isCanBeConsumed = true
        isCanBeResolved = false
        // If you want this configuration to share the same dependencies, otherwise omit this line
        extendsFrom(configurations["implementation"], configurations["runtimeOnly"])
    }

This configuration is consumable, which means it’s an "exchange" meant for consumers. We’re now going to add artifacts to this configuration, that consumers would get when they consume it:

Example 2. Attaching an artifact to an outgoing configuration
producer/build.gradle

    artifacts {
        instrumentedJars(instrumentedJar)
    }

producer/build.gradle.kts

    artifacts {
        add("instrumentedJars", instrumentedJar)
    }

Here the "artifact" we’re attaching is a task that actually generates a Jar. Doing so, Gradle can automatically track dependencies of this task and build them as needed. This is possible because the `Jar` task extends `AbstractArchiveTask`. If it’s not the case, you will need to explicitly declare how the artifact is generated.

Example 3. Explicitly declaring the task dependency of an artifact
producer/build.gradle

    artifacts {
        instrumentedJars(someTask.outputFile) {
            builtBy(someTask)
        }
    }

producer/build.gradle.kts

    artifacts {
        add("instrumentedJars", someTask.outputFile) {
            builtBy(someTask)
        }
    }

Now the consumer needs to depend on this configuration in order to get the right artifact:

Example 4. An explicit configuration dependency
consumer/build.gradle

    dependencies {
        instrumentedClasspath(project(path: ":producer", configuration: 'instrumentedJars'))
    }

consumer/build.gradle.kts

    dependencies {
        instrumentedClasspath(project(mapOf(
            "path" to ":producer",
            "configuration" to "instrumentedJars")))
    }

| Declaring a dependency on an explicit target configuration is not recommended if you plan to publish the component which has this dependency: this would likely lead to broken metadata. If you need to publish the component on a remote repository, follow the instructions of the variant-aware cross publication documentation.

In this case, we’re adding the dependency to the instrumentedClasspath configuration, which is a consumer specific configuration. In Gradle terminology, this is called a resolvable configuration, which is defined this way:

Example 5. Declaring a resolvable configuration on the consumer
consumer/build.gradle

    configurations {
        instrumentedClasspath {
            canBeConsumed = false
            canBeResolved = true
        }
    }

consumer/build.gradle.kts

    val instrumentedClasspath by configurations.creating {
        isCanBeConsumed = false
        isCanBeResolved = true
    }

## Variant-aware sharing of artifacts between projects

In the simple sharing solution, we defined a configuration on the producer side which serves as an exchange of artifacts between the producer and the consumer. However, the consumer has to explicitly tell which configuration it depends on, which is something we want to avoid in variant aware resolution. In fact, we also have explained that it is possible for a consumer to express requirements using attributes and that the producer should provide the appropriate outgoing variants using attributes too. This allows for smarter selection, because using a single dependency declaration, without any explicit target configuration, the consumer may resolve different things. The typical example is that using a single dependency declaration `project(":myLib")`, we would either choose the `arm64` or `i386` version of `myLib` depending on the architecture.

To do this, we will add attributes to both the consumer and the producer.

| It is important to understand that once configurations have attributes, they participate in variant aware resolution, which means that they are candidates considered whenever any notation like `project(":myLib")` is used. In other words, the attributes set on the producer must be consistent with the other variants produced on the same project. They must not, in particular, introduce ambiguity for the existing selection. In practice, it means that the attribute set used on the configuration you create are likely to be dependent on the ecosystem in use (Java, C++, …​) because the relevant plugins for those ecosystems often use different attributes.

Let’s enhance our previous example which happens to be a Java Library project. Java libraries expose a couple of variants to their consumers, `apiElements` and `runtimeElements`. Now, we’re adding a 3rd one, `instrumentedJars`.

Therefore, we need to understand what our new variant is used for in order to set the proper attributes on it. Let’s look at the attributes we find on the `runtimeElements` configuration:

gradle outgoingVariants --variant runtimeElements

    Attributes
        - org.gradle.category            = library
        - org.gradle.dependency.bundling = external
        - org.gradle.jvm.version         = 11
        - org.gradle.libraryelements     = jar
        - org.gradle.usage               = java-runtime

What it tells us is that the Java Library plugin produces variants with 5 attributes:

- `org.gradle.category` tells us that this variant represents a library

- `org.gradle.dependency.bundling` tells us that the dependencies of this variant are found as jars (they are not, for example, repackaged inside the jar)

- `org.gradle.jvm.version` tells us that the minimum Java version this library supports is Java 11

- `org.gradle.libraryelements` tells us this variant contains all elements found in a jar (classes and resources)

- `org.gradle.usage` says that this variant is a Java runtime, therefore suitable for a Java compiler but also at runtime

As a consequence, if we want our instrumented classes to be used in place of this variant when executing tests, we need to attach similar attributes to our variant. In fact, the attribute we care about is `org.gradle.libraryelements` which explains what the variant contains, so we can setup the variant this way:

Example 6. Declaring the variant attributes
producer/build.gradle

    configurations {
        instrumentedJars {
            canBeConsumed = true
            canBeResolved = false
            attributes {
                attribute(Category.CATEGORY_ATTRIBUTE, objects.named(Category, Category.LIBRARY))
                attribute(Usage.USAGE_ATTRIBUTE, objects.named(Usage, Usage.JAVA_RUNTIME))
                attribute(Bundling.BUNDLING_ATTRIBUTE, objects.named(Bundling, Bundling.EXTERNAL))
                attribute(TargetJvmVersion.TARGET_JVM_VERSION_ATTRIBUTE, JavaVersion.current().majorVersion.toInteger())
                attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named(LibraryElements, 'instrumented-jar'))
            }
        }
    }

producer/build.gradle.kts

    val instrumentedJars by configurations.creating {
        isCanBeConsumed = true
        isCanBeResolved = false
        attributes {
            attribute(Category.CATEGORY_ATTRIBUTE, namedAttribute(Category.LIBRARY))
            attribute(Usage.USAGE_ATTRIBUTE, namedAttribute(Usage.JAVA_RUNTIME))
            attribute(Bundling.BUNDLING_ATTRIBUTE, namedAttribute(Bundling.EXTERNAL))
            attribute(TargetJvmVersion.TARGET_JVM_VERSION_ATTRIBUTE, JavaVersion.current().majorVersion.toInt())
            attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, namedAttribute("instrumented-jar"))
        }
    }

    inline fun <reified T: Named> Project.namedAttribute(value: String) = objects.named(T::class.java, value)

| Choosing the right attributes to set is the hardest thing in this process, because they carry the semantics of the variant. Therefore, before adding new attributes, you should always ask yourself if there isn’t an attribute which carries the semantics you need. If there isn’t, then you may add a new attribute. When adding new attributes, you must also be careful because it’s possible that it creates ambiguity during selection. Often adding an attribute means adding it to all existing variants.

What we have done here is that we have added a new variant, which can be used at runtime, but contains instrumented classes instead of the normal classes. However, it now means that for runtime, the consumer has to choose between two variants:

- `runtimeElements`, the regular variant offered by the `java-library` plugin

- `instrumentedJars`, the variant we have created

In particular, say we want the instrumented classes on the test runtime classpath. We can now, on the consumer, declare our dependency as a regular project dependency:

Example 7. Declaring the project dependency
consumer/build.gradle

    dependencies {
        testImplementation 'junit:junit:4.13'
        testImplementation project(':producer')
    }

consumer/build.gradle.kts

    dependencies {
        testImplementation("junit:junit:4.13")
        testImplementation(project(":producer"))
    }

If we stop here, Gradle will still select the `runtimeElements` variant in place of our `instrumentedJars` variant. This is because the `testRuntimeClasspath` configuration asks for a configuration which `libraryelements` attribute is `jar`, and our new `instrumented-jars` value is not compatible.

So we need to change the requested attributes so that we now look for instrumented jars:

Example 8. Changing the consumer attributes
consumer/build.gradle

    configurations {
        testRuntimeClasspath {
            attributes {
                attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named(LibraryElements, 'instrumented-jar'))
            }
        }
    }

consumer/build.gradle.kts

    configurations {
        testRuntimeClasspath {
            attributes {
                attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named(LibraryElements::class.java, "instrumented-jar"))
            }
        }
    }

Now, we’re telling that whenever we’re going to resolve the test runtime classpath, what we are looking for is instrumented classes. There is a problem though: in our dependencies list, we have JUnit, which, obviously, is not instrumented. So if we stop here, Gradle is going to fail, explaining that there’s no variant of JUnit which provide instrumented classes. This is because we didn’t explain that it’s fine to use the regular jar, if no instrumented version is available. To do this, we need to write a compatibility rule:

Example 9. A compatibility rule
consumer/build.gradle

    class InstrumentedJarsRule implements AttributeCompatibilityRule<LibraryElements> {

        @Override
        void execute(CompatibilityCheckDetails<LibraryElements> details) {
            if (details.consumerValue.name == 'instrumented-jar' && details.producerValue.name == 'jar') {
                details.compatible()
            }
        }
    }

consumer/build.gradle.kts

    open class InstrumentedJarsRule: AttributeCompatibilityRule<LibraryElements> {

        override fun execute(details: CompatibilityCheckDetails<LibraryElements>) = details.run {
            if (consumerValue?.name == "instrumented-jar" && producerValue?.name == "jar") {
                compatible()
            }
        }
    }

which we need to declare on the attributes schema:

Example 10. Making use of the compatibility rule
consumer/build.gradle

    dependencies {
        attributesSchema {
            attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE) {
                compatibilityRules.add(InstrumentedJarsRule)
            }
        }
    }

consumer/build.gradle.kts

    dependencies {
        attributesSchema {
            attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE) {
                compatibilityRules.add(InstrumentedJarsRule::class.java)
            }
        }
    }

And that’s it! Now we have:

- added a variant which provides instrumented jars

- explained that this variant is a substitute for the runtime

- explained that the consumer needs this variant only for test runtime

Gradle therefore offers a powerful mechanism to select the right variants based on preferences and compatibility. More details can be found in the variant aware plugins section of the documentation.

| By adding a value to an existing attribute like we have done, or by defining new attributes, we are extending the model. This means that all consumers have to know about this extended model. For local consumers, this is usually not a problem because all projects understand and share the same schema, but if you had to publish this new variant to an external repository, it means that external consumers would have to add the same rules to their builds for them to pass. This is in general not a problem for ecosystem plugins (e.g: the Kotlin plugin) where consumption is in any case not possible without applying the plugin, but it is a problem if you add custom values or attributes. So, avoid publishing custom variants if they are for internal use only.

## Targeting different platforms

It is common for a library to target different platforms. In the Java ecosystem, we often see different artifacts for the same library, distinguished by a different classifier. A typical example is Guava, which is published as this:

- `guava-jre` for JDK 8 and above

- `guava-android` for JDK 7

The problem with this approach is that there’s no semantics associated with the classifier. The dependency resolution engine, in particular, cannot determine automatically which version to use based on the consumer requirements. For example, it would be better to express that you have a dependency on Guava, and let the engine choose between `jre` and `android` based on what is compatible.

Gradle provides an improved model for this, which doesn’t have the weakness of classifiers: attributes.

In particular, in the Java ecosystem, Gradle provides a built-in attribute that library authors can use to express compatibility with the Java ecosystem: `org.gradle.jvm.version`. This attribute expresses the minimal version that a consumer must have in order to work properly.

When you apply the `java` or `java-library` plugins, Gradle will automatically associate this attribute to the outgoing variants. This means that all libraries published with Gradle automatically tell which target platform they use.

By default, the `org.gradle.jvm.version` is set to the target compatibility of the source set.

While this attribute is automatically set, Gradle will not, by default, let you build a project for different JVMs. If you need to do this, then you will need to create additional variants following the instructions on variant-aware matching.

| Future versions of Gradle will provide ways to automatically build for different Java platforms.