WIP: Allow some task reference from source tasks #5179
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TBD
With the merge of #4524, source tasks could no longer depend on other tasks. Due to this change, the logic to calculate changes, especially in the
--watch
mode complicated, could be significantly simplified. Unfortunately, this also broke a common use-case, where sub-modules or cross-modules want to derive there source paths from the outer or parent module to provide a convenient and consistent default behavior. for that reason, thedef sourceFolders: Seq[os.SubPath]
was introduced, which could be modified in an object-orients fashion to cusomites the default behavior ofsources
. Although looking simple on paper, when facing an existing grown setup, it makes it really hard to understand the effective source path just from looking at the build script and modules code.Here are some issues I noticed:
sources
andsourceFolders
can be independently overridden without any consistency guarantees. It's unclear at a concrete module, mixing in various traits what their actual value issourceFolders
isn't a task, hence it can't be easily shown and inspected withmill show
.sources
may subsequently ignore or re-use the value ofsourceFolders
, leaving the maintainer in doubt about the actual paths usedsourceFolders
. In fact, existing source dir customization (e.g. in the Android and Kotlin modules) complete bypassessourceFolders
and just overridesources
, so the refactoring towardssourceFolders
is incompletesources
is a tedious and error-prone process. I tried to understand the value ofAndroidKotlinAppModule.sources
in a PR, where I neither could just runmill show
nor could I simply wald thesuper.sources
tree up. It costs a lot of time to get an understanding of such simple core setting of a module, leaving too little confidence in a setup involving lots of mix-ins.Consequences:
I think have two parallel structures the configure the sources (
sources
andsourceFolders
) is not going to age well. I already find it difficult to work with it. As a maintainer of various larger polyglot projects this change significantly raises the mental overhead. Therefore I think we should try to handleTask.Sources
derivations not outside of it. Instead we should allow task dependencies as we did before and removesourceFolders
. Each module level can beinspect
ed, the effective values can beshow
n. There is no need to useprintln
-debugging or a JVM debugger to understand the effective values ofsourceFolders
.What this POC tries to change:
Re-enable the ability to let
Task.Sources
depend on other task, but limit the allowed task types to justSources
tasks or maybe tasks returning the typeSeq[PathRef]
(and compatible). That way, the logic to re-evaluate the up-to-date state should be kept simple. We keep just checking thePathRef
s the final sources task is returning for changes.