Refactor cabal-install solver config log output

[erikd]

Includes:

• Apply some of @grayjay and @mpickering comments
• Fix Solver "rejecting" message is too verbose #4251

This is the PR #9541 rebased and fixed to build.

---

Template Α: This PR modifies behaviour or interface

Include the following checklist in your PR:

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
  • Is the change significant? If so, remember to add significance: significant in the changelog file.
• The documentation has been updated, if necessary.
• Manual QA notes have been included.
• Tests have been added. (Ask for help if you don’t know how to write them! Ask for an exemption if tests are too complex for too little coverage!)

[ulysses4ever]

Any chance you could add examples of what the new output looks like? Say, in the PR description.

[grayjay]

I haven't had a chance to look at the code yet, but, if I remember correctly, #9541 was a followup to #9159. Would it make sense to first update and merge #9159?

[erikd]

I haven't had a chance to look at the code yet, but, if I remember correctly, #9541 was a followup to #9159. Would it make sense to first update and merge #9159?

According to this comment this seems like a precursor to #9159 .

[grayjay]

According to this comment this seems like a precursor to #9159 .

The original change in #9159 was split into a refactoring change and a fix for #4251. Now the refactoring change is in #9159, and the fix for #4251 is in #9541. #9541 contains #9159, because the fix depends on the refactoring.

#9560 has also been merged since #9541 was written and helps address #4251. Do you know how this fix compares now?

[erikd]

Current version of this PR aims to minimize the differences in the cabal-install:unit-test output.

Still need to provide a information about how this version improves the solver output compared to the current output.

[erikd]

The changes between the output on master and the output in this PR is mostly incredibly minor (as shown by the tiny patch to the tests).

This is the only difference I could find in the cabal-install:unit-tests output:
master

      minimize conflict set with --minimize-conflict-set:                                                                              FAIL
        tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs:274:
        Unexpected error:
        Could not resolve dependencies:
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: D (dependency of A)
        [__1] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__1] fail (backjumping, conflict set: A, D)
        After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: A (5), D (4)
        Use -p '/minimize conflict set with --minimize-conflict-set/' to rerun this test only.
      show original conflict set with --no-minimize-conflict-set:                                                                      FAIL
        tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs:274:
        Unexpected error:
        Could not resolve dependencies:
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: B (dependency of A)
        [__1] rejecting: B-1.0.0 (conflict: A => B==2.0.0)
        [__1] fail (backjumping, conflict set: A, B)
        After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: A (7), B (2), C (2), D (2)
        Try running with --minimize-conflict-set to improve the error message.
        Use -p '/show original conflict set with --no-minimize-conflict-set/' to rerun this test only.

New:

      minimize conflict set with --minimize-conflict-set:                                                                              FAIL
        tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs:268:
        Unexpected solver log:
        targets: A
        constraints: 
          any.base installed (non-reinstallable package)
          any.ghc-bignum installed (non-reinstallable package)
          any.ghc-internal installed (non-reinstallable package)
          any.ghc-prim installed (non-reinstallable package)
          any.ghc installed (non-reinstallable package)
          any.integer-gmp installed (non-reinstallable package)
          any.integer-simple installed (non-reinstallable package)
          any.template-haskell installed (non-reinstallable package)
        preferences: 
        strategy: PreferLatestForSelected
        reorder goals: False
        count conflicts: True
        fine grained conflicts: True
        minimize conflict set: True
        independent goals: False
        avoid reinstalls: False
        shadow packages: False
        strong flags: False
        allow boot library installs: False
        only constrained packages: OnlyConstrainedNone
        max backjumps: infinite
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: B (dependency of A)
        [__1] rejecting: B-1.0.0 (conflict: A => B==2.0.0)
        [__1] fail (backjumping, conflict set: A, B)
        [__0] trying: A-2.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] next goal: C (dependency of A)
        [__2] rejecting: C-1.0.0 (conflict: A => C==2.0.0)
        [__2] fail (backjumping, conflict set: A, C)
        [__0] trying: A-1.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] trying: C-1.0.0 (dependency of A)
        [__3] next goal: D (dependency of A)
        [__3] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__3] fail (backjumping, conflict set: A, D)
        [__0] fail (backjumping, conflict set: A, B, C, D)
        Found no solution after exhaustively searching the dependency tree. Rerunning the dependency solver to minimize the conflict set ({A, B, C, D}).
        Trying to remove variable "A" from the conflict set.
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: B (dependency of A)
        [__1] rejecting: B-1.0.0 (conflict: A => B==2.0.0)
        [__1] fail (backjumping, conflict set: A, B)
        [__0] trying: A-2.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] next goal: C (dependency of A)
        [__2] rejecting: C-1.0.0 (conflict: A => C==2.0.0)
        [__2] fail (backjumping, conflict set: A, C)
        [__0] trying: A-1.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] trying: C-1.0.0 (dependency of A)
        [__3] next goal: D (dependency of A)
        [__3] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__3] fail (backjumping, conflict set: A, D)
        [__0] fail (backjumping, conflict set: A, B, C, D)
        Failed to remove "A" from the conflict set. Continuing with {A, B, C, D}.
        Trying to remove variable "B" from the conflict set.
        [__0] tryingE: A-3.0.0 (user goal)
        [__1] tryingE: C-1.0.0 (dependency of A)
        [__2] next goal: D (dependency of A)
        [__2] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__2] fail (backjumping, conflict set: A, D)
        [__0] skipping: A; 1.0.0, 2.0.0 (has the same characteristics that caused the previous version to fail: depends on 'D' but excludes version 1.0.0)
        [__0] fail (backjumping, conflict set: A, D)
        Successfully removed "B" from the conflict set. Continuing with {A, D}.
        Trying to remove variable "D" from the conflict set.
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: D (dependency of A)
        [__1] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__1] fail (backjumping, conflict set: A, D)
        [__0] skipping: A; 1.0.0, 2.0.0 (has the same characteristics that caused the previous version to fail: depends on 'D' but excludes version 1.0.0)
        [__0] fail (backjumping, conflict set: A, D)
        Failed to remove "D" from the conflict set. Continuing with {A, D}.

I suppose the main benefit of this PR is that in the file cabal-install-solver/src/Distribution/Solver/Modular/Message.hs detection of errors is separated from reporting of errors.

[erikd]

I haven't had a chance to look at the code yet, but, if I remember correctly, #9541 was a followup to #9159. Would it make sense to first update and merge #9159?

I have had a look at #9159 (against master from 18 months ago) but its rather difficult what really changes between them. I am studying #9159 more closely to figure out what it actually does.

Update: I am not able find any significant differences between the behavior of the code in this PR and in #9541 .

[grayjay]

Update: I am not able find any significant differences between the behavior of the code in this PR and in #9541 .

Isn't this PR an updated version of #9541? Do you mean #9560?

[erikd]

Update: I am not able find any significant differences between the behavior of the code in this PR and in #9541 .

Isn't this PR an updated version of #9541? Do you mean #9560?

Yes, I got confused. This is an updated version of #9541. Ad for #9560, that has been merged but does not seem to be related this PR. I do think that #9159 is related.

So the correct comment is, "I am not able find any significant differences between the behavior of the code in this PR and in #9159 ".

[grayjay]

Here is the current state of each of the PRs, as I understand it:

#9159: It contains two commits (b20ea53 and f10dbcf) that refactor the code in preparation for the improvement to error messages in #9541.
#9541: It contains the two commits from #9159, as well as a third commit (e4775b4) to improve error messages by removing duplication of package names.
#9560: It removes duplication of package names in error messages, without significant refactoring.

Since #9560 was already merged, it seems like the main difference between this PR and master is the refactoring. Are you interested in merging this just for the refactoring? Are you planning to make additional changes to the solver log that depend on it?

[erikd]

My hope is to get this refactor merged (after the fix suggested by @mpickering ). Then I intend to work on improving error messages as per commit e4775b4 .

[grayjay]

I meant that this PR already contains the contents of e4775b4 (removing duplicate package names, similar to #9560), so I was wondering if you were planning to make more changes beyond e4775b4 that depend on the refactoring.

[grayjay]

I responded to some of your latest comments and changes, but I haven't done another full review yet.

[grayjay]

I'm not sure that it's possible to convert the RetryLogss efficiently, but I think that it would be easier to remove the calls to retryMap, as I described in my previous comment.

[erikd]

Try rebasing on top of current master (if not already).

It says:

Current branch erikd/cosmetic-changes-2 is up to date.

[ulysses4ever]

Let me try to restart the failing jobs then. It's the same test in all configuration, so this test may be up for the flaky marker if it fails like that ...

[erikd]

I have left the new fixes for PR comments discrete for easier review. When this is approved for merge I will squash them down as appropriate.

[erikd]

One of the "Validate" failures is with ghc-9.6.7 and cabal-install-3.12.1.0. With those versions it passes locally.

The "Bootstrap" failure is a failure in Python code and my PR does not touch any Python code.

I have already tried rebasing against master.

[ulysses4ever]

I'll try to patch up the failing test (it's the same one again) today.

[ulysses4ever]

I restarted the bootstrap job, but it failed again in the same way: with http.client.IncompleteRead: IncompleteRead(3307 bytes read). https://github.com/haskell/cabal/actions/runs/15456775662/job/43546021620?pr=10854 So, network problems in GitHub runners strike again. I feel like GH is forcing us to make the testsuite more hermetic, which is a great thing!

[ulysses4ever]

I rebased the branch here on the latest master, which contains a fix for the failing test. Please, reset your local branch when you get there.

[ulysses4ever]

This PR needs a changelog entry.

Also, the PR description is missing a self-contained description of the change — please add one. I'd expect an example of solver output after this change.

[erikd]

I'd expect an example of solver output after this change.

Example solver output is in: #10854 (comment)

Solver output has not changed since that.

[ulysses4ever]

Example solver output is in: #10854 (comment)

Thank you. I think it should be referenced from or inserted in the PR description. Currently, it's hard to find.

So, the new output is longer than the old one. It's the opposite of what most people I talk to want. @Mikolaj @ffaf1 @geekosaur, thoughts?

[erikd]

So, the new output is longer than the old one. It's the opposite of what most people I talk to want. @Mikolaj @ffaf1 @geekosaur, thoughts?

The output in that example is actually a rare one. In most cases the differences are really minor as show by the differences requried to make the validate tests pass: 1b701f5

[grayjay]

I'm confused now. I had thought that we agreed that this PR was just a refactoring change to add an intermediate log message type (SummarizedMessage) to the processing of the solver log. I thought that all of the behavior changes from #9541 were already merged in #9560 and that any remaining changes to strings in the solver unit tests were just bugs that needed to be worked out.

Was the change in #10854 (comment) automatically caught by tests, or did you need to manually compare the solver output?

[geekosaur]

the new output is longer than the old one. It's the opposite of what most people I talk to want

I think longer is fine as long as it's more comprehensible. That's the real problem most people have with the current solver output, it's a lot of gobbledygook that requires someone who has some idea of what's going on inside the solver to interpret.

[erikd]

Was the change in #10854 (comment) automatically caught by tests, or did you need to manually compare the solver output?

I fiddled with one of the tests (do not even remember which one) to force a failure. The solver output was obviously different.

[grayjay]

I fiddled with one of the tests (do not even remember which one) to force a failure. The solver output was obviously different.

It would help to add a unit test that can detect that change, especially if it's unrelated to the changes caught by the unit tests that had changes to their expected strings.

I didn't look at the specific unit test in that comment, but the output from the first one ("master") looks like the summarized log, which is shown at normal verbosity, and the second one ("new") looks like the part of the log that is only shown at higher verbosity (-v3). Did you change the verbosity when running the tests, or is it possible that this PR changes which lines of the solver log are shown at different verbosities?

EDIT: I just noticed that the two test runs have different error messages. The first one says "Unexpected error", and the second one says "Unexpected solver log". Is it possible that you forced the failures in two different ways, so that there is only a difference in how the test displayed the solver output? Is the difference reproducible without the test framework?

[erikd]

Had to rebase this as chasing down changes was becoming too difficult.

In the validate test there is now a SINGLE difference:

diff --git a/cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs b/cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs
index a1f5eed3c6..0136e7cacf 100644
--- a/cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs
+++ b/cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs
@@ -208,7 +208,7 @@ tests =
               solverSuccess [("base", 1), ("ghc-prim", 1), ("integer-gmp", 1), ("integer-simple", 1)]
       , runTest $
           mkTest dbNonupgrade "Refuse to install newer ghc requested by another library" ["A"] $
-            solverFailure (isInfixOf "rejecting: ghc-2.0.0 (constraint from non-reinstallable package requires installed instance)")
+            solverFailure (isInfixOf "rejecting: ghc-1.0.0/installed-1 (conflict: A => ghc==2.0.0)")
       ]
   , testGroup
       "reject-unconstrained"

that I have been able to track down.

I was not able to find the difference that I found before. I tried subtle modifications to numerous of the validate tests and was comparing the difference between cabal-3.14 and the cabal build from my changes. I was not even able to find the change I note above.

[erikd]

🤦 The changes required to remove most of the changes to the validate test expected results causes the doctests to fail.

[grayjay]

@erikd I'm busy for the next few days, but I'll try to do another review in the next week.