Skip to content

Fix how QUADOBJ section written to MPS files #3609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

dansplain
Copy link

@dansplain dansplain commented May 21, 2025

Fixes: #2122

  • Fixed QUADOBJ section in _print_model_MPS function of ProblemWrite_mps class to correctly output only the non-zero elements of the upper (or lower) triangle of the symmetric Q matrix.
  • Edited/fixed formatting (consistent with the above) of the QUADOBJ section of the pyomo/repn/tests/mps/column_ordering_quadratic.mps.baseline and pyomo/repn/tests/mps/no_column_ordering_quadratic.mps.baseline files.

Summary/Motivation:

  • Closes MPS files - .write() - mischanging QUADOBJ and QMATRIX? #2122
  • The quadratic objective terms written in the QUADOBJ section were being incorrectly output in the format for the QMATRIX section, rather than the format for the QUADOBJ section.
  • Specifically, the QMATRIX section should include all non-zero terms of the symmetric Q matrix, so for example an X01*X02 term must be accompanied by a matching X02*X01 term.
  • While the QUADOBJ section should include only the non-zero terms of the upper (or lower) triangle of the Q matrix, so for example an X01*X02 term will NOT be accompanied by a matching X02*X01 term.
  • Note that having the QUADOBJ incorrectly output in the QMATRIX format causes the coefficients of the off-diagonal quadratic terms of the objective function to be off by a factor of 2.
  • For further documentation on the correct output for the QUADOBJ section see gurobi docs and cplex docs
  • Finally, to see that the current MPS files are being written incorrectly, create a (quadratic objective) pyomo model and solve. Then write that model to an MPS file and solve with a solver. The answers will differ, as shown in the example below.

Example to reproduce error:

# Using model created in pyomo/repn/tests/mps/test_mps.py
import random
from pyomo.environ import (
    ConcreteModel,
    Var,
    Objective,
    Constraint,
    ComponentMap,
    minimize,
    Binary,
    NonNegativeReals,
    NonNegativeIntegers,
    SolverFactory,
    TerminationCondition,
)

# generates an expression in a randomized way so that
# we can test for consistent ordering of expressions
# in the MPS file
def _gen_expression(terms):
    terms = list(terms)
    random.shuffle(terms)
    expr = 0.0
    for term in terms:
        if type(term) is tuple:
            prodterms = list(term)
            random.shuffle(prodterms)
            prodexpr = 1.0
            for x in prodterms:
                prodexpr *= x
            expr += prodexpr
        else:
            expr += term
    return expr

# Function to create model with quadratic objective 
def create_no_column_ordering_quadratic():
    model = ConcreteModel()
    model.a = Var()
    model.b = Var()
    model.c = Var()

    terms = [
        model.a,
        model.b,
        model.c,
        (model.a, model.a),
        (model.b, model.b),
        (model.c, model.c),
        (model.a, model.b),
        (model.a, model.c),
        (model.b, model.c),
    ]
    model.obj = Objective(expr=_gen_expression(terms))
    model.con = Constraint(expr=_gen_expression(terms) <= 1)
    return model

# Function to solve the MPS file output by pyomo with gurobi

def read_and_solve_with_gurobi(mps_file):
    from gurobipy import read, GRB

    # Read the MPS file
    model = read(mps_file)
    # Optimize the model
    model.optimize()
    
    # Print MPS filename
    print(mps_file)
    # Print the solution
    if model.status == GRB.OPTIMAL:
        print("Optimal solution found!")
        print(f"Objective value: {model.objVal}")
    else:
        print("No optimal solution found.")

# Create model and solve in pyomo
m = create_no_column_ordering_quadratic()
solver = SolverFactory('scip')  # You can use other solvers like 'cbc', 'gurobi', etc.
result = solver.solve(m, tee=False)
if result.solver.termination_condition == TerminationCondition.optimal:
    print("Optimal solution found:", m.obj())
else:
    print("Solver did not find an optimal solution.")
# ANS (correct): Optimal solution found: -0.3749998868443072

# write MPS file and read into Gurobi and solve
mps_file = 'pyomo/repn/tests/mps/no_column_ordering_quadratic.mps'
m.write(mps_file, io_options={'symbolic_solver_labels': True})
read_and_solve_with_gurobi(mps_file)
# ANS: Optimal solution found! Objective value: -0.24999983578928786 (incorrect for original pyomo model, because MPS file writes model incorrectly)

Changes proposed in this PR:

  • Correct the QUADOBJ section in _print_model_MPS func, such that each line only outputs a single non-zero value in the upper triangle of the symmetric Q matrix.
  • Correct QUADOBJ section of pyomo/repn/tests/mps/column_ordering_quadratic.mps.baseline and pyomo/repn/tests/mps/no_column_ordering_quadratic.mps.baseline files.

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@dansplain dansplain changed the title Fix writemps quadobj Fix how QUADOBJ section written to MPS files May 21, 2025
@dansplain
Copy link
Author

dansplain commented May 22, 2025

@mrmundt It seems the tests are failing because the *_column_ordering_quadratic.mps.baseline files themselves are formatted incorrectly as well (again, in the format for QMATRIX rather than QUADOBJ). I will convert to draft while I update the appropriate test files.

@dansplain dansplain marked this pull request as draft May 22, 2025 21:49
@dansplain dansplain marked this pull request as ready for review May 28, 2025 18:36
@dansplain
Copy link
Author

@mrmundt this is ready for review now, with updated *quadratic.mps.baseline files so the tests will pass. Thank you!

@dansplain
Copy link
Author

Hello @mrmundt and @blnicho - Please let me know if there is anything else I can help with for this PR, or if it is ready to be reviewed from your perspective. Thank you!

@blnicho
Copy link
Member

blnicho commented Jun 11, 2025

@dansplain we're working on resolving a few broken tests stemming from one of our optional dependencies. Once our testing infrastructure is passing again, we should be able to get this reviewed and merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MPS files - .write() - mischanging QUADOBJ and QMATRIX?
4 participants