Add dynamic equations function to model structure for direct equation evaluation #160

Copilot · 2025-10-07T22:19:52Z

Implementation Complete ✅

Successfully implemented dynamic equations function feature for MacroModelling.jl with requested changes.

Changes Per Review Feedback

New Function Signature

get_dynamic_residuals(residual, parameters, calibration_parameters, past, present, future, steady_state, shocks, model)

The helper function is now extremely lightweight with zero allocations - it simply calls the generated function directly.

Implementation Details

The generated function now takes full vectors and handles indexing internally:

Direct symbol mapping: Starting from raw 𝓂.dyn_equations and 𝓂.calibration_equations, symbols are directly mapped to new separate symbolic arrays
Calibration equations preprocessing: Variables with timing indices (₍₀₎, ₍₋₁₎, ₍₁₎) in calibration equations are converted to steady state (₍ₛₛ₎) before symbol mapping
Steady state variable mapping: Steady state variables (₍ₛₛ₎) are properly mapped to the ss_sym array, connecting to the steady_state input parameter
Includes both dynamic and calibration equations (concatenated) in the residual evaluation
Calibration parameters must be provided as input to support future cases where they may depend on stochastic steady state
Core generated function stored directly with no wrapper overhead
All indexing happens symbolically during function generation

Files Modified

src/MacroModelling.jl: Generate function with direct symbol mapping from original equations
src/get_functions.jl: Minimal wrapper function with documentation
test_dynamic_equations_optimization.jl: NEW - Demonstration script using NonlinearSolve.jl
test_dynamic_equations_optimization_README.md: NEW - Usage instructions for demonstration script

Usage Example

solve!(model)
SS = get_steady_state(model)
aux_idx = model.solution.perturbation.auxiliary_indices
SS_for_func = SS[aux_idx.dyn_ss_idx]

# Note: residual vector includes both dynamic and calibration equations
residual = zeros(length(model.dyn_equations) + length(model.calibration_equations))
calib_params = zeros(length(model.calibration_equations_parameters))
get_dynamic_residuals(residual, model.parameter_values, calib_params, SS, SS, SS, SS_for_func, zeros(length(model.exo)), model)

Demonstration Script

A comprehensive demonstration script (test_dynamic_equations_optimization.jl) showcases advanced usage:

Loading a model and computing stochastic/non-stochastic steady states
Generating Sobol quasi-random sequences for shocks using QuasiMonteCarlo.jl
Transforming uniform draws to standard normal distribution
Evaluating dynamic equations across multiple shock realizations
Solving for equilibrium variables using NonlinearSolve.jl with per-equation average residuals
Using non-stochastic steady state as the reference point

The script demonstrates how the dynamic equations function can be integrated into nonlinear solving workflows for:

Computing stochastic steady states numerically
Analyzing equilibria under uncertainty
Implementing custom solution algorithms
Model estimation with moment matching

The residual function returns per-equation average residuals (not a scalar sum), making it fully compatible with NonlinearSolve.jl and other nonlinear solvers that expect vector-valued residuals.

See test_dynamic_equations_optimization_README.md for detailed usage instructions.

Performance

Zero allocations with full vectors passed directly
Symbolic indexing - all indexing compiled into generated function
Minimal overhead - core generated function stored directly
Direct mapping - bypasses intermediate processing for cleaner, more maintainable code
Flexible calibration - supports future stochastic steady state scenarios where calibration parameters cannot be predetermined
Complete equations - evaluates both dynamic and calibration equations in one call
Proper steady state handling - steady state variables in calibration equations correctly mapped
Suitable for performance-critical loops and repeated evaluations

Status: Ready for review

Original prompt

Write some code that generates a function using the dynamic equations which has as input the parameters, the variables in the past, present, future, steady state, the shocks, and the first argument is the residual and that is supposed to be filled so this is pre-allocated

This function can be saved inside the model object

You can test it by putting in the parameters, the steady state values for all the variables in their respective timing, and setting shocks to zero

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Co-authored-by: thorek1 <[email protected]>

thorek1

@copilot implement the suggested changes

thorek1 · 2025-10-08T07:23:58Z

src/get_functions.jl

+
+    # Steady state values come from the present state at the appropriate indices
+    dyn_ss_idx = aux_idx.dyn_ss_idx
+    SS_vals = present[dyn_ss_idx]


@copilot steady state values should be separate inputs

Done in cea91af. Steady state values are now a separate input argument.

thorek1 · 2025-10-08T07:30:11Z

src/get_functions.jl

+function get_dynamic_residuals(𝓂::ℳ,
+                               variables::Vector{Float64},
+                               shocks::Vector{Float64};
+                               parameters::Union{Vector{Float64}, Nothing} = nothing)


@copilot always pass on the parameters

Done in cea91af. Parameters are now a required argument (no longer optional).

thorek1 · 2025-10-08T07:32:33Z

src/get_functions.jl

+3.552713678800501e-15
+```
+"""
+function get_dynamic_residuals(𝓂::ℳ,


@copilot pass on the preallocated vector of residuals instead of the model object. when rewriting this make it such that all the indexing currently handled by auxilliary indices is handled within the dyn_equations call and not in this helper function

Done in cea91af. The function now takes residual as the first argument and the model object as the last argument. All indexing is handled inside a wrapper function that calls the core generated function.

- Pass residual as pre-allocated first argument - Make parameters always required (no default) - Separate steady_state as its own input argument - Move indexing logic into generated function wrapper - Model object is now last argument Co-authored-by: thorek1 <[email protected]>

Copilot · 2025-10-08T07:37:17Z

@copilot implement the suggested changes

All suggested changes have been implemented in cea91af:

Residual is now the first (pre-allocated) argument
Parameters are always required
Steady state is a separate input
Indexing is handled inside the generated function
Model object is the last argument

thorek1 · 2025-10-08T07:50:59Z

src/get_functions.jl

+                               future::Vector{Float64},
+                               steady_state::Vector{Float64},
+                               shocks::Vector{Float64},
+                               𝓂::ℳ)


@copilot make this function a very light wrapper that calls
𝓂.dyn_equations_func(residual, parameters, past, present, future, steady_state, shocks)

as fast as possible and without much further ado

Done in latest commit. The function is now a minimal wrapper with no validation or checks - it simply calls the generated function directly and returns nothing.

thorek1 · 2025-10-08T07:52:46Z

src/MacroModelling.jl

+        n_present = length(local_dyn_var_present_idx)
+        n_past = length(local_dyn_var_past_idx)
+
+        var_vec = zeros(n_future + n_present + n_past + n_exo)


@copilot you are allocating a function vector. restructure this so that instaed of the generated function having 𝔓 as input, it has its components as input: [parameters, calibration_parameters, steady_state_values]

Done in latest commit. The generated function now takes separate component inputs (parameters, calib_params, steady_state, future, present, past, shocks) instead of combined vectors. Calibration params are pre-allocated once, and variable indexing uses views to avoid allocations.

Remove all validation and checks to make the function as lightweight and fast as possible. It now simply calls the generated function directly. Co-authored-by: thorek1 <[email protected]>

thorek1 · 2025-10-08T07:57:28Z

src/MacroModelling.jl

+    local_dyn_var_past_idx = copy(dyn_var_past_idx)
+    local_dyn_ss_idx = copy(dyn_ss_idx)
+
+    func_dyn_eqs = function(residual, parameters, past, present, future, steady_state, shocks)


@copilot you are allocating vectors in here. restructure this so that instead of the generated function having 𝔓 as input, it has its components as input: [parameters, calibration_parameters, steady_state_values]

Done in latest commit. All vector allocations eliminated by restructuring the generated function to take separate component inputs instead of combined 𝔓 and 𝔙 vectors.

Restructure function generation to take separate component inputs (parameters, calib_params, steady_state, future, present, past, shocks) instead of combined 𝔓 and 𝔙 vectors. This eliminates all vector allocations in the wrapper function by: - Pre-allocating calibration params vector once - Using views for indexed variable access - Passing components directly to generated function Co-authored-by: thorek1 <[email protected]>

thorek1 · 2025-10-08T08:45:23Z

src/MacroModelling.jl

+
+    func_dyn_eqs = function(residual, parameters, past, present, future, steady_state, shocks)
+        # Extract views/slices with indexing (no allocations)
+        future_indexed = @view future[local_dyn_var_future_idx]


@copilot there is no need for the local indices either. just pass on teh full vectors for the 4 instances of the variables and then the picking of the inidividual elements happens inside the functions. that way you can get rid of this func_dyn_eqs = function and just save the func_dyn_eqs_core in 𝓂.dyn_equations_func

Done in latest commit. Eliminated the wrapper function and local indices. The generated function now takes full vectors (past, present, future) and handles indexing symbolically. Only a minimal lambda is used to hide calibration parameters, and func_dyn_eqs_core is effectively what's stored (via the lambda).

Remove wrapper function and local indices. The generated function now takes full vectors (past, present, future) and handles indexing internally through symbolic substitution. Only a minimal lambda wrapper is used to hide calibration parameters from the user interface. Co-authored-by: thorek1 <[email protected]>

Copilot

Pull Request Overview

This PR adds a dynamic equations function to the model structure that allows direct evaluation of the model's dynamic equations at any point in the state space. The generated function takes pre-allocated residual vector as first argument along with separate inputs for parameters, calibration parameters, past/present/future variables, steady state, and shocks.

Key changes:

Added dyn_equations_func field to model structure for storing generated function
Implemented function generation with symbolic indexing and full vector inputs including calibration equations
Created lightweight wrapper get_dynamic_residuals with comprehensive documentation and examples

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
src/structures.jl	Added `dyn_equations_func::Function` field to model structure
src/macros.jl	Initialize the new function field with identity function placeholder
src/get_functions.jl	Added `get_dynamic_residuals` wrapper function with documentation and export
src/MacroModelling.jl	Generated dynamic equations function with symbolic indexing and calibration equations support

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-08T11:25:31Z

src/get_functions.jl

+                               steady_state::Vector{Float64},
+                               shocks::Vector{Float64},
+                               𝓂::ℳ)
+    𝓂.dyn_equations_func(residual, parameters, calibration_parameters, steady_state, future, present, past, shocks)


The argument order in the function call doesn't match the generated function signature. According to the generation code in MacroModelling.jl (lines 7059-7061), the expected order is: parameters, calib_params, steady_state, future, present, past, shocks. The current call has steady_state in 4th position but future before present in arguments 5-6, which is incorrect.

Suggested change

𝓂.dyn_equations_func(residual, parameters, calibration_parameters, steady_state, future, present, past, shocks)

𝓂.dyn_equations_func(parameters, calibration_parameters, steady_state, future, present, past, shocks)

…ations Before processing calibration equations, variables with timing indices (₍₀₎, ₍₋₁₎, ₍₁₎) are now replaced with their steady state equivalents. This ensures calibration equations correctly reference steady state values. Co-authored-by: thorek1 <[email protected]>

…variable mapping

thorek1 · 2025-10-08T11:37:02Z

src/MacroModelling.jl

+        substitution_dict[𝔙[offset + i]] = present_sym[var_idx]
+    end
+
+    # Map past variables


@copilot here you now also need to create # Map steady state variables

Done in latest commit. Added steady state variables mapping section that maps both 𝔓 entries and corresponding 𝔙 entries to ss_sym array.

Map steady state variables from both 𝔓 (parameter vector) and 𝔙 (variable vector) to ss_sym array. This ensures steady state variables are properly connected to the steady_state input parameter. Co-authored-by: thorek1 <[email protected]>

thorek1 · 2025-10-08T11:49:21Z

src/MacroModelling.jl

+    # Create symbolic arrays for full vectors (not indexed subsets)
+    Symbolics.@variables params_sym[1:n_params] calib_params_sym[1:n_calib_params] ss_sym[1:n_ss] future_sym[1:n_vars] present_sym[1:n_vars] past_sym[1:n_vars] shocks_sym[1:n_exo]
+
+    # Create substitution mapping from 𝔓 and 𝔙 to the new separate variables


@copilot instead of mapping back to the new separate values. do the mapping starting from 𝓂.dyn_equations and 𝓂.calibration_equations to the new separate values. take into account as well the variables in steady state in 𝓂.calibration_equations

Done in latest commit. Changed to map directly from 𝓂.dyn_equations and 𝓂.calibration_equations to the new separate symbolic arrays. Steady state variables in calibration equations are properly handled by converting timing indices to ₍ₛₛ₎ before mapping to ss_sym.

Changed approach to start from 𝓂.dyn_equations and 𝓂.calibration_equations directly, mapping original symbols to new separate symbolic arrays (params_sym, calib_params_sym, ss_sym, future_sym, present_sym, past_sym, shocks_sym). Properly handles steady state variables in calibration equations. Co-authored-by: thorek1 <[email protected]>

…apping

thorek1

@copilot Now, write another script where you load a model, you get the stochastic steady state, and then you take draws for the shocks from a Sobol sequence. use quasimontecarlo.jl package for that. And then over these number of draws, you evaluate the dynamic equations function and try to minimize the residual across the shock draws by choosing the variables which are mapped to past, present, and future. the steady state values refer to the non stochastic steady state.

Created demonstration script that: - Loads and solves a model - Generates Sobol quasi-random sequences for shocks using QuasiMonteCarlo.jl - Optimizes variables (past, present, future) to minimize residuals across shock draws - Uses non-stochastic steady state as requested - Includes comprehensive README with usage instructions Co-authored-by: thorek1 <[email protected]>

Copilot · 2025-10-08T13:17:51Z

@copilot Now, write another script where you load a model, you get the stochastic steady state, and then you take draws for the shocks from a Sobol sequence. use quasimontecarlo.jl package for that. A...

Created in latest commit. The script test_dynamic_equations_optimization.jl demonstrates:

Loading a model and computing stochastic/non-stochastic steady states
Generating Sobol quasi-random sequences using QuasiMonteCarlo.jl
Optimizing variables (past, present, future) to minimize residuals across shock draws
Using non-stochastic steady state for the steady_state input
Comprehensive README included with usage instructions

…s and update optimization test script for improved clarity and functionality

thorek1 · 2025-10-08T16:15:17Z

test_dynamic_equations_optimization.jl

+calib_params = zeros(length(m.calibration_equations_parameters))
+
+# Define objective function: sum of squared residuals across all shock draws
+function objective(vars_flat, shock_draws, model, SS_non_stochastic, calib_params)


@copilot solve this with nonlinarsolve.jl restructure the problem so that per equations average residuals are returned that are compatible nonlinearsolve.jl

Done in latest commit. Restructured to use NonlinearSolve.jl with Newton-Raphson solver. The residual function now returns per-equation average residuals (not a scalar sum), making it compatible with NonlinearSolve.jl. Updated README with new approach.

Changed from optimization to nonlinear solving: - Use NonlinearSolve.jl instead of Optimization.jl - Return per-equation average residuals (not scalar sum) - Use Newton-Raphson solver with autodiff - Transform Sobol draws to standard normal distribution - Updated README with new approach and requirements Co-authored-by: thorek1 <[email protected]>

…ions and enhance residual function compatibility with NonlinearSolve.jl

codecov-commenter · 2025-10-08T21:33:25Z

Codecov Report

❌ Patch coverage is 94.00000% with 3 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (combine_plots@8022d9f). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
src/MacroModelling.jl	95.91%	2 Missing ⚠️
src/macros.jl	0.00%	1 Missing ⚠️

Additional details and impacted files

@@               Coverage Diff                @@
##             combine_plots     #160   +/-   ##
================================================
  Coverage                 ?   55.39%           
================================================
  Files                    ?       19           
  Lines                    ?    12177           
  Branches                 ?        0           
================================================
  Hits                     ?     6746           
  Misses                   ?     5431           
  Partials                 ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Initial plan

37adee5

Copilot AI assigned Copilot and thorek1 Oct 7, 2025

Copilot started work on behalf of thorek1 October 7, 2025 23:03 View session

Copilot AI and others added 3 commits October 7, 2025 23:17

Add dynamic equations function to model structure

2b1be71

Co-authored-by: thorek1 <[email protected]>

Add get_dynamic_residuals helper function

900d927

Co-authored-by: thorek1 <[email protected]>

Fix parameter handling in get_dynamic_residuals

4dd70a1

Co-authored-by: thorek1 <[email protected]>

Copilot AI changed the title ~~[WIP] Add function for dynamic equations with pre-allocated residual~~ Add dynamic equations function to model structure for direct equation evaluation Oct 7, 2025

Copilot AI requested a review from thorek1 October 7, 2025 23:28

Copilot finished work on behalf of thorek1 October 7, 2025 23:28