Segmentation fault when two different NeighborLists are passed into one IntegrationMethodTwoStep

[bchen-00]

We have a custom IntegrationMethodTwoStep where we need to pass in two separate NeighborList objects. However, when the two NeighborList objects are independent, running the code results in segmentation fault.

More specifically, we have something like

CustomIntegrator( std::shared_ptr<SystemDefinition> sysdef,
               std::shared_ptr<ParticleGroup> group, 
               std::shared_ptr<NeighborList> nlist_1,
	       std::shared_ptr<NeighborList> nlist_2):  
               IntegrationMethodTwoStep(sysdef, group),  m_nlist_1(nlist_1), m_nlist_2(nlist_2)
{ ... }

void integrateStepOne(uint64_t timestep) 
{
    m_nlist_1->compute(timestep);
    m_nlist_2->compute(timestep); // cannot run past this line, segmentation fault
}

pybind11::class_<Custom, IntegrationMethodTwoStep, std::shared_ptr<Custom>>(
      m, "Custom")
      .def(pybind11::init<
           std::shared_ptr<SystemDefinition>, std::shared_ptr<ParticleGroup>,
           std::shared_ptr<hoomd::md::NeighborList>, std::shared_ptr<hoomd::md::NeighborList>
      >())

In Python, if I initialize with nlist_1 and nlist_2 reference the same NeighborList object

nlist1 = hoomd.md.nlist.Cell( ... );
nlist2 = nlist1;
Custom(filter = hoomd.filter.All(), nlist1, nlist2)

and run with

simulation.run(1)

Everything is OK. However, as long as nlist1 and nlist2 reference different NeighborList objects:

nlist1 = hoomd.md.nlist.Cell( ... );
nlist2 = deepcopy(nlist1);
Custom(filter = hoomd.filter.All(), nlist1, nlist2)

Running simulation.run(1) results in segmentation fault. Is there something wrong with my implementation, or something fishy is going on with the memory access inside NeighborList.compute()?

[joaander]

Please build in debug mode and share a backtrace. Even without your code, I might notice something suspicious. valgrind can also be very helpful to find the root cause of memory errors. Often times a segmentation fault in one place in the code is caused by an out of bounds memory write somewhere else.

I'm not aware of any issues with deepcopy -- but could you also try

nlist1 = hoomd.md.nlist.Cell( ... );
nlist2 = hoomd.md.nlist.Cell( ... );

to see if that helps?

The unit tests instantiate hundreds of neighbor lists when they are run without problems. valgrind checks pass without any out of bounds memory writes or reads of uninitialized memory inside HOOMD-blue source code. I have seen many users employ multiple neighbor lists in production simulations that have run for billions of timesteps.

[bchen-00]

I have tried

nlist1 = hoomd.md.nlist.Cell( ... );
nlist2 = hoomd.md.nlist.Cell( ... );

which gives the same thing.

Also, I tried a simplest test case, where I directly called compute(0) in the constructor, before any other custom actions

Stokes::Stokes(std::shared_ptr<SystemDefinition> sysdef,
               std::shared_ptr<ParticleGroup> group, 
               std::shared_ptr<Variant> T,
               std::shared_ptr<hoomd::md::NeighborList> nlist_ewald, 
                std::shared_ptr<hoomd::md::NeighborList> nlist_res,
               Scalar xi,
               Scalar error)
    : IntegrationMethodTwoStep(sysdef, group), m_T(T), m_xi(xi), m_error(error) {

  //use system seed.
  m_seed = m_sysdef->getSeed();
  nlist_ewald -> compute(0);
  nlist_res -> compute(0);
}

I still get the same segmentation fault (but goes away if I make nlist_res and nlist_ewald to reference the same object). Here is the debug backtrace

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
Stokes::Stokes (this=0x5555577954f0, sysdef=..., group=..., T=..., nlist_ewald=..., nlist_res=..., xi=0.200000003, error=0.00100000005) at /home/boyuan/afsd/hoomd-blue/hoomd/example_plugins/PSEv3_plugin/Stokes.cc:63
63        nlist_res -> compute(0);
(gdb) backtrace
#0  Stokes::Stokes (this=0x5555577954f0, sysdef=..., group=..., T=..., nlist_ewald=..., nlist_res=..., 
    xi=0.200000003, error=0.00100000005)
    at /home/boyuan/afsd/hoomd-blue/hoomd/example_plugins/PSEv3_plugin/Stokes.cc:63
#1  0x00007ffed3e8b460 in pybind11::detail::initimpl::construct_or_initialize<Stokes, std::shared_ptr<hoomd::SystemDefinition>, std::shared_ptr<hoomd::ParticleGroup>, std::shared_ptr<hoomd::Variant>, std::shared_ptr<hoomd::md::NeighborList>, std::shared_ptr<hoomd::md::NeighborList>, float, float, 0> ()
    at /home/boyuan/anaconda3/envs/afsd-test/include/pybind11/detail/init.h:76

I tried using valgrind but the output is quite lengthy and the errors are traced to malloc by UnkownInlinedFun.

I don't doubt hoomd's ability to handle many NeighborList objects, as I also use more than one in my simulations, but those are usually passed in to md.pair objects that are attached to the integrator, not directly passed into the IntegrationMethodTwoStep object itself.

[joaander]

If the backtrace ends in your Stokes.cc file then I would guess that nlist_res is a null pointer. You can confirm by printing out the pointer, either in gdb or in the code itself.

Did you configure the Python side of your code to attach the 2nd nlist? That is the point that the C++ objects are created.

[bchen-00]

Thanks for the response - you are correct that the nlist_res is passed in as a null pointer. I forgot to call nlist_res._attach(self._simulation_) in the python _attach_hook function. Thanks again!