Refactor communicator types to use C++ inheritance #983

AvivBenchorin · 2025-08-19T19:25:33Z

Currently, the communicators data types have an inheritance hierarchy where the base communicator type is inherited by base listen, receive, and send communciator types, and the specialized base comm types are inherited by the transport-specific comm types. This PR replaces the C-style inheritance of having the parent comm type as the first member of the derived comm types with C++ inheritance, and refactors all function pointers for the base listen, receive, and send communicator classes into virtual member functions that the protocol-specific communicator classes implement.

This PR only handle the direct translation of the C-style comm structs into C++ classes with inheritance, and turning function pointers into member functions. A future PR will need to:

Refactor comm class initialization and clean-up functions into constructors and destructors
Refactor remaining stand-alone functions associated with communicator classes into member functions
Enforce appropriate access control (add the protected keyword for members that shouldn't be publicly accessible)

(Continuing C++ refactor started with #851, #938, and #950, #955, and #971).

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AvivBenchorin · 2025-08-21T16:42:08Z

bot:aws:retest

AvivBenchorin · 2025-08-25T22:10:23Z

include/nccl_ofi_rdma.h

+	inline nccl_net_ofi_rdma_ep_t *rdma_send_comm_get_ep()
+	{
+		return reinterpret_cast<nccl_net_ofi_rdma_ep_t *>(ep);
+	}


Note: I have been thinking about sane ways to abstract the get_ep RDMA and SENDRECV communicator functions and the get_rail RDMA send and recv comm functions so that they can be defined once per protocol. I experimented with have another protocol-specific comm base class (e.g. nccl_net_ofi_rdma_comm_t) that just defined a generic <protocol>_comm_get_ep function that could be used across the listen/send/recv comm derived classes, but I wasn't able to get it working.

I think we were probably too cute here and shouldn't have an type in the base class. If we need to get access to the base ep somewhere, we should just have a pure virtual get_base_ep() call in the base communicator class that everyone must implement (and pass their ep).

For moving the endpoint out of the base communicator class, were you envisioning having an endpoint ptr and get_base_ep override in each of the specialized protocol-specific comms (e.g. separate nccl_net_ofi_rdma_ep_t * ep in the RDMA listen, send, and recv comms)? Or adding a new "generic" protocol-specific comm with the endpoint ptr (e.g. a nccl_net_ofi_rdma_comm_t class) that becomes a second parent for the specialized protocol-specific comm (e.g.:

class nccl_net_ofi_rdma_send_comm_t : public nccl_net_ofi_send_comm_t, public nccl_net_ofi_rdma_comm_t {

).

AvivBenchorin · 2025-08-26T18:21:32Z

src/nccl_ofi_rdma.cpp

 	pthread_wrapper domain_lock(&domain->domain_lock);

-	CHECK_DOMAIN_ACTIVE(domain, "reg_mr_send_comm");
+	CHECK_DOMAIN_ACTIVE(domain, "send regMr");


Note: to differentiate between regMr function calls by send comms and recv comms, I passed in "<send/recv> regMr" as an arg to CHECK_DOMAIN_ACTIVE (even though it doesn't exactly match the function name like before) since the arg is only used for a print.

bwbarrett · 2025-09-05T21:15:34Z

include/nccl_ofi.h

-		     uint64_t dest, uint64_t mr_key, nccl_net_ofi_req_t **req);
-	int (*write_inline)(nccl_net_ofi_send_comm_t *, void* src, size_t size,
-			    uint64_t dest, uint64_t mr_key, nccl_net_ofi_req_t **request);
+	virtual int write(void* src, size_t size, void* src_mhandle, uint64_t dest,


Why is write() different than send()? I'd just make it pure virtual like the others.

bwbarrett · 2025-09-05T21:22:58Z

include/nccl_ofi_rdma.h

+	inline nccl_net_ofi_rdma_ep_t *rdma_send_comm_get_ep()
+	{
+		return reinterpret_cast<nccl_net_ofi_rdma_ep_t *>(ep);
+	}


I think we were probably too cute here and shouldn't have an type in the base class. If we need to get access to the base ep somewhere, we should just have a pure virtual get_base_ep() call in the base communicator class that everyone must implement (and pass their ep).

src/nccl_ofi_rdma.cpp

rongbingzhou · 2025-09-11T23:27:56Z

src/nccl_ofi_rdma.cpp

 				bool recv_completion_optional)
 {
-	nccl_net_ofi_rdma_ep_t *ep = (nccl_net_ofi_rdma_ep_t *)r_comm->base.base.ep;
+	nccl_net_ofi_rdma_ep_t *ep = (nccl_net_ofi_rdma_ep_t *)r_comm->ep;


We have lots of mixed C and C++ typecastings in this PR. For example, at this line, you keep the C typecasting. However, at line 2964, you upgrade C typecasting to C++ typecasting: from (nccl_net_ofi_rdma_mr_handle_t *)mhandle to static_cast<nccl_net_ofi_rdma_mr_handle_t *>(mhandle).

Do we have a plan to upgrade all C typecastings to C++ typecastings?

Thus far, my approach has been to avoid updating the typecast and NULL --> nullptr in functions I am not generally refactoring to keep my commits more self-contained. E.g. the code here is in the request type function insert_send_ctrl_req, since I'm only updating the usage of r_comm to reflect removing the base struct member variables I would wait on the typecast update until I properly refactor the request type and turn this function into a member function.

rongbingzhou · 2025-09-11T23:32:00Z

src/nccl_ofi_rdma.cpp

 	rdma_req_flush_data_t *flush_data = NULL;
 	*ret_req = NULL;


Nit: we can replace NULL with nullptr at these two lines

See #983 (comment), since this code was in the request type function rdma_comm_alloc_flush_req that I didn't generally refactor, I would wait on updating NULL --> nullptr until I refactor the request type and turn this function into a member function.

rongbingzhou · 2025-09-11T23:44:24Z

src/nccl_ofi_rdma.cpp


-	free(l_comm);
-	ret = ep->release_ep(false, false);
+	free(this);


Should this be delete this?

Also, is it a good practice to free an object by itself in its own function? After this function is called, the object will become a dangling pointer and we need to make sure it's not used any more.

src/nccl_ofi_rdma.cpp

src/nccl_ofi_sendrecv.cpp

src/nccl_ofi_api.cpp

src/nccl_ofi_rdma.cpp

hershys-aws · 2025-09-15T23:54:46Z

include/nccl_ofi.h

+class nccl_net_ofi_comm_t {
+public:
+	virtual ~nccl_net_ofi_comm_t() = default;
+


Do we expect anyone to ever instantiate an object of this class? If not, we should make this abstract by setting the destructor to be pure virtual.

hershys-aws · 2025-09-16T00:01:07Z

include/nccl_ofi.h

 */
-struct nccl_net_ofi_listen_comm {
-	nccl_net_ofi_comm_t base;
+class nccl_net_ofi_listen_comm_t : public nccl_net_ofi_comm_t {


This might be an understanding issue on my end or spin out into a bigger discussion, but I don't quite see benefits to having several abstract classes "chained" up like this. Why not just have 1 abstract class nccl_net_ofi_comm_t which can be either. In the actual RDMA class where the implementation occurs, we could set the irrelevant functions to a no-op. This way we have a communicator "interface" and the actual implementation is left to the child classes (rdma etc). Seems cleaner, easier to understand and less code potentially. Not sure if there are other thoughts on this?

hershys-aws · 2025-09-16T00:07:50Z

include/nccl_ofi_rdma.h

-	 * this struct. This allows casting between pointers of this
-	 * struct and its base struct. */
-	nccl_net_ofi_recv_comm_t base;
+class nccl_net_ofi_rdma_recv_comm_t : public nccl_net_ofi_recv_comm_t {


Another point for discussion: do we have to stick with these lengthy names? Seems a great place for namespaces if we just call this recv_comm_t under the nccl_net_ofi::rdma namespace.

hershys-aws · 2025-09-26T20:57:23Z

include/nccl_ofi.h

-struct nccl_net_ofi_comm {
+class nccl_net_ofi_listen_comm_t {
+public:
+	virtual ~nccl_net_ofi_listen_comm_t() = default;


As this is an abstract class this should be a pure virtual destructor.

hershys-aws · 2025-09-26T23:04:03Z

include/nccl_ofi.h

+	enum nccl_net_ofi_comm_type_t type;
+	nccl_net_ofi_ep_t *ep;
+	int dev_id;


Out of curiosity, is there a reasoning these should remain public?

These should be made protected, I intended to wait on enforcing the access control until the follow-up comm refactor PR where all stand-alone functions associated with the comm classes are made members and can easily access these member variables without going through getters.

hershys-aws · 2025-09-29T16:52:14Z

include/nccl_ofi_sendrecv.h

+	/* write and write_inline operation not support for SENDRECV protocol */
+	int write(void *src, size_t size, void *src_mhandle, uint64_t dest,
+		  uint64_t mr_key, nccl_net_ofi_req_t **req) override
+	{
+		NCCL_OFI_WARN("Protocol does not support iwrite API function");
+		return -ENOTSUP;
+	}
+
+	int write_inline(void *src, size_t size, uint64_t dest, uint64_t mr_key,
+			 nccl_net_ofi_req_t **request) override
+	{
+		NCCL_OFI_WARN("Protocol does not support iwrite_inline API function");
+		return -ENOTSUP;
+	}
+
+	/* recv/flush/read operations not supported in send communicator */
+	int recv(int n, void **data, size_t *sizes, int *tags,
+		 nccl_net_ofi_mr_handle_t **mhandles, nccl_net_ofi_req_t **req) override
+	{
+		NCCL_OFI_WARN("Send comm does not support irecv API function");
+		return -EINVAL;
+	}
+
+	int flush(int n, void **data, int *sizes,
+		  nccl_net_ofi_mr_handle_t **mhandles, nccl_net_ofi_req_t **req) override
+	{
+		NCCL_OFI_WARN("Send comm does not support iflush API function");
+		return -EINVAL;
+	}
+	int read(void *dest, size_t size, void *dest_mhandle,
+		 uint64_t src, uint64_t mr_key, nccl_net_ofi_req_t **req) override
+	{
+		NCCL_OFI_WARN("Send comm does not support iread API function");
+		return -EINVAL;
+	}


Maybe have a protected function in the based class called not supported which warns + returns -EINVAL or -ENOTSUP? I figure you will probably be writing this across all the child classes a good amount.

I've added protected function log_unsupported_operation to the nccl_net_ofi_xfer_comm_t base class to implement this.

hershys-aws · 2025-09-29T16:58:04Z

src/nccl_ofi_api.cpp

 {
-	nccl_net_ofi_send_comm_t *send_comm =
-		(nccl_net_ofi_send_comm_t *)sendComm;
+	nccl_net_ofi_xfer_comm_t *send_comm =


Any reason to stick with c-style castings here?

I discuss this in #983 (comment), but thus far I've avoided updating the C --> C++ style casts in functions I am not generally updating to limit how much I'm touching code outside the scope of the datatype I'm refactoring. I would wait on refactoring these to C++-style casts until broadly refactoring the API functions.

hershys-aws · 2025-09-29T17:18:24Z

src/nccl_ofi_rdma.cpp

+	delete this;
+	ret = ep_ptr->release_ep(false, false);


IIUC calling delete on an object calls the destructors of its member variables provided they are not static....I guess I don't understand how this works and that we are sure the behavior is what we want.

Since the communicator stores a raw pointer of the endpoint, the endpoint's destructor won't get automatically called by the comm's destructor. When changing the endpoint raw pointer to directly pointing the endpoint, however, you are right that this would not longer be valid. I will make a note to track cleaning up these deletes after migrating the transport datatype raw pointers to direct objects.

hershys-aws · 2025-09-29T17:57:44Z

src/nccl_ofi_rdma.cpp

+	if (r_comm == nullptr) {
+		NCCL_OFI_WARN("Recv comm is null in post_rma_read");
+		return -EINVAL;
+	}


This or next might be a good PR to also add this as a macro and drop in places to reduce LOC of all the files.

I added macros RET_IF_NULLPTR and GOTO_IF_NULLPTR to try and reduce the line count footprint of the added nullptr checks.

Moves the MR functions (regMr, deregMr) from the base communicator types to the base domain class. Also moves the stand-alone SENDRECV protocol regMr and deregMr implementation functions to the SENDRECV protocol domain class. Signed-off-by: Aviv Benchorin <[email protected]>

Changes the listen, receive, and send communicator structs into classes, with the RDMA and SENDRECV transport communicators classes being derived from a base listen comm and a base xfer comm (for recv and send comms). Does not yet add constructor/destructor or convert function pointers to virtual functions. Signed-off-by: Aviv Benchorin <[email protected]>

Moves stand-alone member getter functions for RDMA and SENDRECV protocol into inline member functions for communicator classes, e.g. getting the communicator's endpoint member. Signed-off-by: Aviv Benchorin <[email protected]>

To protect against static analysis null pointer dereference errors when refactoring comm class function pointers into member functions, this adds null pointer checking when trying to access recv and send comm class members such as endpoint and comm rail pointers. Signed-off-by: Aviv Benchorin <[email protected]>

Refactors the base listen comm class function pointers into proper virtual member functions that are overriden by the RDMA and SENDRECV transport listen comm classes. Signed-off-by: Aviv Benchorin <[email protected]>

Refactors the base xfer comm class function pointers into proper virtual member functions that are overriden by the RDMA and SENDRECV transport protocol send and recv comm classes. Signed-off-by: Aviv Benchorin <[email protected]>

AvivBenchorin force-pushed the feature/comm_refactor branch 3 times, most recently from b709d7f to 3046067 Compare August 20, 2025 23:23

AvivBenchorin force-pushed the feature/comm_refactor branch from 3046067 to c89dc96 Compare August 26, 2025 18:20

AvivBenchorin commented Aug 26, 2025

View reviewed changes

bwbarrett reviewed Sep 5, 2025

View reviewed changes

AvivBenchorin force-pushed the feature/comm_refactor branch from c89dc96 to 110a7ec Compare September 10, 2025 22:35

AvivBenchorin marked this pull request as ready for review September 10, 2025 22:36

AvivBenchorin requested a review from a team as a code owner September 10, 2025 22:36

rongbingzhou reviewed Sep 12, 2025

View reviewed changes

AvivBenchorin force-pushed the feature/comm_refactor branch 3 times, most recently from 61aff9f to 6442737 Compare September 15, 2025 18:56

hershys-aws reviewed Sep 16, 2025

View reviewed changes

AvivBenchorin force-pushed the feature/comm_refactor branch 2 times, most recently from 60adc31 to d0d39cf Compare September 26, 2025 00:44

hershys-aws reviewed Sep 29, 2025

View reviewed changes

AvivBenchorin force-pushed the feature/comm_refactor branch 2 times, most recently from 5ff90aa to 72f2661 Compare October 1, 2025 19:55

AvivBenchorin added 6 commits October 2, 2025 00:05

refactor: move comm class inline funcs to members

b054b68

Moves stand-alone member getter functions for RDMA and SENDRECV protocol into inline member functions for communicator classes, e.g. getting the communicator's endpoint member. Signed-off-by: Aviv Benchorin <[email protected]>

refactor: turn listen comm func pointer to members

ada5a72

Refactors the base listen comm class function pointers into proper virtual member functions that are overriden by the RDMA and SENDRECV transport listen comm classes. Signed-off-by: Aviv Benchorin <[email protected]>

refactor: turn send/recv comm func ptrs to members

a3c883a

Refactors the base xfer comm class function pointers into proper virtual member functions that are overriden by the RDMA and SENDRECV transport protocol send and recv comm classes. Signed-off-by: Aviv Benchorin <[email protected]>

AvivBenchorin force-pushed the feature/comm_refactor branch from 72f2661 to a3c883a Compare October 2, 2025 00:13

hershys-aws mentioned this pull request Oct 8, 2025

Refactor communicator types to use C++ inheritance #1016

Open

Refactor communicator types to use C++ inheritance #983

Are you sure you want to change the base?

Refactor communicator types to use C++ inheritance #983

Uh oh!

Conversation

AvivBenchorin commented Aug 19, 2025

Uh oh!

AvivBenchorin commented Aug 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants