diff --git a/api/v1.15/html/Architecture.png b/api/v1.15/html/Architecture.png new file mode 100644 index 00000000000..dd09d211c69 Binary files /dev/null and b/api/v1.15/html/Architecture.png differ diff --git a/api/v1.15/html/UCX_Logo_80x80.png b/api/v1.15/html/UCX_Logo_80x80.png new file mode 100644 index 00000000000..1a60fd766dd Binary files /dev/null and b/api/v1.15/html/UCX_Logo_80x80.png differ diff --git a/api/v1.15/html/annotated.html b/api/v1.15/html/annotated.html new file mode 100644 index 00000000000..09e2f5b5c74 --- /dev/null +++ b/api/v1.15/html/annotated.html @@ -0,0 +1,208 @@ + + +
+ + + ++ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
ucp_am_handler_param | Active Message handler parameters passed to ucp_worker_set_am_recv_handler routine |
ucp_am_recv_param | Operation parameters provided in ucp_am_recv_callback_t callback |
ucp_conn_request_attr | UCP listener's connection request attributes |
ucp_context_attr | Context attributes |
ucp_datatype_attr | UCP datatype attributes |
ucp_dt_iov | Structure for scatter-gather I/O |
ucp_ep_attr | UCP endpoint attributes |
ucp_ep_evaluate_perf_attr_t | UCP endpoint performance evaluation result attributes |
ucp_ep_evaluate_perf_param_t | UCP endpoint performance evaluation request attributes |
ucp_ep_params | Tuning parameters for the UCP endpoint |
ucp_err_handler | UCP endpoint error handling context |
ucp_generic_dt_ops | UCP generic data type descriptor |
ucp_lib_attr | Lib attributes |
ucp_listener_accept_handler | |
ucp_listener_attr | UCP listener attributes |
ucp_listener_conn_handler | UCP callback to handle the connection request in a client-server connection establishment flow |
ucp_listener_params | Parameters for a UCP listener object |
ucp_mem_advise_params | Tuning parameters for the UCP memory advice |
ucp_mem_attr | Attributes of the UCP Memory handle, filled by ucp_mem_query function |
ucp_mem_map_params | Tuning parameters for the UCP memory mapping |
ucp_memh_buffer_release_params | Memory handle release parameters passed to ucp_memh_buffer_release |
ucp_memh_pack_params | Memory handle pack parameters passed to ucp_memh_pack |
ucp_params | Tuning parameters for UCP library |
ucp_request_attr_t | Attributes of a particular request |
ucp_request_param_t | Operation parameters passed to ucp_tag_send_nbx, ucp_tag_send_sync_nbx, ucp_tag_recv_nbx, ucp_put_nbx, ucp_get_nbx, ucp_am_send_nbx and ucp_am_recv_data_nbx |
ucp_request_param_t.cb | |
ucp_request_param_t.recv_info | |
ucp_stream_poll_ep | Output parameter of ucp_stream_worker_poll function |
ucp_tag_recv_info | UCP receive information descriptor |
ucp_transport_entry_t | The ucp_transports_t and ucp_transport_entry_t structures are used when ucp_ep_query is called to return an array of transport name and device name pairs that are used by an active endpoint |
ucp_transports_t | Structure containing an array of transport layers and device names used by an endpoint |
ucp_worker_address_attr | UCP worker address attributes |
ucp_worker_attr | UCP worker attributes |
ucp_worker_params | Tuning parameters for the UCP worker |
ucs_callbackq | |
ucs_callbackq_elem | |
ucs_config_allow_list_t | |
ucs_log_component_config | |
ucs_sock_addr | |
uct_allocated_memory | Describes a memory allocated by UCT |
uct_cm_attr | Connection manager attributes, capabilities and limitations |
uct_cm_ep_client_connect_args | Arguments to the client's connect callback |
uct_cm_ep_priv_data_pack_args | Arguments to the client-server private data pack callback |
uct_cm_ep_resolve_args | Arguments to the client-server resolved callback |
uct_cm_ep_server_conn_notify_args | Arguments to the server's notify callback |
uct_cm_listener_conn_request_args | Arguments to the listener's connection request callback |
uct_cm_remote_data | Data received from the remote peer |
uct_completion | Completion handle |
uct_component_attr | UCT component attributes |
uct_ep_connect_params | Parameters for connecting a UCT endpoint by uct_ep_connect |
uct_ep_params | Parameters for creating a UCT endpoint by uct_ep_create |
uct_iface_attr | Interface attributes: capabilities and limitations |
uct_iface_attr.cap | |
uct_iface_attr.cap.am | |
uct_iface_attr.cap.atomic32 | |
uct_iface_attr.cap.atomic64 | |
uct_iface_attr.cap.get | |
uct_iface_attr.cap.put | |
uct_iface_attr.cap.tag | |
uct_iface_attr.cap.tag.eager | |
uct_iface_attr.cap.tag.recv | |
uct_iface_attr.cap.tag.rndv | |
uct_iface_params | Parameters used for interface creation |
uct_iface_params.mode | |
uct_iface_params.mode.device | |
uct_iface_params.mode.sockaddr | |
uct_iov | Structure for scatter-gather I/O |
uct_listener_attr | UCT listener attributes, capabilities and limitations |
uct_listener_params | Parameters for creating a listener object uct_listener_h by uct_listener_create |
uct_md_attr | Memory domain attributes |
uct_md_attr.cap | |
uct_md_mem_attr | Memory domain attributes |
uct_md_resource_desc | Memory domain resource descriptor |
uct_mem_alloc_params_t | Parameters for allocating memory using uct_mem_alloc |
uct_mem_alloc_params_t.mds | |
uct_pending_req | Pending request |
uct_ppn_bandwidth | |
uct_rkey_bundle | Remote key with its type |
uct_tag_context | Posted tag context |
uct_tl_resource_desc | Communication resource descriptor |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Modules | |
UCP Application Context | |
UCP Worker | |
UCP Memory routines | |
UCP Wake-up routines | |
UCP Endpoint | |
UCP Communication routines | |
UCP Configuration | |
UCP Data type routines | |
This section describes UCP API.
++ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | ucp_request_attr_t |
Attributes of a particular request. More... | |
struct | ucp_err_handler |
UCP endpoint error handling context. More... | |
+Typedefs | |
typedef uint64_t | ucp_tag_t |
UCP Tag Identifier. More... | |
typedef struct ucp_recv_desc * | ucp_tag_message_h |
UCP Message descriptor. More... | |
typedef uint64_t | ucp_datatype_t |
UCP Datatype Identifier. More... | |
typedef void(* | ucp_send_callback_t )(void *request, ucs_status_t status) |
Completion callback for non-blocking sends. More... | |
typedef void(* | ucp_send_nbx_callback_t )(void *request, ucs_status_t status, void *user_data) |
Completion callback for non-blocking sends. More... | |
typedef void(* | ucp_err_handler_cb_t )(void *arg, ucp_ep_h ep, ucs_status_t status) |
Callback to process peer failure. More... | |
typedef struct ucp_err_handler | ucp_err_handler_t |
UCP endpoint error handling context. More... | |
typedef void(* | ucp_stream_recv_callback_t )(void *request, ucs_status_t status, size_t length) |
Completion callback for non-blocking stream oriented receives. More... | |
typedef void(* | ucp_stream_recv_nbx_callback_t )(void *request, ucs_status_t status, size_t length, void *user_data) |
Completion callback for non-blocking stream receives ucp_stream_recv_nbx call. More... | |
typedef void(* | ucp_tag_recv_callback_t )(void *request, ucs_status_t status, ucp_tag_recv_info_t *info) |
Completion callback for non-blocking tag receives. More... | |
typedef void(* | ucp_tag_recv_nbx_callback_t )(void *request, ucs_status_t status, const ucp_tag_recv_info_t *tag_info, void *user_data) |
Completion callback for non-blocking tag receives ucp_tag_recv_nbx call. More... | |
typedef void(* | ucp_am_recv_data_nbx_callback_t )(void *request, ucs_status_t status, size_t length, void *user_data) |
Completion callback for non-blocking Active Message receives. More... | |
+Enumerations | |
enum | ucp_atomic_op_t { + UCP_ATOMIC_OP_ADD, +UCP_ATOMIC_OP_SWAP, +UCP_ATOMIC_OP_CSWAP, +UCP_ATOMIC_OP_AND, + + UCP_ATOMIC_OP_OR, +UCP_ATOMIC_OP_XOR, +UCP_ATOMIC_OP_LAST + + } |
Atomic operation requested for ucp_atomic_op_nbx. More... | |
enum | ucp_stream_recv_flags_t { UCP_STREAM_RECV_FLAG_WAITALL = UCS_BIT(0) + } |
Flags to define behavior of ucp_stream_recv_nb function. More... | |
enum | ucp_op_attr_t { + UCP_OP_ATTR_FIELD_REQUEST = UCS_BIT(0), +UCP_OP_ATTR_FIELD_CALLBACK = UCS_BIT(1), +UCP_OP_ATTR_FIELD_USER_DATA = UCS_BIT(2), +UCP_OP_ATTR_FIELD_DATATYPE = UCS_BIT(3), + + UCP_OP_ATTR_FIELD_FLAGS = UCS_BIT(4), +UCP_OP_ATTR_FIELD_REPLY_BUFFER = UCS_BIT(5), +UCP_OP_ATTR_FIELD_MEMORY_TYPE = UCS_BIT(6), +UCP_OP_ATTR_FIELD_RECV_INFO = UCS_BIT(7), + + UCP_OP_ATTR_FIELD_MEMH = UCS_BIT(8), +UCP_OP_ATTR_FLAG_NO_IMM_CMPL = UCS_BIT(16), +UCP_OP_ATTR_FLAG_FAST_CMPL = UCS_BIT(17), +UCP_OP_ATTR_FLAG_FORCE_IMM_CMPL = UCS_BIT(18), + + UCP_OP_ATTR_FLAG_MULTI_SEND = UCS_BIT(19) + + } |
UCP operation fields and flags. More... | |
enum | ucp_req_attr_field { UCP_REQUEST_ATTR_FIELD_INFO_STRING = UCS_BIT(0), +UCP_REQUEST_ATTR_FIELD_INFO_STRING_SIZE = UCS_BIT(1), +UCP_REQUEST_ATTR_FIELD_STATUS = UCS_BIT(2), +UCP_REQUEST_ATTR_FIELD_MEM_TYPE = UCS_BIT(3) + } |
UCP request query attributes. More... | |
enum | ucp_am_recv_attr_t { UCP_AM_RECV_ATTR_FIELD_REPLY_EP = UCS_BIT(0), +UCP_AM_RECV_ATTR_FLAG_DATA = UCS_BIT(16), +UCP_AM_RECV_ATTR_FLAG_RNDV = UCS_BIT(17) + } |
UCP AM receive data parameter fields and flags. More... | |
enum | ucp_am_handler_param_field { UCP_AM_HANDLER_PARAM_FIELD_ID = UCS_BIT(0), +UCP_AM_HANDLER_PARAM_FIELD_FLAGS = UCS_BIT(1), +UCP_AM_HANDLER_PARAM_FIELD_CB = UCS_BIT(2), +UCP_AM_HANDLER_PARAM_FIELD_ARG = UCS_BIT(3) + } |
UCP AM receive data parameters fields and flags. More... | |
enum | ucp_atomic_post_op_t { + UCP_ATOMIC_POST_OP_ADD, +UCP_ATOMIC_POST_OP_AND, +UCP_ATOMIC_POST_OP_OR, +UCP_ATOMIC_POST_OP_XOR, + + UCP_ATOMIC_POST_OP_LAST + + } |
Atomic operation requested for ucp_atomic_post. More... | |
enum | ucp_atomic_fetch_op_t { + UCP_ATOMIC_FETCH_OP_FADD, +UCP_ATOMIC_FETCH_OP_SWAP, +UCP_ATOMIC_FETCH_OP_CSWAP, +UCP_ATOMIC_FETCH_OP_FAND, + + UCP_ATOMIC_FETCH_OP_FOR, +UCP_ATOMIC_FETCH_OP_FXOR, +UCP_ATOMIC_FETCH_OP_LAST + + } |
Atomic operation requested for ucp_atomic_fetch. More... | |
+Functions | |
ucs_status_t | ucp_request_query (void *request, ucp_request_attr_t *attr) |
Get information about ucp_request. More... | |
ucs_status_ptr_t | ucp_am_send_nbx (ucp_ep_h ep, unsigned id, const void *header, size_t header_length, const void *buffer, size_t count, const ucp_request_param_t *param) |
Send Active Message. More... | |
ucs_status_ptr_t | ucp_am_recv_data_nbx (ucp_worker_h worker, void *data_desc, void *buffer, size_t count, const ucp_request_param_t *param) |
Receive Active Message as defined by provided data descriptor. More... | |
void | ucp_am_data_release (ucp_worker_h worker, void *data) |
Releases Active Message data. More... | |
ucs_status_ptr_t | ucp_stream_send_nbx (ucp_ep_h ep, const void *buffer, size_t count, const ucp_request_param_t *param) |
Non-blocking stream send operation. More... | |
ucs_status_ptr_t | ucp_tag_send_nbx (ucp_ep_h ep, const void *buffer, size_t count, ucp_tag_t tag, const ucp_request_param_t *param) |
Non-blocking tagged-send operation. More... | |
ucs_status_ptr_t | ucp_tag_send_sync_nbx (ucp_ep_h ep, const void *buffer, size_t count, ucp_tag_t tag, const ucp_request_param_t *param) |
Non-blocking synchronous tagged-send operation. More... | |
ucs_status_ptr_t | ucp_stream_recv_nbx (ucp_ep_h ep, void *buffer, size_t count, size_t *length, const ucp_request_param_t *param) |
Non-blocking stream receive operation of structured data into a user-supplied buffer. More... | |
ucs_status_ptr_t | ucp_stream_recv_data_nb (ucp_ep_h ep, size_t *length) |
Non-blocking stream receive operation of unstructured data into a UCP-supplied buffer. More... | |
ucs_status_ptr_t | ucp_tag_recv_nbx (ucp_worker_h worker, void *buffer, size_t count, ucp_tag_t tag, ucp_tag_t tag_mask, const ucp_request_param_t *param) |
Non-blocking tagged-receive operation. More... | |
ucp_tag_message_h | ucp_tag_probe_nb (ucp_worker_h worker, ucp_tag_t tag, ucp_tag_t tag_mask, int remove, ucp_tag_recv_info_t *info) |
Non-blocking probe and return a message. More... | |
ucs_status_ptr_t | ucp_tag_msg_recv_nbx (ucp_worker_h worker, void *buffer, size_t count, ucp_tag_message_h message, const ucp_request_param_t *param) |
Non-blocking receive operation for a probed message. More... | |
ucs_status_ptr_t | ucp_put_nbx (ucp_ep_h ep, const void *buffer, size_t count, uint64_t remote_addr, ucp_rkey_h rkey, const ucp_request_param_t *param) |
Non-blocking remote memory put operation. More... | |
ucs_status_ptr_t | ucp_get_nbx (ucp_ep_h ep, void *buffer, size_t count, uint64_t remote_addr, ucp_rkey_h rkey, const ucp_request_param_t *param) |
Non-blocking remote memory get operation. More... | |
ucs_status_ptr_t | ucp_atomic_op_nbx (ucp_ep_h ep, ucp_atomic_op_t opcode, const void *buffer, size_t count, uint64_t remote_addr, ucp_rkey_h rkey, const ucp_request_param_t *param) |
Post an atomic memory operation. More... | |
ucs_status_t | ucp_request_check_status (void *request) |
Check the status of non-blocking request. More... | |
ucs_status_t | ucp_tag_recv_request_test (void *request, ucp_tag_recv_info_t *info) |
Check the status and currently available state of non-blocking request returned from ucp_tag_recv_nb routine. More... | |
ucs_status_t | ucp_stream_recv_request_test (void *request, size_t *length_p) |
Check the status and currently available state of non-blocking request returned from ucp_stream_recv_nb routine. More... | |
void | ucp_request_cancel (ucp_worker_h worker, void *request) |
Cancel an outstanding communications request. More... | |
void | ucp_stream_data_release (ucp_ep_h ep, void *data) |
Release UCP data buffer returned by ucp_stream_recv_data_nb. More... | |
void | ucp_request_free (void *request) |
Release a communications request. More... | |
void * | ucp_request_alloc (ucp_worker_h worker) |
Create an empty communications request. More... | |
int | ucp_request_is_completed (void *request) |
ucs_status_t | ucp_put (ucp_ep_h ep, const void *buffer, size_t length, uint64_t remote_addr, ucp_rkey_h rkey) |
Blocking remote memory put operation. More... | |
ucs_status_t | ucp_get (ucp_ep_h ep, void *buffer, size_t length, uint64_t remote_addr, ucp_rkey_h rkey) |
Blocking remote memory get operation. More... | |
ucs_status_t | ucp_atomic_add32 (ucp_ep_h ep, uint32_t add, uint64_t remote_addr, ucp_rkey_h rkey) |
Blocking atomic add operation for 32 bit integers. More... | |
ucs_status_t | ucp_atomic_add64 (ucp_ep_h ep, uint64_t add, uint64_t remote_addr, ucp_rkey_h rkey) |
Blocking atomic add operation for 64 bit integers. More... | |
ucs_status_t | ucp_atomic_fadd32 (ucp_ep_h ep, uint32_t add, uint64_t remote_addr, ucp_rkey_h rkey, uint32_t *result) |
Blocking atomic fetch and add operation for 32 bit integers. More... | |
ucs_status_t | ucp_atomic_fadd64 (ucp_ep_h ep, uint64_t add, uint64_t remote_addr, ucp_rkey_h rkey, uint64_t *result) |
Blocking atomic fetch and add operation for 64 bit integers. More... | |
ucs_status_t | ucp_atomic_swap32 (ucp_ep_h ep, uint32_t swap, uint64_t remote_addr, ucp_rkey_h rkey, uint32_t *result) |
Blocking atomic swap operation for 32 bit values. More... | |
ucs_status_t | ucp_atomic_swap64 (ucp_ep_h ep, uint64_t swap, uint64_t remote_addr, ucp_rkey_h rkey, uint64_t *result) |
Blocking atomic swap operation for 64 bit values. More... | |
ucs_status_t | ucp_atomic_cswap32 (ucp_ep_h ep, uint32_t compare, uint32_t swap, uint64_t remote_addr, ucp_rkey_h rkey, uint32_t *result) |
Blocking atomic conditional swap (cswap) operation for 32 bit values. More... | |
ucs_status_t | ucp_atomic_cswap64 (ucp_ep_h ep, uint64_t compare, uint64_t swap, uint64_t remote_addr, ucp_rkey_h rkey, uint64_t *result) |
Blocking atomic conditional swap (cswap) operation for 64 bit values. More... | |
ucs_status_ptr_t | ucp_am_send_nb (ucp_ep_h ep, uint16_t id, const void *buffer, size_t count, ucp_datatype_t datatype, ucp_send_callback_t cb, unsigned flags) |
Send Active Message. More... | |
ucs_status_ptr_t | ucp_stream_send_nb (ucp_ep_h ep, const void *buffer, size_t count, ucp_datatype_t datatype, ucp_send_callback_t cb, unsigned flags) |
Non-blocking stream send operation. More... | |
ucs_status_ptr_t | ucp_stream_recv_nb (ucp_ep_h ep, void *buffer, size_t count, ucp_datatype_t datatype, ucp_stream_recv_callback_t cb, size_t *length, unsigned flags) |
Non-blocking stream receive operation of structured data into a user-supplied buffer. More... | |
ucs_status_ptr_t | ucp_tag_send_nb (ucp_ep_h ep, const void *buffer, size_t count, ucp_datatype_t datatype, ucp_tag_t tag, ucp_send_callback_t cb) |
Non-blocking tagged-send operations. More... | |
ucs_status_t | ucp_tag_send_nbr (ucp_ep_h ep, const void *buffer, size_t count, ucp_datatype_t datatype, ucp_tag_t tag, void *req) |
Non-blocking tagged-send operations with user provided request. More... | |
ucs_status_ptr_t | ucp_tag_send_sync_nb (ucp_ep_h ep, const void *buffer, size_t count, ucp_datatype_t datatype, ucp_tag_t tag, ucp_send_callback_t cb) |
Non-blocking synchronous tagged-send operation. More... | |
ucs_status_ptr_t | ucp_tag_recv_nb (ucp_worker_h worker, void *buffer, size_t count, ucp_datatype_t datatype, ucp_tag_t tag, ucp_tag_t tag_mask, ucp_tag_recv_callback_t cb) |
Non-blocking tagged-receive operation. More... | |
ucs_status_t | ucp_tag_recv_nbr (ucp_worker_h worker, void *buffer, size_t count, ucp_datatype_t datatype, ucp_tag_t tag, ucp_tag_t tag_mask, void *req) |
Non-blocking tagged-receive operation. More... | |
ucs_status_ptr_t | ucp_tag_msg_recv_nb (ucp_worker_h worker, void *buffer, size_t count, ucp_datatype_t datatype, ucp_tag_message_h message, ucp_tag_recv_callback_t cb) |
Non-blocking receive operation for a probed message. More... | |
ucs_status_t | ucp_put_nbi (ucp_ep_h ep, const void *buffer, size_t length, uint64_t remote_addr, ucp_rkey_h rkey) |
Non-blocking implicit remote memory put operation. More... | |
ucs_status_ptr_t | ucp_put_nb (ucp_ep_h ep, const void *buffer, size_t length, uint64_t remote_addr, ucp_rkey_h rkey, ucp_send_callback_t cb) |
Non-blocking remote memory put operation. More... | |
ucs_status_t | ucp_get_nbi (ucp_ep_h ep, void *buffer, size_t length, uint64_t remote_addr, ucp_rkey_h rkey) |
Non-blocking implicit remote memory get operation. More... | |
ucs_status_ptr_t | ucp_get_nb (ucp_ep_h ep, void *buffer, size_t length, uint64_t remote_addr, ucp_rkey_h rkey, ucp_send_callback_t cb) |
Non-blocking remote memory get operation. More... | |
ucs_status_t | ucp_atomic_post (ucp_ep_h ep, ucp_atomic_post_op_t opcode, uint64_t value, size_t op_size, uint64_t remote_addr, ucp_rkey_h rkey) |
Post an atomic memory operation. More... | |
ucs_status_ptr_t | ucp_atomic_fetch_nb (ucp_ep_h ep, ucp_atomic_fetch_op_t opcode, uint64_t value, void *result, size_t op_size, uint64_t remote_addr, ucp_rkey_h rkey, ucp_send_callback_t cb) |
Post an atomic fetch operation. More... | |
UCP Communication routines
+struct ucp_request_attr_t | +
Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_req_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+char * | ++debug_string | +
+ Pointer to allocated string of size debug_string_size that will be filled with debug information about transports and protocols that were selected to complete the request. + |
+size_t | ++debug_string_size | +
+ Size of the debug_string. String will be filled up to this size. Maximum possible size debug string can be obtained by querying the worker via ucp_worker_query. + |
+ucs_status_t | ++status | +
+ Status of the request. The same as ucp_request_check_status. + |
+ucs_memory_type_t | ++mem_type | +
+ Detected memory type of the buffer passed to the operation. + |
struct ucp_err_handler | +
This structure should be initialized in ucp_ep_params_t to handle peer failure
+Data Fields | ||
---|---|---|
+ucp_err_handler_cb_t | ++cb | +
+ Error handler callback, if NULL, will not be called. + |
+void * | ++arg | +
+ User defined argument associated with an endpoint, it will be overridden by ucp_ep_params_t::user_data if both are set. + |
typedef uint64_t ucp_tag_t | +
UCP tag identifier is a 64bit object used for message identification. UCP tag send and receive operations use the object for an implementation tag matching semantics (derivative of MPI tag matching semantics).
+ +typedef struct ucp_recv_desc* ucp_tag_message_h | +
UCP Message descriptor is an opaque handle for a message returned by ucp_tag_probe_nb. This handle can be passed to ucp_tag_msg_recv_nb in order to receive the message data to a specific buffer.
+ +typedef uint64_t ucp_datatype_t | +
UCP datatype identifier is a 64bit object used for datatype identification. Predefined UCP identifiers are defined by ucp_dt_type.
+ +typedef void(* ucp_send_callback_t)(void *request, ucs_status_t status) | +
This callback routine is invoked whenever the send operation is completed. It is important to note that the call-back is only invoked in a case when the operation cannot be completed in place.
+[in] | request | The completed send request. |
[in] | status | Completion status. If the send operation was completed successfully UCS_OK is returned. If send operation was canceled UCS_ERR_CANCELED is returned. Otherwise, an error status is returned. |
typedef void(* ucp_send_nbx_callback_t)(void *request, ucs_status_t status, void *user_data) | +
This callback routine is invoked whenever the ucp_tag_send_nbx, ucp_am_send_nbx, ucp_stream_send_nbx, ucp_put_nbx, ucp_get_nbx, ucp_atomic_op_nbx or any other "send operation" is completed.
+[in] | request | The completed send request. |
[in] | status | Completion status. If the send operation was completed successfully UCS_OK is returned. If send operation was canceled UCS_ERR_CANCELED is returned. Otherwise, an error status is returned. |
[in] | user_data | User data passed to "user_data" value, see ucp_request_param_t |
typedef void(* ucp_err_handler_cb_t)(void *arg, ucp_ep_h ep, ucs_status_t status) | +
This callback routine is invoked when transport level error detected.
+[in] | arg | User argument to be passed to the callback. |
[in] | ep | Endpoint to handle transport level error. Upon return from the callback, this ep is no longer usable and all subsequent operations on this ep will fail with the error code passed in status. |
[in] | status | error status. |
typedef struct ucp_err_handler ucp_err_handler_t | +
This structure should be initialized in ucp_ep_params_t to handle peer failure
+ +typedef void(* ucp_stream_recv_callback_t)(void *request, ucs_status_t status, size_t length) | +
This callback routine is invoked whenever the receive operation is completed and the data is ready in the receive buffer.
+[in] | request | The completed receive request. |
[in] | status | Completion status. If the send operation was completed successfully UCS_OK is returned. Otherwise, an error status is returned. |
[in] | length | The size of the received data in bytes, always boundary of base datatype size. The value is valid only if the status is UCS_OK. |
typedef void(* ucp_stream_recv_nbx_callback_t)(void *request, ucs_status_t status, size_t length, void *user_data) | +
This callback routine is invoked whenever the receive operation is completed and the data is ready in the receive buffer.
+[in] | request | The completed receive request. |
[in] | status | Completion status. If the send operation was completed successfully UCS_OK is returned. Otherwise, an error status is returned. |
[in] | length | The size of the received data in bytes, always on the boundary of base datatype size. The value is valid only if the status is UCS_OK. |
[in] | user_data | User data passed to "user_data" value, see ucp_request_param_t. |
typedef void(* ucp_tag_recv_callback_t)(void *request, ucs_status_t status, ucp_tag_recv_info_t *info) | +
This callback routine is invoked whenever the receive operation is completed and the data is ready in the receive buffer.
+[in] | request | The completed receive request. |
[in] | status | Completion status. If the send operation was completed successfully UCS_OK is returned. If send operation was canceled UCS_ERR_CANCELED is returned. If the data can not fit into the receive buffer the UCS_ERR_MESSAGE_TRUNCATED error code is returned. Otherwise, an error status is returned. |
[in] | info | Completion information The info descriptor is Valid only if the status is UCS_OK. |
typedef void(* ucp_tag_recv_nbx_callback_t)(void *request, ucs_status_t status, const ucp_tag_recv_info_t *tag_info, void *user_data) | +
This callback routine is invoked whenever the receive operation is completed and the data is ready in the receive buffer.
+[in] | request | The completed receive request. |
[in] | status | Completion status. If the receive operation was completed successfully UCS_OK is returned. If send operation was canceled, UCS_ERR_CANCELED is returned. If the data can not fit into the receive buffer the UCS_ERR_MESSAGE_TRUNCATED error code is returned. Otherwise, an error status is returned. |
[in] | tag_info | Completion information The info descriptor is Valid only if the status is UCS_OK. |
[in] | user_data | User data passed to "user_data" value, see ucp_request_param_t |
typedef void(* ucp_am_recv_data_nbx_callback_t)(void *request, ucs_status_t status, size_t length, void *user_data) | +
This callback routine is invoked whenever the receive operation is completed and the data is ready in the receive buffer.
+[in] | request | The completed receive request. |
[in] | status | Completion status. If the receive operation was completed successfully UCS_OK is returned. Otherwise, an error status is returned. |
[in] | length | The size of the received data in bytes, always boundary of base datatype size. The value is valid only if the status is UCS_OK. |
[in] | user_data | User data passed to "user_data" value, see ucp_request_param_t |
enum ucp_atomic_op_t | +
This enumeration defines which atomic memory operation should be performed by the ucp_atomic_op_nbx routine.
+Enumerator | |
---|---|
UCP_ATOMIC_OP_ADD |
+ Atomic add + |
UCP_ATOMIC_OP_SWAP |
+ Atomic swap + |
UCP_ATOMIC_OP_CSWAP |
+ Atomic conditional swap + |
UCP_ATOMIC_OP_AND |
+ Atomic and + |
UCP_ATOMIC_OP_OR |
+ Atomic or + |
UCP_ATOMIC_OP_XOR |
+ Atomic xor + |
UCP_ATOMIC_OP_LAST | + |
enum ucp_stream_recv_flags_t | +
This enumeration defines behavior of ucp_stream_recv_nb function.
+Enumerator | |
---|---|
UCP_STREAM_RECV_FLAG_WAITALL |
+ This flag requests that the operation will not be completed until all requested data is received and placed in the user buffer. + |
enum ucp_op_attr_t | +
The enumeration allows specifying which fields in ucp_request_param_t are present and operation flags are used. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_OP_ATTR_FIELD_REQUEST |
+ request field + |
UCP_OP_ATTR_FIELD_CALLBACK |
+ cb field + |
UCP_OP_ATTR_FIELD_USER_DATA |
+ user_data field + |
UCP_OP_ATTR_FIELD_DATATYPE |
+ datatype field + |
UCP_OP_ATTR_FIELD_FLAGS |
+ operation-specific flags + |
UCP_OP_ATTR_FIELD_REPLY_BUFFER |
+ reply_buffer field + |
UCP_OP_ATTR_FIELD_MEMORY_TYPE |
+ memory type field + |
UCP_OP_ATTR_FIELD_RECV_INFO |
+ recv_info field + |
UCP_OP_ATTR_FIELD_MEMH |
+ memory handle field + |
UCP_OP_ATTR_FLAG_NO_IMM_CMPL |
+ Deny immediate completion, i.e NULL cannot be returned. If a completion callback is provided, it can be called before the function returns. + |
UCP_OP_ATTR_FLAG_FAST_CMPL |
+ expedite local completion, even if it delays remote data delivery. Note for implementer: this option can disable zero copy and/or rendezvous protocols which require synchronization with the remote peer before releasing the local send buffer + |
UCP_OP_ATTR_FLAG_FORCE_IMM_CMPL |
+ force immediate complete operation, fail if the operation cannot be completed immediately + |
UCP_OP_ATTR_FLAG_MULTI_SEND |
+ optimize for bandwidth of multiple in-flight operations, rather than for the latency of a single operation. This flag and UCP_OP_ATTR_FLAG_FAST_CMPL are mutually exclusive. + |
enum ucp_req_attr_field | +
The enumeration allows specifying which fields in ucp_request_attr_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_REQUEST_ATTR_FIELD_INFO_STRING | + |
UCP_REQUEST_ATTR_FIELD_INFO_STRING_SIZE | + |
UCP_REQUEST_ATTR_FIELD_STATUS | + |
UCP_REQUEST_ATTR_FIELD_MEM_TYPE | + |
enum ucp_am_recv_attr_t | +
The enumeration allows specifying which fields in ucp_am_recv_param_t are present and receive operation flags are used. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_AM_RECV_ATTR_FIELD_REPLY_EP |
+ reply_ep field + |
UCP_AM_RECV_ATTR_FLAG_DATA |
+ Indicates that the data provided in ucp_am_recv_callback_t callback can be held by the user. If UCS_INPROGRESS is returned from the callback, the data parameter will persist and the user has to call ucp_am_data_release when data is no longer needed. This flag is mutually exclusive with UCP_AM_RECV_ATTR_FLAG_RNDV. + |
UCP_AM_RECV_ATTR_FLAG_RNDV |
+ Indicates that the arriving data was sent using rendezvous protocol. In this case data parameter of the ucp_am_recv_callback_t points to the internal UCP descriptor, which can be used for obtaining the actual data by calling ucp_am_recv_data_nbx routine. This flag is mutually exclusive with UCP_AM_RECV_ATTR_FLAG_DATA. + |
enum ucp_am_handler_param_field | +
The enumeration allows specifying which fields in ucp_am_handler_param_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_AM_HANDLER_PARAM_FIELD_ID |
+ Indicates that ucp_am_handler_param_t::id field is valid. + |
UCP_AM_HANDLER_PARAM_FIELD_FLAGS |
+ Indicates that ucp_am_handler_param_t::flags field is valid. + |
UCP_AM_HANDLER_PARAM_FIELD_CB |
+ Indicates that ucp_am_handler_param_t::cb field is valid. + |
UCP_AM_HANDLER_PARAM_FIELD_ARG |
+ Indicates that ucp_am_handler_param_t::arg field is valid. + |
enum ucp_atomic_post_op_t | +
This enumeration defines which atomic memory operation should be performed by the ucp_atomic_post family of functions. All of these are non-fetching atomics and will not result in a request handle.
+Enumerator | |
---|---|
UCP_ATOMIC_POST_OP_ADD |
+ Atomic add + |
UCP_ATOMIC_POST_OP_AND |
+ Atomic and + |
UCP_ATOMIC_POST_OP_OR |
+ Atomic or + |
UCP_ATOMIC_POST_OP_XOR |
+ Atomic xor + |
UCP_ATOMIC_POST_OP_LAST | + |
enum ucp_atomic_fetch_op_t | +
This enumeration defines which atomic memory operation should be performed by the ucp_atomic_fetch family of functions. All of these functions will fetch data from the remote node.
+Enumerator | |
---|---|
UCP_ATOMIC_FETCH_OP_FADD |
+ Atomic Fetch and add + |
UCP_ATOMIC_FETCH_OP_SWAP |
+ Atomic swap + |
UCP_ATOMIC_FETCH_OP_CSWAP |
+ Atomic conditional swap + |
UCP_ATOMIC_FETCH_OP_FAND |
+ Atomic Fetch and and + |
UCP_ATOMIC_FETCH_OP_FOR |
+ Atomic Fetch and or + |
UCP_ATOMIC_FETCH_OP_FXOR |
+ Atomic Fetch and xor + |
UCP_ATOMIC_FETCH_OP_LAST | + |
ucs_status_t ucp_request_query | +( | +void * | +request, | +
+ | + | ucp_request_attr_t * | +attr | +
+ | ) | ++ |
[in] | request | Non-blocking request to query. |
[out] | attr | Filled with attributes of the request. |
ucs_status_ptr_t ucp_am_send_nbx | +( | +ucp_ep_h | +ep, | +
+ | + | unsigned | +id, | +
+ | + | const void * | +header, | +
+ | + | size_t | +header_length, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine sends an Active Message to an ep. If the operation completes immediately, then the routine returns NULL and the callback function is ignored, even if specified. Otherwise, if no error is reported and a callback is requested (i.e. the UCP_OP_ATTR_FIELD_CALLBACK flag is set in the op_attr_mask field of param), then the UCP library will schedule invocation of the callback routine param->cb.send upon completion of the operation.
+[in] | ep | UCP endpoint where the Active Message will be run. |
[in] | id | Active Message id. Specifies which registered callback to run. |
[in] | header | User defined Active Message header. NULL value is allowed if no header needed. In this case header_length must be set to 0. By default the header must be valid until the active message send operation completes. If the flag UCP_AM_SEND_FLAG_COPY_HEADER is specified, the header is only required to be valid until this function call returns. |
[in] | header_length | Active message header length in bytes. |
[in] | buffer | Pointer to the data to be sent to the target node of the Active Message. |
[in] | count | Number of elements to send. |
[in] | param | Operation parameters, see ucp_request_param_t. |
ucs_status_ptr_t ucp_am_recv_data_nbx | +( | +ucp_worker_h | +worker, | +
+ | + | void * | +data_desc, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine receives a message that is described by the data descriptor data_desc, local address buffer, size count and param parameters on the worker. The routine is non-blocking and therefore returns immediately. The receive operation is considered completed when the message is delivered to the buffer. If the receive operation cannot be started the routine returns an error.
+[in] | worker | Worker that is used for the receive operation. |
[in] | data_desc | Data descriptor, provided in ucp_am_recv_callback_t routine. |
[in] | buffer | Pointer to the buffer to receive the data. |
[in] | count | Number of elements to receive into buffer. |
[in] | param | Operation parameters, see ucp_request_param_t. |
void ucp_am_data_release | +( | +ucp_worker_h | +worker, | +
+ | + | void * | +data | +
+ | ) | ++ |
This routine releases data that persisted through an Active Message callback because that callback returned UCS_INPROGRESS.
+[in] | worker | Worker which received the Active Message. |
[in] | data | Pointer to data that was passed into the Active Message callback as the data parameter. |
ucs_status_ptr_t ucp_stream_send_nbx | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine sends data that is described by the local address buffer, size count object to the destination endpoint ep. The routine is non-blocking and therefore returns immediately, however the actual send operation may be delayed. The send operation is considered completed when it is safe to reuse the source buffer. If the send operation is completed immediately the routine returns UCS_OK.
+[in] | ep | Destination endpoint handle. |
[in] | buffer | Pointer to the message buffer (payload). |
[in] | count | Number of elements to send. |
[in] | param | Operation parameters, see ucp_request_param_t. |
ucs_status_ptr_t ucp_tag_send_nbx | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_tag_t | +tag, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine sends a messages that is described by the local address buffer, size count object to the destination endpoint ep. Each message is associated with a tag value that is used for message matching on the ucp_tag_recv_nb or receiver. The routine is non-blocking and therefore returns immediately, however the actual send operation may be delayed. The send operation is considered completed when it is safe to reuse the source buffer. If the send operation is completed immediately the routine returns UCS_OK and the call-back function is not invoked. If the operation is not completed immediately and no error reported then the UCP library will schedule to invoke the call-back whenever the send operation is completed. In other words, the completion of a message can be signaled by the return code or the call-back. Immediate completion signals can be fine-tuned via the ucp_request_param_t::op_attr_mask field in the ucp_request_param_t structure. The values of this field are a bit-wise OR of the ucp_op_attr_t enumeration.
+[in] | ep | Destination endpoint handle. |
[in] | buffer | Pointer to the message buffer (payload). |
[in] | count | Number of elements to send |
[in] | tag | Message tag. |
[in] | param | Operation parameters, see ucp_request_param_t |
ucs_status_ptr_t ucp_tag_send_sync_nbx | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_tag_t | +tag, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
Same as ucp_tag_send_nbx, except the request completes only after there is a remote tag match on the message (which does not always mean the remote receive has been completed). This function never completes "in-place", and always returns a request handle.
+[in] | ep | Destination endpoint handle. |
[in] | buffer | Pointer to the message buffer (payload). |
[in] | count | Number of elements to send |
[in] | tag | Message tag. |
[in] | param | Operation parameters, see ucp_request_param_t |
ucs_status_ptr_t ucp_stream_recv_nbx | +( | +ucp_ep_h | +ep, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | size_t * | +length, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine receives data that is described by the local address buffer, size count object on the endpoint ep. The routine is non-blocking and therefore returns immediately. The receive operation is considered complete when the message is delivered to the buffer. If the receive operation cannot be started, then the routine returns an error.
+[in] | ep | UCP endpoint that is used for the receive operation. |
[in] | buffer | Pointer to the buffer that will receive the data. |
[in] | count | Number of elements to receive into buffer. |
[out] | length | Size of the received data in bytes. The value is valid only if return code is NULL. |
[in] | param | Operation parameters, see ucp_request_param_t. This operation supports specific flags, which can be passed in param by ucp_request_param_t::flags. The exact set of flags is defined by ucp_stream_recv_flags_t. |
ucs_status_ptr_t ucp_stream_recv_data_nb | +( | +ucp_ep_h | +ep, | +
+ | + | size_t * | +length | +
+ | ) | ++ |
This routine receives any available data from endpoint ep. Unlike ucp_stream_recv_nb, the returned data is unstructured and is treated as an array of bytes. If data is immediately available, UCS_STATUS_PTR(_ptr) is returned as a pointer to the data, and length is set to the size of the returned data buffer. The routine is non-blocking and therefore returns immediately.
+[in] | ep | UCP endpoint that is used for the receive operation. |
[out] | length | Length of received data. |
ucs_status_ptr_t ucp_tag_recv_nbx | +( | +ucp_worker_h | +worker, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_tag_t | +tag, | +
+ | + | ucp_tag_t | +tag_mask, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine receives a message that is described by the local address buffer, size count, and info object on the worker. The tag value of the receive message has to match the tag and tag_mask values, where the tag_mask indicates what bits of the tag have to be matched. The routine is a non-blocking and therefore returns immediately. The receive operation is considered completed when the message is delivered to the buffer. In order to notify the application about completion of the receive operation the UCP library will invoke the call-back cb when the received message is in the receive buffer and ready for application access. If the receive operation cannot be started, then the routine returns an error.
+[in] | worker | UCP worker that is used for the receive operation. |
[in] | buffer | Pointer to the buffer to receive the data. |
[in] | count | Number of elements to receive |
[in] | tag | Message tag to expect. |
[in] | tag_mask | Bit mask that indicates the bits that are used for the matching of the incoming tag against the expected tag. |
[in] | param | Operation parameters, see ucp_request_param_t |
ucp_tag_message_h ucp_tag_probe_nb | +( | +ucp_worker_h | +worker, | +
+ | + | ucp_tag_t | +tag, | +
+ | + | ucp_tag_t | +tag_mask, | +
+ | + | int | +remove, | +
+ | + | ucp_tag_recv_info_t * | +info | +
+ | ) | ++ |
This routine probes (checks) if a messages described by the tag and tag_mask was received (fully or partially) on the worker. The tag value of the received message has to match the tag and tag_mask values, where the tag_mask indicates what bits of the tag have to be matched. The function returns immediately and if the message is matched it returns a handle for the message.
+[in] | worker | UCP worker that is used for the probe operation. |
[in] | tag | Message tag to probe for. |
[in] | tag_mask | Bit mask that indicates the bits that are used for the matching of the incoming tag against the expected tag. |
[in] | remove | The flag indicates if the matched message has to be removed from UCP library. If true (1), the message handle is removed from the UCP library and the application is responsible to call ucp_tag_msg_recv_nb() in order to receive the data and release the resources associated with the message handle. If false (0), the return value is merely an indication to whether a matching message is present, and it cannot be used in any other way, and in particular it cannot be passed to ucp_tag_msg_recv_nb(). |
[out] | info | If the matching message is found the descriptor is filled with the details about the message. |
ucs_status_ptr_t ucp_tag_msg_recv_nbx | +( | +ucp_worker_h | +worker, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_tag_message_h | +message, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine receives a message that is described by the local address buffer, size count, and message handle on the worker. The message handle can be obtained by calling the ucp_tag_probe_nb() routine. The ucp_tag_msg_recv_nbx() routine is non-blocking and therefore returns immediately. The receive operation is considered completed when the message is delivered to the buffer. In order to notify the application about completion of the receive operation the UCP library will invoke the call-back cb when the received message is in the receive buffer and ready for application access. If the receive operation cannot be started, then the routine returns an error.
+[in] | worker | UCP worker that is used for the receive operation. |
[in] | buffer | Pointer to the buffer that will receive the data. |
[in] | count | Number of elements to receive |
[in] | message | Message handle. |
[in] | param | Operation parameters, see ucp_request_param_t |
ucs_status_ptr_t ucp_put_nbx | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine initiates a storage of contiguous block of data that is described by the local address buffer in the remote contiguous memory region described by remote_addr address and the memoryhandle" rkey. The routine returns immediately and does not guarantee re-usability of the source address buffer. If the operation is completed immediately the routine return UCS_OK, otherwise UCS_INPROGRESS or an error is returned to user. If the put operation completes immediately, the routine returns UCS_OK and the call-back routine param.cb.send is not invoked. If the operation is not completed immediately and no error is reported, then the UCP library will schedule invocation of the call-back routine param.cb.send upon completion of the put operation. In other words, the completion of a put operation can be signaled by the return code or execution of the call-back. Immediate completion signals can be fine-tuned via the ucp_request_param_t::op_attr_mask field in the ucp_request_param_t structure. The values of this field are a bit-wise OR of the ucp_op_attr_t enumeration.
+[in] | ep | Remote endpoint handle. |
[in] | buffer | Pointer to the local source address. |
[in] | count | Number of elements of type ucp_request_param_t::datatype to put. If ucp_request_param_t::datatype is not specified, the type defaults to ucp_dt_make_contig(1), which corresponds to byte elements. |
[in] | remote_addr | Pointer to the destination remote memory address to write to. |
[in] | rkey | Remote memory key associated with the remote memory address. |
[in] | param | Operation parameters, see ucp_request_param_t |
ucs_status_ptr_t ucp_get_nbx | +( | +ucp_ep_h | +ep, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine initiates a load of a contiguous block of data that is described by the remote memory address remote_addr and the memory handle rkey in the local contiguous memory region described by buffer address. The routine returns immediately and does not guarantee that remote data is loaded and stored under the local address buffer. If the operation is completed immediately the routine return UCS_OK, otherwise UCS_INPROGRESS or an error is returned to user. If the get operation completes immediately, the routine returns UCS_OK and the call-back routine param.cb.send is not invoked. If the operation is not completed immediately and no error is reported, then the UCP library will schedule invocation of the call-back routine param.cb.send upon completion of the get operation. In other words, the completion of a get operation can be signaled by the return code or execution of the call-back.
+[in] | ep | Remote endpoint handle. |
[in] | buffer | Pointer to the local destination address. |
[in] | count | Number of elements of type ucp_request_param_t::datatype to put. If ucp_request_param_t::datatype is not specified, the type defaults to ucp_dt_make_contig(1), which corresponds to byte elements. |
[in] | remote_addr | Pointer to the source remote memory address to read from. |
[in] | rkey | Remote memory key associated with the remote memory address. |
[in] | param | Operation parameters, see ucp_request_param_t. |
ucs_status_ptr_t ucp_atomic_op_nbx | +( | +ucp_ep_h | +ep, | +
+ | + | ucp_atomic_op_t | +opcode, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine will post an atomic operation to remote memory. The remote value is described by the combination of the remote memory address remote_addr and the remote memory handle rkey. The routine is non-blocking and therefore returns immediately. However, the actual atomic operation may be delayed. In order to enable fetching semantics for atomic operations user has to specify param.reply_buffer. Please see table below for more details.
+Atomic Operation | Pseudo code | X | Y | Z | Result |
---|---|---|---|---|---|
UCP_ATOMIC_OP_ADD | Result=Y; Y+=X | buffer | remote_addr | - | param.reply_buffer(optional) |
UCP_ATOMIC_OP_SWAP | Result=Y; Y=X | buffer | remote_addr | - | param.reply_buffer |
UCP_ATOMIC_OP_CSWAP | Result=Y; if (X==Y) then Y=Z | buffer | remote_addr | param.reply_buffer | param.reply_buffer |
UCP_ATOMIC_OP_AND | Result=Y; Y&=X | buffer | remote_addr | - | param.reply_buffer(optional) |
UCP_ATOMIC_OP_OR | Result=Y; Y|=X | buffer | remote_addr | - | param.reply_buffer(optional) |
UCP_ATOMIC_OP_XOR | Result=Y; Y^=X | buffer | remote_addr | - | param.reply_buffer(optional) |
[in] | ep | UCP endpoint. |
[in] | opcode | One of ucp_atomic_op_t. |
[in] | buffer | Address of operand for the atomic operation. See Atomic Operations Semantic table for exact usage by different atomic operations. |
[in] | count | Number of elements in buffer and result. The size of each element is specified by ucp_request_param_t::datatype |
[in] | remote_addr | Remote address to operate on. |
[in] | rkey | Remote key handle for the remote memory address. |
[in] | param | Operation parameters, see ucp_request_param_t. |
ucs_status_t ucp_request_check_status | +( | +void * | +request | ) | ++ |
This routine checks the state of the request and returns its current status. Any value different from UCS_INPROGRESS means that request is in a completed state.
+[in] | request | Non-blocking request to check. |
ucs_status_t ucp_tag_recv_request_test | +( | +void * | +request, | +
+ | + | ucp_tag_recv_info_t * | +info | +
+ | ) | ++ |
This routine checks the state and returns current status of the request returned from ucp_tag_recv_nb routine or the user allocated request for ucp_tag_recv_nbr. Any value different from UCS_INPROGRESS means that the request is in a completed state.
+[in] | request | Non-blocking request to check. |
[out] | info | It is filled with the details about the message available at the moment of calling. |
ucs_status_t ucp_stream_recv_request_test | +( | +void * | +request, | +
+ | + | size_t * | +length_p | +
+ | ) | ++ |
This routine checks the state and returns current status of the request returned from ucp_stream_recv_nb routine. Any value different from UCS_INPROGRESS means that the request is in a completed state.
+[in] | request | Non-blocking request to check. |
[out] | length_p | The size of the received data in bytes. This value is only valid if the status is UCS_OK. If valid, it is always an integral multiple of the datatype size associated with the request. |
void ucp_request_cancel | +( | +ucp_worker_h | +worker, | +
+ | + | void * | +request | +
+ | ) | ++ |
[in] | worker | UCP worker. |
[in] | request | Non-blocking request to cancel. |
This routine tries to cancels an outstanding communication request. After calling this routine, the request will be in completed or canceled (but not both) state regardless of the status of the target endpoint associated with the communication request. If the request is completed successfully, the send or receive completion callbacks (based on the type of the request) will be called with the status argument of the callback set to UCS_OK, and in a case it is canceled the status argument is set to UCS_ERR_CANCELED. It is important to note that in order to release the request back to the library the application is responsible for calling ucp_request_free().
+ +void ucp_stream_data_release | +( | +ucp_ep_h | +ep, | +
+ | + | void * | +data | +
+ | ) | ++ |
[in] | ep | Endpoint data received from. |
[in] | data | Data pointer to release, which was returned from ucp_stream_recv_data_nb. |
This routine releases internal UCP data buffer returned by ucp_stream_recv_data_nb when data is processed, the application can't use this buffer after calling this function.
+ +void ucp_request_free | +( | +void * | +request | ) | ++ |
[in] | request | Non-blocking request to release. |
This routine releases the non-blocking request back to the library, regardless of its current state. Communications operations associated with this request will make progress internally, however no further notifications or callbacks will be invoked for this request.
+void* ucp_request_alloc | +( | +ucp_worker_h | +worker | ) | ++ |
[in] | worker | UCP worker. |
This routine creates request which may be used in functions ucp_tag_send_nbx, ucp_tag_recv_nbx, etc. The application is responsible for releasing the handle using the ucp_request_free routine
+ +int ucp_request_is_completed | +( | +void * | +request | ) | ++ |
ucs_status_t ucp_put | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +length, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey | +
+ | ) | ++ |
This routine stores contiguous block of data that is described by the local address buffer in the remote contiguous memory region described by remote_addr address and the memory handle rkey. The routine returns when it is safe to reuse the source address buffer.
+[in] | ep | Remote endpoint handle. |
[in] | buffer | Pointer to the local source address. |
[in] | length | Length of the data (in bytes) stored under the source address. |
[in] | remote_addr | Pointer to the destination remote address to write to. |
[in] | rkey | Remote memory key associated with the remote address. |
ucs_status_t ucp_get | +( | +ucp_ep_h | +ep, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +length, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey | +
+ | ) | ++ |
This routine loads contiguous block of data that is described by the remote address remote_addr and the memory handle rkey in the local contiguous memory region described by buffer address. The routine returns when remote data is loaded and stored under the local address buffer.
+[in] | ep | Remote endpoint handle. |
[in] | buffer | Pointer to the local source address. |
[in] | length | Length of the data (in bytes) stored under the source address. |
[in] | remote_addr | Pointer to the destination remote address to write to. |
[in] | rkey | Remote memory key associated with the remote address. |
ucs_status_t ucp_atomic_add32 | +( | +ucp_ep_h | +ep, | +
+ | + | uint32_t | +add, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey | +
+ | ) | ++ |
This routine performs an add operation on a 32 bit integer value atomically. The remote integer value is described by the combination of the remote memory address remote_addr and the remote memory handle rkey. The add value is the value that is used for the add operation. When the operation completes the sum of the original remote value and the operand value (add) is stored in remote memory. The call to the routine returns immediately, independent of operation completion.
+[in] | ep | Remote endpoint handle. |
[in] | add | Value to add. |
[in] | remote_addr | Pointer to the destination remote address of the atomic variable. |
[in] | rkey | Remote memory key associated with the remote address. |
ucs_status_t ucp_atomic_add64 | +( | +ucp_ep_h | +ep, | +
+ | + | uint64_t | +add, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey | +
+ | ) | ++ |
This routine performs an add operation on a 64 bit integer value atomically. The remote integer value is described by the combination of the remote memory address remote_addr and the remote memory handle rkey. The add value is the value that is used for the add operation. When the operation completes the sum of the original remote value and the operand value (add) is stored in remote memory. The call to the routine returns immediately, independent of operation completion.
+[in] | ep | Remote endpoint handle. |
[in] | add | Value to add. |
[in] | remote_addr | Pointer to the destination remote address of the atomic variable. |
[in] | rkey | Remote memory key associated with the remote address. |
ucs_status_t ucp_atomic_fadd32 | +( | +ucp_ep_h | +ep, | +
+ | + | uint32_t | +add, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | uint32_t * | +result | +
+ | ) | ++ |
This routine performs an add operation on a 32 bit integer value atomically. The remote integer value is described by the combination of the remote memory address remote_addr and the remote memory handle rkey. The add value is the value that is used for the add operation. When the operation completes, the original remote value is stored in the local memory result, and the sum of the original remote value and the operand value is stored in remote memory. The call to the routine returns when the operation is completed and the result value is updated.
+[in] | ep | Remote endpoint handle. |
[in] | add | Value to add. |
[in] | remote_addr | Pointer to the destination remote address of the atomic variable. |
[in] | rkey | Remote memory key associated with the remote address. |
[out] | result | Pointer to the address that is used to store the previous value of the atomic variable described by the remote_addr |
ucs_status_t ucp_atomic_fadd64 | +( | +ucp_ep_h | +ep, | +
+ | + | uint64_t | +add, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | uint64_t * | +result | +
+ | ) | ++ |
This routine performs an add operation on a 64 bit integer value atomically. The remote integer value is described by the combination of the remote memory address remote_addr and the remote memory handle rkey. The add value is the value that is used for the add operation. When the operation completes, the original remote value is stored in the local memory result, and the sum of the original remote value and the operand value is stored in remote memory. The call to the routine returns when the operation is completed and the result value is updated.
+[in] | ep | Remote endpoint handle. |
[in] | add | Value to add. |
[in] | remote_addr | Pointer to the destination remote address of the atomic variable. |
[in] | rkey | Remote memory key associated with the remote address. |
[out] | result | Pointer to the address that is used to store the previous value of the atomic variable described by the remote_addr |
ucs_status_t ucp_atomic_swap32 | +( | +ucp_ep_h | +ep, | +
+ | + | uint32_t | +swap, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | uint32_t * | +result | +
+ | ) | ++ |
This routine swaps a 32 bit value between local and remote memory. The remote value is described by the combination of the remote memory address remote_addr and the remote memory handle rkey. The swap value is the value that is used for the swap operation. When the operation completes, the remote value is stored in the local memory result, and the operand value (swap) is stored in remote memory. The call to the routine returns when the operation is completed and the result value is updated.
+[in] | ep | Remote endpoint handle. |
[in] | swap | Value to swap. |
[in] | remote_addr | Pointer to the destination remote address of the atomic variable. |
[in] | rkey | Remote memory key associated with the remote address. |
[out] | result | Pointer to the address that is used to store the previous value of the atomic variable described by the remote_addr |
ucs_status_t ucp_atomic_swap64 | +( | +ucp_ep_h | +ep, | +
+ | + | uint64_t | +swap, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | uint64_t * | +result | +
+ | ) | ++ |
This routine swaps a 64 bit value between local and remote memory. The remote value is described by the combination of the remote memory address remote_addr and the remote memory handle rkey. The swap value is the value that is used for the swap operation. When the operation completes, the remote value is stored in the local memory result, and the operand value (swap) is stored in remote memory. The call to the routine returns when the operation is completed and the result value is updated.
+[in] | ep | Remote endpoint handle. |
[in] | swap | Value to swap. |
[in] | remote_addr | Pointer to the destination remote address of the atomic variable. |
[in] | rkey | Remote memory key associated with the remote address. |
[out] | result | Pointer to the address that is used to store the previous value of the atomic variable described by the remote_addr |
ucs_status_t ucp_atomic_cswap32 | +( | +ucp_ep_h | +ep, | +
+ | + | uint32_t | +compare, | +
+ | + | uint32_t | +swap, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | uint32_t * | +result | +
+ | ) | ++ |
This routine conditionally swaps a 32 bit value between local and remote memory. The swap occurs only if the condition value (continue) is equal to the remote value, otherwise the remote memory is not modified. The remote value is described by the combination of the remote memory address remote_addr
and the remote memory handle rkey. The swap
value is the value that is used to update the remote memory if the condition is true. The call to the routine returns when the operation is completed and the result value is updated.
[in] | ep | Remote endpoint handle. |
[in] | compare | Value to compare to. |
[in] | swap | Value to swap. |
[in] | remote_addr | Pointer to the destination remote address of the atomic variable. |
[in] | rkey | Remote memory key associated with the remote address. |
[out] | result | Pointer to the address that is used to store the previous value of the atomic variable described by the remote_addr |
ucs_status_t ucp_atomic_cswap64 | +( | +ucp_ep_h | +ep, | +
+ | + | uint64_t | +compare, | +
+ | + | uint64_t | +swap, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | uint64_t * | +result | +
+ | ) | ++ |
This routine conditionally swaps a 64 bit value between local and remote memory. The swap occurs only if the condition value (continue) is equal to the remote value, otherwise the remote memory is not modified. The remote value is described by the combination of the remote memory address remote_addr
and the remote memory handle rkey. The swap
value is the value that is used to update the remote memory if the condition is true. The call to the routine returns when the operation is completed and the result value is updated.
[in] | ep | Remote endpoint handle. |
[in] | compare | Value to compare to. |
[in] | swap | Value to swap. |
[in] | remote_addr | Pointer to the destination remote address of the atomic variable. |
[in] | rkey | Remote memory key associated with the remote address. |
[out] | result | Pointer to the address that is used to store the previous value of the atomic variable described by the remote_addr |
ucs_status_ptr_t ucp_am_send_nb | +( | +ucp_ep_h | +ep, | +
+ | + | uint16_t | +id, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_datatype_t | +datatype, | +
+ | + | ucp_send_callback_t | +cb, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
This routine sends an Active Message to an ep. It does not support CUDA memory.
+[in] | ep | UCP endpoint where the Active Message will be run. |
[in] | id | Active Message id. Specifies which registered callback to run. |
[in] | buffer | Pointer to the data to be sent to the target node of the Active Message. |
[in] | count | Number of elements to send. |
[in] | datatype | Datatype descriptor for the elements in the buffer. |
[in] | cb | Callback that is invoked upon completion of the data transfer if it is not completed immediately. |
[in] | flags | Operation flags as defined by ucp_send_am_flags. |
ucs_status_ptr_t ucp_stream_send_nb | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_datatype_t | +datatype, | +
+ | + | ucp_send_callback_t | +cb, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
This routine sends data that is described by the local address buffer, size count, and datatype object to the destination endpoint ep. The routine is non-blocking and therefore returns immediately, however the actual send operation may be delayed. The send operation is considered completed when it is safe to reuse the source buffer. If the send operation is completed immediately the routine returns UCS_OK and the callback function cb is not invoked. If the operation is not completed immediately and no error reported, then the UCP library will schedule invocation of the callback cb upon completion of the send operation. In other words, the completion of the operation will be signaled either by the return code or by the callback.
+[in] | ep | Destination endpoint handle. |
[in] | buffer | Pointer to the message buffer (payload). |
[in] | count | Number of elements to send. |
[in] | datatype | Datatype descriptor for the elements in the buffer. |
[in] | cb | Callback function that is invoked whenever the send operation is completed. It is important to note that the callback is only invoked in the event that the operation cannot be completed in place. |
[in] | flags | Reserved for future use. |
ucs_status_ptr_t ucp_stream_recv_nb | +( | +ucp_ep_h | +ep, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_datatype_t | +datatype, | +
+ | + | ucp_stream_recv_callback_t | +cb, | +
+ | + | size_t * | +length, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
This routine receives data that is described by the local address buffer, size count, and datatype object on the endpoint ep. The routine is non-blocking and therefore returns immediately. The receive operation is considered complete when the message is delivered to the buffer. If data is not immediately available, the operation will be scheduled for receive and a request handle will be returned. In order to notify the application about completion of a scheduled receive operation, the UCP library will invoke the call-back cb when data is in the receive buffer and ready for application access. If the receive operation cannot be started, the routine returns an error.
+[in] | ep | UCP endpoint that is used for the receive operation. |
[in] | buffer | Pointer to the buffer to receive the data. |
[in] | count | Number of elements to receive into buffer. |
[in] | datatype | Datatype descriptor for the elements in the buffer. |
[in] | cb | Callback function that is invoked whenever the receive operation is completed and the data is ready in the receive buffer. It is important to note that the call-back is only invoked in a case when the operation cannot be completed immediately. |
[out] | length | Size of the received data in bytes. The value is valid only if return code is UCS_OK. |
[in] | flags | Flags defined in ucp_stream_recv_flags_t. |
ucs_status_ptr_t ucp_tag_send_nb | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_datatype_t | +datatype, | +
+ | + | ucp_tag_t | +tag, | +
+ | + | ucp_send_callback_t | +cb | +
+ | ) | ++ |
This routine sends a messages that is described by the local address buffer, size count, and datatype object to the destination endpoint ep. Each message is associated with a tag value that is used for message matching on the receiver. The routine is non-blocking and therefore returns immediately, however the actual send operation may be delayed. The send operation is considered completed when it is safe to reuse the source buffer. If the send operation is completed immediately the routine return UCS_OK and the call-back function cb is not invoked. If the operation is not completed immediately and no error reported then the UCP library will schedule to invoke the call-back cb whenever the send operation will be completed. In other words, the completion of a message can be signaled by the return code or the call-back.
+[in] | ep | Destination endpoint handle. |
[in] | buffer | Pointer to the message buffer (payload). |
[in] | count | Number of elements to send |
[in] | datatype | Datatype descriptor for the elements in the buffer. |
[in] | tag | Message tag. |
[in] | cb | Callback function that is invoked whenever the send operation is completed. It is important to note that the call-back is only invoked in a case when the operation cannot be completed in place. |
ucs_status_t ucp_tag_send_nbr | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_datatype_t | +datatype, | +
+ | + | ucp_tag_t | +tag, | +
+ | + | void * | +req | +
+ | ) | ++ |
This routine provides a convenient and efficient way to implement a blocking send pattern. It also completes requests faster than ucp_tag_send_nb() because:
+This routine sends a messages that is described by the local address buffer, size count, and datatype object to the destination endpoint ep. Each message is associated with a tag value that is used for message matching on the receiver.
+The routine is non-blocking and therefore returns immediately, however the actual send operation may be delayed. The send operation is considered completed when it is safe to reuse the source buffer. If the send operation is completed immediately the routine returns UCS_OK.
+If the operation is not completed immediately and no error reported then the UCP library will fill a user provided req and return UCS_INPROGRESS status. In order to monitor completion of the operation ucp_request_check_status() should be used.
+Following pseudo code implements a blocking send function:
+[in] | ep | Destination endpoint handle. |
[in] | buffer | Pointer to the message buffer (payload). |
[in] | count | Number of elements to send |
[in] | datatype | Datatype descriptor for the elements in the buffer. |
[in] | tag | Message tag. |
[in] | req | Request handle allocated by the user. There should be at least UCP request size bytes of available space before the req. The size of UCP request can be obtained by ucp_context_query function. |
ucs_status_ptr_t ucp_tag_send_sync_nb | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_datatype_t | +datatype, | +
+ | + | ucp_tag_t | +tag, | +
+ | + | ucp_send_callback_t | +cb | +
+ | ) | ++ |
Same as ucp_tag_send_nb, except the request completes only after there is a remote tag match on the message (which does not always mean the remote receive has been completed). This function never completes "in-place", and always returns a request handle.
+[in] | ep | Destination endpoint handle. |
[in] | buffer | Pointer to the message buffer (payload). |
[in] | count | Number of elements to send |
[in] | datatype | Datatype descriptor for the elements in the buffer. |
[in] | tag | Message tag. |
[in] | cb | Callback function that is invoked whenever the send operation is completed. |
ucs_status_ptr_t ucp_tag_recv_nb | +( | +ucp_worker_h | +worker, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_datatype_t | +datatype, | +
+ | + | ucp_tag_t | +tag, | +
+ | + | ucp_tag_t | +tag_mask, | +
+ | + | ucp_tag_recv_callback_t | +cb | +
+ | ) | ++ |
This routine receives a message that is described by the local address buffer, size count, and datatype object on the worker. The tag value of the receive message has to match the tag and tag_mask values, where the tag_mask indicates which bits of the tag have to be matched. The routine is non-blocking and therefore returns immediately. The receive operation is considered completed when the message is delivered to the buffer. In order to notify the application about completion of the receive operation the UCP library will invoke the call-back cb when the received message is in the receive buffer and ready for application access. If the receive operation cannot be stated the routine returns an error.
+[in] | worker | UCP worker that is used for the receive operation. |
[in] | buffer | Pointer to the buffer to receive the data. |
[in] | count | Number of elements to receive |
[in] | datatype | Datatype descriptor for the elements in the buffer. |
[in] | tag | Message tag to expect. |
[in] | tag_mask | Bit mask that indicates the bits that are used for the matching of the incoming tag against the expected tag. |
[in] | cb | Callback function that is invoked whenever the receive operation is completed and the data is ready in the receive buffer. |
ucs_status_t ucp_tag_recv_nbr | +( | +ucp_worker_h | +worker, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_datatype_t | +datatype, | +
+ | + | ucp_tag_t | +tag, | +
+ | + | ucp_tag_t | +tag_mask, | +
+ | + | void * | +req | +
+ | ) | ++ |
This routine receives a message that is described by the local address buffer, size count, and datatype object on the worker. The tag value of the receive message has to match the tag and tag_mask values, where the tag_mask indicates which bits of the tag have to be matched. The routine is non-blocking and therefore returns immediately. The receive operation is considered completed when the message is delivered to the buffer. In order to monitor completion of the operation ucp_request_check_status or ucp_tag_recv_request_test should be used.
+[in] | worker | UCP worker that is used for the receive operation. |
[in] | buffer | Pointer to the buffer to receive the data. |
[in] | count | Number of elements to receive |
[in] | datatype | Datatype descriptor for the elements in the buffer. |
[in] | tag | Message tag to expect. |
[in] | tag_mask | Bit mask that indicates the bits that are used for the matching of the incoming tag against the expected tag. |
[in] | req | Request handle allocated by the user. There should be at least UCP request size bytes of available space before the req. The size of UCP request can be obtained by ucp_context_query function. |
ucs_status_ptr_t ucp_tag_msg_recv_nb | +( | +ucp_worker_h | +worker, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +count, | +
+ | + | ucp_datatype_t | +datatype, | +
+ | + | ucp_tag_message_h | +message, | +
+ | + | ucp_tag_recv_callback_t | +cb | +
+ | ) | ++ |
This routine receives a message that is described by the local address buffer, size count, message handle, and datatype object on the worker. The message handle can be obtained by calling the ucp_tag_probe_nb() routine. The ucp_tag_msg_recv_nb() routine is non-blocking and therefore returns immediately. The receive operation is considered completed when the message is delivered to the buffer. In order to notify the application about completion of the receive operation the UCP library will invoke the call-back cb when the received message is in the receive buffer and ready for application access. If the receive operation cannot be started the routine returns an error.
+[in] | worker | UCP worker that is used for the receive operation. |
[in] | buffer | Pointer to the buffer that will receive the data. |
[in] | count | Number of elements to receive |
[in] | datatype | Datatype descriptor for the elements in the buffer. |
[in] | message | Message handle. |
[in] | cb | Callback function that is invoked whenever the receive operation is completed and the data is ready in the receive buffer. |
ucs_status_t ucp_put_nbi | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +length, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey | +
+ | ) | ++ |
This routine initiates a storage of contiguous block of data that is described by the local address buffer in the remote contiguous memory region described by remote_addr address and the memoryhandle" rkey. The routine returns immediately and does not guarantee re-usability of the source address buffer. If the operation is completed immediately the routine return UCS_OK, otherwise UCS_INPROGRESS or an error is returned to user.
+[in] | ep | Remote endpoint handle. |
[in] | buffer | Pointer to the local source address. |
[in] | length | Length of the data (in bytes) stored under the source address. |
[in] | remote_addr | Pointer to the destination remote memory address to write to. |
[in] | rkey | Remote memory key associated with the remote memory address. |
ucs_status_ptr_t ucp_put_nb | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | size_t | +length, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | ucp_send_callback_t | +cb | +
+ | ) | ++ |
This routine initiates a storage of contiguous block of data that is described by the local address buffer in the remote contiguous memory region described by remote_addr address and the memoryhandle" rkey. The routine returns immediately and does not guarantee re-usability of the source address buffer. If the operation is completed immediately the routine return UCS_OK, otherwise UCS_INPROGRESS or an error is returned to user. If the put operation completes immediately, the routine returns UCS_OK and the call-back routine cb is not invoked. If the operation is not completed immediately and no error is reported, then the UCP library will schedule invocation of the call-back routine cb upon completion of the put operation. In other words, the completion of a put operation can be signaled by the return code or execution of the call-back.
+[in] | ep | Remote endpoint handle. |
[in] | buffer | Pointer to the local source address. |
[in] | length | Length of the data (in bytes) stored under the source address. |
[in] | remote_addr | Pointer to the destination remote memory address to write to. |
[in] | rkey | Remote memory key associated with the remote memory address. |
[in] | cb | Call-back function that is invoked whenever the put operation is completed and the local buffer can be modified. Does not guarantee remote completion. |
ucs_status_t ucp_get_nbi | +( | +ucp_ep_h | +ep, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +length, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey | +
+ | ) | ++ |
This routine initiate a load of contiguous block of data that is described by the remote memory address remote_addr and the memory handle rkey in the local contiguous memory region described by buffer address. The routine returns immediately and does not guarantee that remote data is loaded and stored under the local address buffer.
+[in] | ep | Remote endpoint handle. |
[in] | buffer | Pointer to the local destination address. |
[in] | length | Length of the data (in bytes) stored under the destination address. |
[in] | remote_addr | Pointer to the source remote memory address to read from. |
[in] | rkey | Remote memory key associated with the remote memory address. |
ucs_status_ptr_t ucp_get_nb | +( | +ucp_ep_h | +ep, | +
+ | + | void * | +buffer, | +
+ | + | size_t | +length, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | ucp_send_callback_t | +cb | +
+ | ) | ++ |
This routine initiates a load of a contiguous block of data that is described by the remote memory address remote_addr and the memory handle rkey in the local contiguous memory region described by buffer address. The routine returns immediately and does not guarantee that remote data is loaded and stored under the local address buffer. If the operation is completed immediately the routine return UCS_OK, otherwise UCS_INPROGRESS or an error is returned to user. If the get operation completes immediately, the routine returns UCS_OK and the call-back routine cb is not invoked. If the operation is not completed immediately and no error is reported, then the UCP library will schedule invocation of the call-back routine cb upon completion of the get operation. In other words, the completion of a get operation can be signaled by the return code or execution of the call-back.
+[in] | ep | Remote endpoint handle. |
[in] | buffer | Pointer to the local destination address. |
[in] | length | Length of the data (in bytes) stored under the destination address. |
[in] | remote_addr | Pointer to the source remote memory address to read from. |
[in] | rkey | Remote memory key associated with the remote memory address. |
[in] | cb | Call-back function that is invoked whenever the get operation is completed and the data is visible to the local process. |
ucs_status_t ucp_atomic_post | +( | +ucp_ep_h | +ep, | +
+ | + | ucp_atomic_post_op_t | +opcode, | +
+ | + | uint64_t | +value, | +
+ | + | size_t | +op_size, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey | +
+ | ) | ++ |
This routine posts an atomic memory operation to a remote value. The remote value is described by the combination of the remote memory address remote_addr and the remote memory handle rkey. Return from the function does not guarantee completion. A user must call ucp_ep_flush_nb or ucp_worker_flush_nb to guarantee that the remote value has been updated.
+[in] | ep | UCP endpoint. |
[in] | opcode | One of ucp_atomic_post_op_t. |
[in] | value | Source operand for the atomic operation. |
[in] | op_size | Size of value in bytes |
[in] | remote_addr | Remote address to operate on. |
[in] | rkey | Remote key handle for the remote memory address. |
ucs_status_ptr_t ucp_atomic_fetch_nb | +( | +ucp_ep_h | +ep, | +
+ | + | ucp_atomic_fetch_op_t | +opcode, | +
+ | + | uint64_t | +value, | +
+ | + | void * | +result, | +
+ | + | size_t | +op_size, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | ucp_rkey_h | +rkey, | +
+ | + | ucp_send_callback_t | +cb | +
+ | ) | ++ |
This routine will post an atomic fetch operation to remote memory. The remote value is described by the combination of the remote memory address remote_addr and the remote memory handle rkey. The routine is non-blocking and therefore returns immediately. However the actual atomic operation may be delayed. The atomic operation is not considered complete until the values in remote and local memory are completed. If the atomic operation completes immediately, the routine returns UCS_OK and the call-back routine cb is not invoked. If the operation is not completed immediately and no error is reported, then the UCP library will schedule invocation of the call-back routine cb upon completion of the atomic operation. In other words, the completion of an atomic operation can be signaled by the return code or execution of the call-back.
+[in] | ep | UCP endpoint. |
[in] | opcode | One of ucp_atomic_fetch_op_t. |
[in] | value | Source operand for atomic operation. In the case of CSWAP this is the conditional for the swap. For SWAP this is the value to be placed in remote memory. |
[in,out] | result | Local memory address to store resulting fetch to. In the case of CSWAP the value in result will be swapped into the remote_addr if the condition is true. |
[in] | op_size | Size of value in bytes and pointer type for result |
[in] | remote_addr | Remote address to operate on. |
[in] | rkey | Remote key handle for the remote memory address. |
[in] | cb | Call-back function that is invoked whenever the send operation is completed. It is important to note that the call-back function is only invoked in a case when the operation cannot be completed in place. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | ucp_params |
Tuning parameters for UCP library. More... | |
+Typedefs | |
typedef struct ucp_params | ucp_params_t |
Tuning parameters for UCP library. More... | |
typedef struct ucp_config | ucp_config_t |
UCP configuration descriptor. More... | |
+Functions | |
ucs_status_t | ucp_config_read (const char *env_prefix, const char *filename, ucp_config_t **config_p) |
Read UCP configuration descriptor. More... | |
void | ucp_config_release (ucp_config_t *config) |
Release configuration descriptor. More... | |
ucs_status_t | ucp_config_modify (ucp_config_t *config, const char *name, const char *value) |
Modify context configuration. More... | |
void | ucp_config_print (const ucp_config_t *config, FILE *stream, const char *title, ucs_config_print_flags_t print_flags) |
Print configuration information. More... | |
This section describes routines for configuration of the UCP network layer
+struct ucp_params | +
The structure defines the parameters that are used for UCP library tuning during UCP library initialization.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_params_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+uint64_t | ++features | +
+ UCP features that are used for library initialization. It is recommended for applications only to request the features that are required for an optimal functionality This field must be specified. + |
+size_t | ++request_size | +
+ The size of a reserved space in a non-blocking requests. Typically applications use this space for caching own structures in order to avoid costly memory allocations, pointer dereferences, and cache misses. For example, MPI implementation can use this memory for caching MPI descriptors This field defaults to 0 if not specified. + |
+ucp_request_init_callback_t | ++request_init | +
+ Pointer to a routine that is used for the request initialization. This function will be called only on the very first time a request memory is initialized, and may not be called again if a request is reused. If a request should be reset before the next reuse, it can be done before calling ucp_request_free. +NULL can be used if no such is function required, which is also the default if this field is not specified by field_mask. + |
+ucp_request_cleanup_callback_t | ++request_cleanup | +
+ Pointer to a routine that is responsible for final cleanup of the memory associated with the request. This routine may not be called every time a request is released. For some implementations, the cleanup call may be delayed and only invoked at ucp_worker_destroy. +NULL can be used if no such function is required, which is also the default if this field is not specified by field_mask. + |
+uint64_t | ++tag_sender_mask | +
+ Mask which specifies particular bits of the tag which can uniquely identify the sender (UCP endpoint) in tagged operations. This field defaults to 0 if not specified. + |
+int | ++mt_workers_shared | +
+ This flag indicates if this context is shared by multiple workers from different threads. If so, this context needs thread safety support; otherwise, the context does not need to provide thread safety. For example, if the context is used by single worker, and that worker is shared by multiple threads, this context does not need thread safety; if the context is used by worker 1 and worker 2, and worker 1 is used by thread 1 and worker 2 is used by thread 2, then this context needs thread safety. Note that actual thread mode may be different from mode passed to ucp_init. To get actual thread mode use ucp_context_query. + |
+size_t | ++estimated_num_eps | +
+ An optimization hint of how many endpoints will be created on this context. For example, when used from MPI or SHMEM libraries, this number will specify the number of ranks (or processing elements) in the job. Does not affect semantics, but only transport selection criteria and the resulting performance. The value can be also set by UCX_NUM_EPS environment variable. In such case it will override the number of endpoints set by estimated_num_eps + |
+size_t | ++estimated_num_ppn | +
+ An optimization hint for a single node. For example, when used from MPI or OpenSHMEM libraries, this number will specify the number of Processes Per Node (PPN) in the job. Does not affect semantics, only transport selection criteria and the resulting performance. The value can be also set by the UCX_NUM_PPN environment variable, which will override the number of endpoints set by estimated_num_ppn + |
+const char * | ++name | +
+ Tracing and analysis tools can identify the context using this name. To retrieve the context's name, use ucp_context_query, as the name you supply may be changed by UCX under some circumstances, e.g. a name conflict. This field is only assigned if you set UCP_PARAM_FIELD_NAME in the field mask. If not, then a default unique name will be created for you. + |
typedef struct ucp_params ucp_params_t | +
The structure defines the parameters that are used for UCP library tuning during UCP library initialization.
+typedef struct ucp_config ucp_config_t | +
This descriptor defines the configuration for UCP application context. The configuration is loaded from the run-time environment (using configuration files of environment variables) using ucp_config_read routine and can be printed using ucp_config_print routine. In addition, application is responsible to release the descriptor using ucp_config_release routine.
+ +ucs_status_t ucp_config_read | +( | +const char * | +env_prefix, | +
+ | + | const char * | +filename, | +
+ | + | ucp_config_t ** | +config_p | +
+ | ) | ++ |
The routine fetches the information about UCP library configuration from the run-time environment. Then, the fetched descriptor is used for UCP library initialization. The Application can print out the descriptor using print routine. In addition the application is responsible for releasing the descriptor back to the UCP library.
+[in] | env_prefix | If non-NULL, the routine searches for the environment variables that start with <env_prefix>_UCX_ prefix. Otherwise, the routine searches for the environment variables that start with UCX_ prefix. |
[in] | filename | If non-NULL, read configuration from the file defined by filename. If the file does not exist, it will be ignored and no error reported to the application. |
[out] | config_p | Pointer to configuration descriptor as defined by ucp_config_t. |
void ucp_config_release | +( | +ucp_config_t * | +config | ) | ++ |
The routine releases the configuration descriptor that was allocated through ucp_config_read() routine.
+[out] | config | Configuration descriptor as defined by ucp_config_t. |
ucs_status_t ucp_config_modify | +( | +ucp_config_t * | +config, | +
+ | + | const char * | +name, | +
+ | + | const char * | +value | +
+ | ) | ++ |
The routine changes one configuration setting stored in configuration descriptor.
+[in] | config | Configuration to modify. |
[in] | name | Configuration variable name. |
[in] | value | Value to set. |
void ucp_config_print | +( | +const ucp_config_t * | +config, | +
+ | + | FILE * | +stream, | +
+ | + | const char * | +title, | +
+ | + | ucs_config_print_flags_t | +print_flags | +
+ | ) | ++ |
The routine prints the configuration information that is stored in configuration descriptor.
+[in] | config | Configuration descriptor to print. |
[in] | stream | Output stream to print the configuration to. |
[in] | title | Configuration title to print. |
[in] | print_flags | Flags that control various printing options. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | ucp_lib_attr |
Lib attributes. More... | |
struct | ucp_context_attr |
Context attributes. More... | |
struct | ucp_tag_recv_info |
UCP receive information descriptor. More... | |
struct | ucp_request_param_t |
Operation parameters passed to ucp_tag_send_nbx, ucp_tag_send_sync_nbx, ucp_tag_recv_nbx, ucp_put_nbx, ucp_get_nbx, ucp_am_send_nbx and ucp_am_recv_data_nbx. More... | |
union | ucp_request_param_t.cb |
union | ucp_request_param_t.recv_info |
+Macros | |
#define | UCP_ENTITY_NAME_MAX 32 |
Maximum size of the UCP entity name in structure of entity attributes provided by a query method. More... | |
+Typedefs | |
typedef struct ucp_lib_attr | ucp_lib_attr_t |
Lib attributes. More... | |
typedef struct ucp_context_attr | ucp_context_attr_t |
Context attributes. More... | |
typedef struct ucp_tag_recv_info | ucp_tag_recv_info_t |
UCP receive information descriptor. More... | |
typedef struct ucp_context * | ucp_context_h |
UCP Application Context. More... | |
typedef void(* | ucp_request_init_callback_t )(void *request) |
Request initialization callback. More... | |
typedef void(* | ucp_request_cleanup_callback_t )(void *request) |
Request cleanup callback. More... | |
+Enumerations | |
enum | ucp_params_field { + UCP_PARAM_FIELD_FEATURES = UCS_BIT(0), +UCP_PARAM_FIELD_REQUEST_SIZE = UCS_BIT(1), +UCP_PARAM_FIELD_REQUEST_INIT = UCS_BIT(2), +UCP_PARAM_FIELD_REQUEST_CLEANUP = UCS_BIT(3), + + UCP_PARAM_FIELD_TAG_SENDER_MASK = UCS_BIT(4), +UCP_PARAM_FIELD_MT_WORKERS_SHARED = UCS_BIT(5), +UCP_PARAM_FIELD_ESTIMATED_NUM_EPS = UCS_BIT(6), +UCP_PARAM_FIELD_ESTIMATED_NUM_PPN = UCS_BIT(7), + + UCP_PARAM_FIELD_NAME = UCS_BIT(8) + + } |
UCP context parameters field mask. More... | |
enum | ucp_feature { + UCP_FEATURE_TAG = UCS_BIT(0), +UCP_FEATURE_RMA = UCS_BIT(1), +UCP_FEATURE_AMO32 = UCS_BIT(2), +UCP_FEATURE_AMO64 = UCS_BIT(3), + + UCP_FEATURE_WAKEUP = UCS_BIT(4), +UCP_FEATURE_STREAM = UCS_BIT(5), +UCP_FEATURE_AM = UCS_BIT(6), +UCP_FEATURE_EXPORTED_MEMH = UCS_BIT(7) + + } |
UCP configuration features. More... | |
enum | ucp_lib_attr_field { UCP_LIB_ATTR_FIELD_MAX_THREAD_LEVEL = UCS_BIT(0) + } |
UCP library attributes field mask. More... | |
enum | ucp_context_attr_field { UCP_ATTR_FIELD_REQUEST_SIZE = UCS_BIT(0), +UCP_ATTR_FIELD_THREAD_MODE = UCS_BIT(1), +UCP_ATTR_FIELD_MEMORY_TYPES = UCS_BIT(2), +UCP_ATTR_FIELD_NAME = UCS_BIT(3) + } |
UCP context attributes field mask. More... | |
+Functions | |
ucs_status_t | ucp_lib_query (ucp_lib_attr_t *attr) |
Get attributes of the UCP library. More... | |
void | ucp_get_version (unsigned *major_version, unsigned *minor_version, unsigned *release_number) |
Get UCP library version. More... | |
const char * | ucp_get_version_string (void) |
Get UCP library version as a string. More... | |
static ucs_status_t | ucp_init (const ucp_params_t *params, const ucp_config_t *config, ucp_context_h *context_p) |
UCP context initialization. More... | |
void | ucp_cleanup (ucp_context_h context_p) |
Release UCP application context. More... | |
ucs_status_t | ucp_context_query (ucp_context_h context_p, ucp_context_attr_t *attr) |
Get attributes specific to a particular context. More... | |
void | ucp_context_print_info (const ucp_context_h context, FILE *stream) |
Print context information. More... | |
Application context is a primary concept of UCP design which provides an isolation mechanism, allowing resources associated with the context to separate or share network communication context across multiple instances of applications.
+This section provides a detailed description of this concept and routines associated with it.
+struct ucp_lib_attr | +
The structure defines the attributes that characterize the Library.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_lib_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+ucs_thread_mode_t | ++max_thread_level | +
+ Maximum level of thread support of the library, which is permanent throughout the lifetime of the library. Accordingly, the user can call ucp_worker_create with appropriate ucp_worker_params_t::thread_mode. For supported thread levels please see ucs_thread_mode_t. + |
struct ucp_context_attr | +
The structure defines the attributes that characterize the particular context.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_context_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+size_t | ++request_size | +
+ Size of UCP non-blocking request. When pre-allocated request is used (e.g. in ucp_tag_recv_nbr) it should have enough space to fit UCP request data, which is defined by this value. + |
+ucs_thread_mode_t | ++thread_mode | +
+ Thread safe level of the context. For supported thread levels please see ucs_thread_mode_t. + |
+uint64_t | ++memory_types | +
+ Mask of which memory types are supported, for supported memory types please see ucs_memory_type_t. + |
+char | ++name[UCP_ENTITY_NAME_MAX] | +
+ Tracing and analysis tools can use name to identify this UCX context. + |
struct ucp_tag_recv_info | +
The UCP receive information descriptor is allocated by application and filled in with the information about the received message by ucp_tag_probe_nb or ucp_tag_recv_request_test routines or ucp_tag_recv_callback_t callback argument.
+Data Fields | ||
---|---|---|
+ucp_tag_t | ++sender_tag | +
+ Sender tag + |
+size_t | ++length | +
+ The size of the received data + |
struct ucp_request_param_t | +
The structure ucp_request_param_t is used to specify datatype of operation, provide user request in case the external request is used, set completion callback and custom user data passed to this callback.
+Example: implementation of function to send contiguous buffer to ep and invoke callback function at operation completion. If the operation completed immediately (status == UCS_OK) then callback is not called.
+Data Fields | ||
---|---|---|
+uint32_t | ++op_attr_mask | +
+ Mask of valid fields in this structure and operation flags, using bits from ucp_op_attr_t. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+uint32_t | ++flags | ++ |
+void * | ++request | +
+ Request handle allocated by the user. There should be at least UCP request size bytes of available space before the request. The size of the UCP request can be obtained by ucp_context_query function. + |
+union ucp_request_param_t | ++cb | +
+ Callback function that is invoked whenever the send or receive operation is completed. + |
+ucp_datatype_t | ++datatype | +
+ Datatype descriptor for the elements in the buffer. In case the op_attr_mask & UCP_OP_ATTR_FIELD_DATATYPE bit is not set, then use default datatype ucp_dt_make_contig(1) + |
+void * | ++user_data | +
+ Pointer to user data passed to callback function. + |
+void * | ++reply_buffer | +
+ Reply buffer. Can be used for storing operation result, for example by ucp_atomic_op_nbx. + |
+ucs_memory_type_t | ++memory_type | +
+ Memory type of the buffer. see ucs_memory_type_t for possible memory types. An optimization hint to avoid memory type detection for request buffer. If this value is not set (along with its corresponding bit in the op_attr_mask - UCP_OP_ATTR_FIELD_MEMORY_TYPE), then use default UCS_MEMORY_TYPE_UNKNOWN which means the memory type will be detected internally. + |
+union ucp_request_param_t | ++recv_info | +
+ Pointer to the information where received data details are stored in case of an immediate completion of receive operation. The user has to provide a pointer to valid memory/variable which will be updated on function return. + |
+ucp_mem_h | ++memh | +
+ Memory handle for pre-registered buffer. If the handle is provided, protocols that require registered memory can skip the registration step. As a result, the communication request overhead can be reduced and the request can be completed faster. The memory handle should be obtained by calling ucp_mem_map. + |
union ucp_request_param_t.cb | +
Callback function that is invoked whenever the send or receive operation is completed.
+Data Fields | ||
---|---|---|
+ucp_send_nbx_callback_t | ++send | ++ |
+ucp_tag_recv_nbx_callback_t | ++recv | ++ |
+ucp_stream_recv_nbx_callback_t | ++recv_stream | ++ |
+ucp_am_recv_data_nbx_callback_t | ++recv_am | ++ |
union ucp_request_param_t.recv_info | +
Pointer to the information where received data details are stored in case of an immediate completion of receive operation. The user has to provide a pointer to valid memory/variable which will be updated on function return.
+Data Fields | ||
---|---|---|
+size_t * | ++length | ++ |
+ucp_tag_recv_info_t * | ++tag_info | ++ |
#define UCP_ENTITY_NAME_MAX 32 | +
typedef struct ucp_lib_attr ucp_lib_attr_t | +
The structure defines the attributes that characterize the Library.
+ +typedef struct ucp_context_attr ucp_context_attr_t | +
The structure defines the attributes that characterize the particular context.
+ +typedef struct ucp_tag_recv_info ucp_tag_recv_info_t | +
The UCP receive information descriptor is allocated by application and filled in with the information about the received message by ucp_tag_probe_nb or ucp_tag_recv_request_test routines or ucp_tag_recv_callback_t callback argument.
+ +typedef struct ucp_context* ucp_context_h | +
UCP application context (or just a context) is an opaque handle that holds a UCP communication instance's global information. It represents a single UCP communication instance. The communication instance could be an OS process (an application) that uses UCP library. This global information includes communication resources, endpoints, memory, temporary file storage, and other communication information directly associated with a specific UCP instance. The context also acts as an isolation mechanism, allowing resources associated with the context to manage multiple concurrent communication instances. For example, users using both MPI and OpenSHMEM sessions simultaneously can isolate their communication by allocating and using separate contexts for each of them. Alternatively, users can share the communication resources (memory, network resource context, etc.) between them by using the same application context. A message sent or a RMA operation performed in one application context cannot be received in any other application context.
+ +typedef void(* ucp_request_init_callback_t)(void *request) | +
This callback routine is responsible for the request initialization.
+[in] | request | Request handle to initialize. |
typedef void(* ucp_request_cleanup_callback_t)(void *request) | +
This callback routine is responsible for cleanup of the memory associated with the request.
+[in] | request | Request handle to cleanup. |
enum ucp_params_field | +
The enumeration allows specifying which fields in ucp_params_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_PARAM_FIELD_FEATURES |
+ features + |
UCP_PARAM_FIELD_REQUEST_SIZE |
+ request_size + |
UCP_PARAM_FIELD_REQUEST_INIT |
+ request_init + |
UCP_PARAM_FIELD_REQUEST_CLEANUP |
+ request_cleanup + |
UCP_PARAM_FIELD_TAG_SENDER_MASK |
+ tag_sender_mask + |
UCP_PARAM_FIELD_MT_WORKERS_SHARED |
+ mt_workers_shared + |
UCP_PARAM_FIELD_ESTIMATED_NUM_EPS |
+ estimated_num_eps + |
UCP_PARAM_FIELD_ESTIMATED_NUM_PPN |
+ estimated_num_ppn + |
UCP_PARAM_FIELD_NAME |
+ name + |
enum ucp_feature | +
The enumeration list describes the features supported by UCP. An application can request the features using UCP parameters during UCP initialization process.
+Enumerator | |
---|---|
UCP_FEATURE_TAG |
+ Request tag matching support + |
UCP_FEATURE_RMA |
+ Request remote memory access support + |
UCP_FEATURE_AMO32 |
+ Request 32-bit atomic operations support + |
UCP_FEATURE_AMO64 |
+ Request 64-bit atomic operations support + |
UCP_FEATURE_WAKEUP |
+ Request interrupt notification support + |
UCP_FEATURE_STREAM |
+ Request stream support + |
UCP_FEATURE_AM |
+ Request Active Message support + |
UCP_FEATURE_EXPORTED_MEMH |
+ Request support mapping a peer's memory handle that was created by ucp_mem_map and packed by ucp_memh_pack with the flag UCP_MEMH_PACK_FLAG_EXPORT and use it for local operations + |
enum ucp_lib_attr_field | +
The enumeration allows specifying which fields in ucp_lib_attr_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_LIB_ATTR_FIELD_MAX_THREAD_LEVEL |
+ UCP library maximum supported thread level flag + |
enum ucp_context_attr_field | +
The enumeration allows specifying which fields in ucp_context_attr_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_ATTR_FIELD_REQUEST_SIZE |
+ UCP request size + |
UCP_ATTR_FIELD_THREAD_MODE |
+ UCP context thread flag + |
UCP_ATTR_FIELD_MEMORY_TYPES |
+ UCP supported memory types + |
UCP_ATTR_FIELD_NAME |
+ UCP context name + |
ucs_status_t ucp_lib_query | +( | +ucp_lib_attr_t * | +attr | ) | ++ |
This routine fetches information about the UCP library attributes.
+[out] | attr | Filled with attributes of the UCP library. |
void ucp_get_version | +( | +unsigned * | +major_version, | +
+ | + | unsigned * | +minor_version, | +
+ | + | unsigned * | +release_number | +
+ | ) | ++ |
This routine returns the UCP library version.
+[out] | major_version | Filled with library major version. |
[out] | minor_version | Filled with library minor version. |
[out] | release_number | Filled with library release number. |
const char* ucp_get_version_string | +( | +void | +) | ++ |
This routine returns the UCP library version as a string which consists of: "major.minor.release".
+ +
+
|
+ +inlinestatic | +
This routine creates and initializes a UCP application context.
+This routine checks API version compatibility, then discovers the available network interfaces, and initializes the network resources required for discovering of the network and memory related devices. This routine is responsible for initialization all information required for a particular application scope, for example, MPI application, OpenSHMEM application, etc.
+[in] | config | UCP configuration descriptor allocated through ucp_config_read() routine. |
[in] | params | User defined ucp_params_t configurations for the UCP application context. |
[out] | context_p | Initialized UCP application context. |
void ucp_cleanup | +( | +ucp_context_h | +context_p | ) | ++ |
This routine finalizes and releases the resources associated with a UCP application context.
+The cleanup process releases and shuts down all resources associated with the application context. After calling this routine, calling any UCP routine without calling UCP initialization routine is invalid.
+[in] | context_p | Handle to UCP application context. |
ucs_status_t ucp_context_query | +( | +ucp_context_h | +context_p, | +
+ | + | ucp_context_attr_t * | +attr | +
+ | ) | ++ |
This routine fetches information about the context.
+[in] | context_p | Handle to UCP application context. |
[out] | attr | Filled with attributes of context_p context. |
void ucp_context_print_info | +( | +const ucp_context_h | +context, | +
+ | + | FILE * | +stream | +
+ | ) | ++ |
This routine prints information about the context configuration: including memory domains, transport resources, and other useful information associated with the context.
+[in] | context | Print this context object's configuration. |
[in] | stream | Output stream on which to print the information. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | ucp_dt_iov |
Structure for scatter-gather I/O. More... | |
struct | ucp_generic_dt_ops |
UCP generic data type descriptor. More... | |
struct | ucp_datatype_attr |
UCP datatype attributes. More... | |
+Macros | |
#define | ucp_dt_make_contig(_elem_size) (((ucp_datatype_t)(_elem_size) << UCP_DATATYPE_SHIFT) | UCP_DATATYPE_CONTIG) |
Generate an identifier for contiguous data type. More... | |
#define | ucp_dt_make_iov() ((ucp_datatype_t)UCP_DATATYPE_IOV) |
Generate an identifier for Scatter-gather IOV data type. More... | |
+Typedefs | |
typedef struct ucp_dt_iov | ucp_dt_iov_t |
Structure for scatter-gather I/O. More... | |
typedef struct ucp_generic_dt_ops | ucp_generic_dt_ops_t |
UCP generic data type descriptor. More... | |
typedef struct ucp_datatype_attr | ucp_datatype_attr_t |
UCP datatype attributes. More... | |
+Enumerations | |
enum | ucp_dt_type { + UCP_DATATYPE_CONTIG = 0, +UCP_DATATYPE_STRIDED = 1, +UCP_DATATYPE_IOV = 2, +UCP_DATATYPE_GENERIC = 7, + + UCP_DATATYPE_SHIFT = 3, +UCP_DATATYPE_CLASS_MASK = UCS_MASK(UCP_DATATYPE_SHIFT) + + } |
UCP data type classification. More... | |
enum | ucp_datatype_attr_field { UCP_DATATYPE_ATTR_FIELD_PACKED_SIZE = UCS_BIT(0), +UCP_DATATYPE_ATTR_FIELD_BUFFER = UCS_BIT(1), +UCP_DATATYPE_ATTR_FIELD_COUNT = UCS_BIT(2) + } |
UCP datatype attributes field mask. More... | |
+Functions | |
ucs_status_t | ucp_dt_create_generic (const ucp_generic_dt_ops_t *ops, void *context, ucp_datatype_t *datatype_p) |
Create a generic datatype. More... | |
void | ucp_dt_destroy (ucp_datatype_t datatype) |
Destroy a datatype and release its resources. More... | |
ucs_status_t | ucp_dt_query (ucp_datatype_t datatype, ucp_datatype_attr_t *attr) |
Query attributes of a datatype. More... | |
+Variables | |
void *(* | ucp_generic_dt_ops::start_pack )(void *context, const void *buffer, size_t count) |
Start a packing request. More... | |
void *(* | ucp_generic_dt_ops::start_unpack )(void *context, void *buffer, size_t count) |
Start an unpacking request. More... | |
size_t(* | ucp_generic_dt_ops::packed_size )(void *state) |
Get the total size of packed data. More... | |
size_t(* | ucp_generic_dt_ops::pack )(void *state, size_t offset, void *dest, size_t max_length) |
Pack data. More... | |
ucs_status_t(* | ucp_generic_dt_ops::unpack )(void *state, size_t offset, const void *src, size_t length) |
Unpack data. More... | |
void(* | ucp_generic_dt_ops::finish )(void *state) |
Finish packing/unpacking. More... | |
UCP Data type routines
+struct ucp_dt_iov | +
This structure is used to specify a list of buffers which can be used within a single data transfer function call. This list should remain valid until the data transfer request is completed.
+Data Fields | ||
---|---|---|
+void * | ++buffer | +
+ Pointer to a data buffer + |
+size_t | ++length | +
+ Length of the buffer in bytes + |
struct ucp_datatype_attr | +
This structure provides attributes of a UCP datatype.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_datatype_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+size_t | ++packed_size | +
+ Packed size of the given datatype. (output parameter) + |
+const void * | ++buffer | +
+ Pointer to a data buffer of the associated data type. This field is optional. If UCP_DATATYPE_ATTR_FIELD_BUFFER is not set in field_mask, this field defaults to NULL. + |
+size_t | ++count | +
+ Number of elements in buffer. This value is optional. If UCP_DATATYPE_ATTR_FIELD_COUNT is not set in field_mask, the value of this field defaults to 1. + |
#define ucp_dt_make_contig | +( | ++ | _elem_size | ) | +(((ucp_datatype_t)(_elem_size) << UCP_DATATYPE_SHIFT) | UCP_DATATYPE_CONTIG) | +
This macro creates an identifier for contiguous datatype that is defined by the size of the basic element.
+[in] | _elem_size | Size of the basic element of the type. |
#define ucp_dt_make_iov | +( | +) | +((ucp_datatype_t)UCP_DATATYPE_IOV) | +
This macro creates an identifier for datatype of scatter-gather list with multiple pointers
+typedef struct ucp_dt_iov ucp_dt_iov_t | +
This structure is used to specify a list of buffers which can be used within a single data transfer function call. This list should remain valid until the data transfer request is completed.
+typedef struct ucp_generic_dt_ops ucp_generic_dt_ops_t | +
This structure provides a generic datatype descriptor that is used for definition of application defined datatypes.
+Typically, the descriptor is used for an integration with datatype engines implemented within MPI and SHMEM implementations.
+typedef struct ucp_datatype_attr ucp_datatype_attr_t | +
This structure provides attributes of a UCP datatype.
+ +enum ucp_dt_type | +
The enumeration list describes the datatypes supported by UCP.
+Enumerator | |
---|---|
UCP_DATATYPE_CONTIG |
+ Contiguous datatype + |
UCP_DATATYPE_STRIDED |
+ Strided datatype + |
UCP_DATATYPE_IOV |
+ Scatter-gather list with multiple pointers + |
UCP_DATATYPE_GENERIC |
+ Generic datatype with user-defined pack/unpack routines + |
UCP_DATATYPE_SHIFT |
+ Number of bits defining the datatype classification + |
UCP_DATATYPE_CLASS_MASK |
+ Data-type class mask + |
enum ucp_datatype_attr_field | +
The enumeration allows specifying which fields in ucp_datatype_attr_t are present and which datatype attributes are queried.
+Enumerator | |
---|---|
UCP_DATATYPE_ATTR_FIELD_PACKED_SIZE |
+ ucp_datatype_attr_t::packed_size field is queried. + |
UCP_DATATYPE_ATTR_FIELD_BUFFER |
+ ucp_datatype_attr_t::buffer field is set. + |
UCP_DATATYPE_ATTR_FIELD_COUNT |
+ ucp_datatype_attr_t::count field is set. + |
ucs_status_t ucp_dt_create_generic | +( | +const ucp_generic_dt_ops_t * | +ops, | +
+ | + | void * | +context, | +
+ | + | ucp_datatype_t * | +datatype_p | +
+ | ) | ++ |
This routine create a generic datatype object. The generic datatype is described by the ops object which provides a table of routines defining the operations for generic datatype manipulation. Typically, generic datatypes are used for integration with datatype engines provided with MPI implementations (MPICH, Open MPI, etc). The application is responsible for releasing the datatype_p object using ucp_dt_destroy() routine.
+[in] | ops | Generic datatype function table as defined by ucp_generic_dt_ops_t . |
[in] | context | Application defined context passed to this routine. The context is passed as a parameter to the routines in the ops table. |
[out] | datatype_p | A pointer to datatype object. |
void ucp_dt_destroy | +( | +ucp_datatype_t | +datatype | ) | ++ |
This routine destroys the datatype object and releases any resources that are associated with the object. The datatype object must be allocated using ucp_dt_create_generic() routine.
+[in] | datatype | Datatype object to destroy. |
ucs_status_t ucp_dt_query | +( | +ucp_datatype_t | +datatype, | +
+ | + | ucp_datatype_attr_t * | +attr | +
+ | ) | ++ |
This routine fetches information about the attributes of a datatype. When UCP_DATATYPE_ATTR_FIELD_PACKED_SIZE is set in field_mask of attr, the field packed_size is set to the packed size (bytes) of the datatype.
+[in] | datatype | Datatype object to query. |
[in,out] | attr | Filled with attributes of the datatype. |
void*(* ucp_generic_dt_ops::start_pack)(void *context, const void *buffer, size_t count) | +
The pointer refers to application defined start-to-pack routine. It will be called from the ucp_tag_send_nb routine.
+[in] | context | User-defined context. |
[in] | buffer | Buffer to pack. |
[in] | count | Number of elements to pack into the buffer. |
void*(* ucp_generic_dt_ops::start_unpack)(void *context, void *buffer, size_t count) | +
The pointer refers to application defined start-to-unpack routine. It will be called from the ucp_tag_recv_nb routine.
+[in] | context | User-defined context. |
[in] | buffer | Buffer to unpack to. |
[in] | count | Number of elements to unpack in the buffer. |
size_t(* ucp_generic_dt_ops::packed_size)(void *state) | +
The pointer refers to user defined routine that returns the size of data in a packed format.
+[in] | state | State as returned by start_pack() routine. |
size_t(* ucp_generic_dt_ops::pack)(void *state, size_t offset, void *dest, size_t max_length) | +
The pointer refers to application defined pack routine.
+[in] | state | State as returned by start_pack() routine. |
[in] | offset | Virtual offset in the output stream. |
[in] | dest | Destination buffer to pack the data. |
[in] | max_length | Maximum length to pack. |
ucs_status_t(* ucp_generic_dt_ops::unpack)(void *state, size_t offset, const void *src, size_t length) | +
The pointer refers to application defined unpack routine.
+[in] | state | State as returned by start_unpack() routine. |
[in] | offset | Virtual offset in the input stream. |
[in] | src | Source to unpack the data from. |
[in] | length | Length to unpack. |
void(* ucp_generic_dt_ops::finish)(void *state) | +
The pointer refers to application defined finish routine.
+[in] | state | State as returned by start_pack() and start_unpack() routines. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | ucp_ep_evaluate_perf_param_t |
UCP endpoint performance evaluation request attributes. More... | |
struct | ucp_ep_evaluate_perf_attr_t |
UCP endpoint performance evaluation result attributes. More... | |
struct | ucp_stream_poll_ep |
Output parameter of ucp_stream_worker_poll function. More... | |
struct | ucp_ep_attr |
UCP endpoint attributes. More... | |
struct | ucp_ep_params |
Tuning parameters for the UCP endpoint. More... | |
struct | ucp_transport_entry_t |
The ucp_transports_t and ucp_transport_entry_t structures are used when ucp_ep_query is called to return an array of transport name and device name pairs that are used by an active endpoint. More... | |
struct | ucp_transports_t |
Structure containing an array of transport layers and device names used by an endpoint. More... | |
+Typedefs | |
typedef enum +ucp_ep_perf_param_field | ucp_ep_perf_param_field_t |
UCP performance fields and flags. More... | |
typedef enum ucp_ep_perf_attr_field | ucp_ep_perf_attr_field_t |
UCP performance fields and flags. More... | |
typedef struct ucp_stream_poll_ep | ucp_stream_poll_ep_t |
Output parameter of ucp_stream_worker_poll function. More... | |
typedef struct ucp_ep_attr | ucp_ep_attr_t |
UCP endpoint attributes. More... | |
typedef struct ucp_ep * | ucp_ep_h |
UCP Endpoint. More... | |
typedef struct ucp_conn_request * | ucp_conn_request_h |
UCP connection request. More... | |
typedef ucs_status_t(* | ucp_am_callback_t )(void *arg, void *data, size_t length, ucp_ep_h reply_ep, unsigned flags) |
Callback to process incoming Active Message. More... | |
typedef ucs_status_t(* | ucp_am_recv_callback_t )(void *arg, const void *header, size_t header_length, void *data, size_t length, const ucp_am_recv_param_t *param) |
Callback to process incoming Active Message sent by ucp_am_send_nbx routine. More... | |
typedef struct ucp_ep_params | ucp_ep_params_t |
Tuning parameters for the UCP endpoint. More... | |
+Enumerations | |
enum | ucp_ep_params_field { + UCP_EP_PARAM_FIELD_REMOTE_ADDRESS = UCS_BIT(0), +UCP_EP_PARAM_FIELD_ERR_HANDLING_MODE = UCS_BIT(1), +UCP_EP_PARAM_FIELD_ERR_HANDLER = UCS_BIT(2), +UCP_EP_PARAM_FIELD_USER_DATA = UCS_BIT(3), + + UCP_EP_PARAM_FIELD_SOCK_ADDR = UCS_BIT(4), +UCP_EP_PARAM_FIELD_FLAGS = UCS_BIT(5), +UCP_EP_PARAM_FIELD_CONN_REQUEST = UCS_BIT(6), +UCP_EP_PARAM_FIELD_NAME = UCS_BIT(7), + + UCP_EP_PARAM_FIELD_LOCAL_SOCK_ADDR = UCS_BIT(8) + + } |
UCP endpoint parameters field mask. More... | |
enum | ucp_ep_params_flags_field { UCP_EP_PARAMS_FLAGS_CLIENT_SERVER = UCS_BIT(0), +UCP_EP_PARAMS_FLAGS_NO_LOOPBACK = UCS_BIT(1), +UCP_EP_PARAMS_FLAGS_SEND_CLIENT_ID = UCS_BIT(2) + } |
UCP endpoint parameters flags. More... | |
enum | ucp_ep_close_flags_t { UCP_EP_CLOSE_FLAG_FORCE = UCS_BIT(0) + } |
Close UCP endpoint modes. More... | |
enum | ucp_ep_perf_param_field { UCP_EP_PERF_PARAM_FIELD_MESSAGE_SIZE = UCS_BIT(0) + } |
UCP performance fields and flags. More... | |
enum | ucp_ep_perf_attr_field { UCP_EP_PERF_ATTR_FIELD_ESTIMATED_TIME = UCS_BIT(0) + } |
UCP performance fields and flags. More... | |
enum | ucp_cb_param_flags { UCP_CB_PARAM_FLAG_DATA = UCS_BIT(0) + } |
Descriptor flags for Active Message callback. More... | |
enum | ucp_ep_attr_field { UCP_EP_ATTR_FIELD_NAME = UCS_BIT(0), +UCP_EP_ATTR_FIELD_LOCAL_SOCKADDR = UCS_BIT(1), +UCP_EP_ATTR_FIELD_REMOTE_SOCKADDR = UCS_BIT(2), +UCP_EP_ATTR_FIELD_TRANSPORTS = UCS_BIT(3) + } |
UCP endpoint attributes field mask. More... | |
enum | ucp_ep_close_mode { UCP_EP_CLOSE_MODE_FORCE = 0, +UCP_EP_CLOSE_MODE_FLUSH = 1 + } |
Close UCP endpoint modes. More... | |
enum | ucp_err_handling_mode_t { UCP_ERR_HANDLING_MODE_NONE, +UCP_ERR_HANDLING_MODE_PEER + } |
Error handling mode for the UCP endpoint. More... | |
+Functions | |
ucs_status_t | ucp_ep_create (ucp_worker_h worker, const ucp_ep_params_t *params, ucp_ep_h *ep_p) |
Create and connect an endpoint. More... | |
ucs_status_ptr_t | ucp_ep_close_nbx (ucp_ep_h ep, const ucp_request_param_t *param) |
Non-blocking endpoint closure. More... | |
void | ucp_ep_print_info (ucp_ep_h ep, FILE *stream) |
Print endpoint information. More... | |
ucs_status_ptr_t | ucp_ep_flush_nbx (ucp_ep_h ep, const ucp_request_param_t *param) |
Non-blocking flush of outstanding AMO and RMA operations on the endpoint. More... | |
ucs_status_t | ucp_ep_evaluate_perf (ucp_ep_h ep, const ucp_ep_evaluate_perf_param_t *param, ucp_ep_evaluate_perf_attr_t *attr) |
Estimate performance characteristics of a specific endpoint. More... | |
ucs_status_t | ucp_ep_query (ucp_ep_h ep, ucp_ep_attr_t *attr) |
Get attributes of a given endpoint. More... | |
void | ucp_request_release (void *request) |
void | ucp_ep_destroy (ucp_ep_h ep) |
ucs_status_ptr_t | ucp_disconnect_nb (ucp_ep_h ep) |
ucs_status_t | ucp_request_test (void *request, ucp_tag_recv_info_t *info) |
ucs_status_t | ucp_ep_flush (ucp_ep_h ep) |
ucs_status_ptr_t | ucp_ep_modify_nb (ucp_ep_h ep, const ucp_ep_params_t *params) |
Modify endpoint parameters. More... | |
ucs_status_ptr_t | ucp_ep_close_nb (ucp_ep_h ep, unsigned mode) |
Non-blocking endpoint closure. More... | |
ucs_status_ptr_t | ucp_ep_flush_nb (ucp_ep_h ep, unsigned flags, ucp_send_callback_t cb) |
Non-blocking flush of outstanding AMO and RMA operations on the endpoint. More... | |
UCP Endpoint routines
+struct ucp_ep_evaluate_perf_param_t | +
The structure defines the attributes which characterize the request for performance estimation of a particular endpoint.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_ep_perf_param_field_t. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+size_t | ++message_size | +
+ Message size to use for determining performance. This field must be initialized by the caller. + |
struct ucp_ep_evaluate_perf_attr_t | +
The structure defines the attributes which characterize the result of performance estimation of a particular endpoint.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_ep_perf_attr_field_t. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+double | ++estimated_time | +
+ Estimated time (in seconds) required to send a message of a given size on this endpoint. This field is set by the ucp_ep_evaluate_perf function. + |
struct ucp_stream_poll_ep | +
The structure defines the endpoint and its user data.
+Data Fields | ||
---|---|---|
+ucp_ep_h | ++ep | +
+ Endpoint handle. + |
+void * | ++user_data | +
+ User data associated with an endpoint passed in ucp_ep_params_t::user_data. + |
+unsigned | ++flags | +
+ Reserved for future use. + |
+uint8_t | ++reserved[16] | +
+ Reserved for future use. + |
struct ucp_ep_attr | +
The structure defines the attributes that characterize the particular endpoint.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_ep_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+char | ++name[UCP_ENTITY_NAME_MAX] | +
+ Endpoint name. Tracing and analysis tools can identify the endpoint using this name. + |
+struct sockaddr_storage | ++local_sockaddr | +
+ Local socket address for this endpoint. Valid only for endpoints created by connecting to a socket address. If this field is specified for an endpoint not connected to a socket address, UCS_ERR_NOT_CONNECTED will be returned. + |
+struct sockaddr_storage | ++remote_sockaddr | +
+ Remote socket address this endpoint is connected to. Valid only for endpoints created by connecting to a socket address. If this field is specified for an endpoint not connected to a socket address, UCS_ERR_NOT_CONNECTED will be returned. + |
+ucp_transports_t | ++transports | +
+ Structure defining an array containing transport and device names used by this endpoint. The caller is responsible for allocation and deallocation of this array. + |
struct ucp_ep_params | +
The structure defines the parameters that are used for the UCP endpoint tuning during the UCP ep creation.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_ep_params_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+const ucp_address_t * | ++address | +
+ Destination address; this field should be set along with its corresponding bit in the field_mask - UCP_EP_PARAM_FIELD_REMOTE_ADDRESS and must be obtained using ucp_worker_get_address. + |
+ucp_err_handling_mode_t | ++err_mode | +
+ Desired error handling mode, optional parameter. Default value is UCP_ERR_HANDLING_MODE_NONE. + |
+ucp_err_handler_t | ++err_handler | +
+ Handler to process transport level failure. + |
+void * | ++user_data | +
+ User data associated with an endpoint. See ucp_stream_poll_ep_t and ucp_err_handler_t + |
+unsigned | ++flags | +
+ Endpoint flags from ucp_ep_params_flags_field. This value is optional. If it's not set (along with its corresponding bit in the field_mask - UCP_EP_PARAM_FIELD_FLAGS), the ucp_ep_create() routine will consider the flags as set to zero. + |
+ucs_sock_addr_t | ++sockaddr | +
+ Destination address in the form of a sockaddr; this field should be set along with its corresponding bit in the field_mask - UCP_EP_PARAM_FIELD_SOCK_ADDR and must be obtained from the user, it means that this type of the endpoint creation is possible only on client side in client-server connection establishment flow. + |
+ucp_conn_request_h | ++conn_request | +
+ Connection request from client; this field should be set along with its corresponding bit in the field_mask - UCP_EP_PARAM_FIELD_CONN_REQUEST and must be obtained from ucp_listener_conn_callback_t, it means that this type of the endpoint creation is possible only on server side in client-server connection establishment flow. + |
+const char * | ++name | +
+ Endpoint name. Tracing and analysis tools can identify the endpoint using this name. To retrieve the endpoint's name, use ucp_ep_query, as the name you supply may be changed by UCX under some circumstances, e.g. a name conflict. This field is only assigned if you set UCP_EP_PARAM_FIELD_NAME in the field mask. If not, then a default unique name will be created for you. + |
+ucs_sock_addr_t | ++local_sockaddr | +
+ The sockaddr to bind locally. Specifies the associated network device to bind locally to establish new connections. To retrieve the endpoint's local_sockaddr, use ucp_ep_query. This setting is optional. To enable it, the corresponding - UCP_EP_PARAM_FIELD_LOCAL_SOCK_ADDR bit in the field mask must be set. + |
struct ucp_transport_entry_t | +
The ucp_transport_t structure specifies the characteristics of the ucp_transport_entry_t array.
+The caller is responsible for the allocation and de-allocation of the ucp_transport_entry_t array.
+Example: Implementation of a function to query the set of transport and device name pairs used by the specified endpoint.
+A transport name and device name pair used by this endpoint. The caller is responsible for the allocation and deallocation of an array of these structures large enough to contain the desired number of transport and device name pairs.
+Any new fields must be added to the end of this structure.
+Data Fields | ||
---|---|---|
+const char * | ++transport_name | +
+ The name of a transport layer used by this endpoint. This '\0'-terminated string is valid until the endpoint is closed using a ucp_ep_close_nbx call. + |
+const char * | ++device_name | +
+ The name of the device used with this transport by this endpoint. This '\0'-terminated string is valid until the endpoint is closed using a ucp_ep_close_nbx call. + |
struct ucp_transports_t | +
The caller is responsible for allocation and deallocation of this structure.
+Data Fields | ||
---|---|---|
+ucp_transport_entry_t * | ++entries | +
+ Pointer to array of transport/device name pairs used by this endpoint. The caller is responsible for the allocation and deallocation of this array. + |
+unsigned | ++num_entries | +
+ Number of transport/device name pairs. The caller must set this to the maximum number of pairs the structure can contain. On return, this is set to the actual number of transport and device name pairs used by the endpoint. + |
+size_t | ++entry_size | +
+ Size of a single ucp_transport_entry_t object. The caller sets this to the size of the ucp_transport_entry_t they are using. UCP code must not set any fields in the ucp_transport_entry_t structure beyond this size. + |
typedef enum ucp_ep_perf_param_field ucp_ep_perf_param_field_t | +
The enumeration allows specifying which fields in ucp_ep_evaluate_perf_param_t are present and operation flags are used. It is used to enable backward compatibility support.
+ +typedef enum ucp_ep_perf_attr_field ucp_ep_perf_attr_field_t | +
The enumeration allows specifying which fields in ucp_ep_evaluate_perf_attr_t are present and operation flags are used. It is used to enable backward compatibility support.
+ +typedef struct ucp_stream_poll_ep ucp_stream_poll_ep_t | +
The structure defines the endpoint and its user data.
+ +typedef struct ucp_ep_attr ucp_ep_attr_t | +
The structure defines the attributes that characterize the particular endpoint.
+ +typedef struct ucp_ep* ucp_ep_h | +
The endpoint handle is an opaque object that is used to address a remote worker. It typically provides a description of source, destination, or both. All UCP communication routines address a destination with the endpoint handle. The endpoint handle is associated with only one UCP context. UCP provides the endpoint create routine to create the endpoint handle and the destroy routine to destroy the endpoint handle.
+ +typedef struct ucp_conn_request* ucp_conn_request_h | +
A server-side handle to incoming connection request. Can be used to create an endpoint which connects back to the client.
+ +typedef ucs_status_t(* ucp_am_callback_t)(void *arg, void *data, size_t length, ucp_ep_h reply_ep, unsigned flags) | +
When the callback is called, flags indicates how data should be handled.
+[in] | arg | User-defined argument. |
[in] | data | Points to the received data. This data may persist after the callback returns and needs to be freed with ucp_am_data_release. |
[in] | length | Length of data. |
[in] | reply_ep | If the Active Message is sent with the UCP_AM_SEND_FLAG_REPLY flag, the sending ep will be passed in. If not, NULL will be passed. |
[in] | flags | If this flag is set to UCP_CB_PARAM_FLAG_DATA, the callback can return UCS_INPROGRESS and data will persist after the callback returns. |
typedef ucs_status_t(* ucp_am_recv_callback_t)(void *arg, const void *header, size_t header_length, void *data, size_t length, const ucp_am_recv_param_t *param) | +
The callback is always called from the progress context, therefore calling ucp_worker_progress() is not allowed. It is recommended to define callbacks with relatively short execution time to avoid blocking of communication progress.
+[in] | arg | User-defined argument. |
[in] | header | User defined active message header. If header_length is 0, this value is undefined and must not be accessed. |
[in] | header_length | Active message header length in bytes. |
[in] | data | Points to the received data if UCP_AM_RECV_ATTR_FLAG_RNDV flag is not set in ucp_am_recv_param_t::recv_attr. Otherwise it points to the internal UCP descriptor which can further be used for initiating data receive by using ucp_am_recv_data_nbx routine. |
[in] | length | Length of data. If UCP_AM_RECV_ATTR_FLAG_RNDV flag is set in ucp_am_recv_param_t::recv_attr, it indicates the required receive buffer size for initiating rendezvous protocol. |
[in] | param | Data receive parameters. |
typedef struct ucp_ep_params ucp_ep_params_t | +
The structure defines the parameters that are used for the UCP endpoint tuning during the UCP ep creation.
+ +enum ucp_ep_params_field | +
The enumeration allows specifying which fields in ucp_ep_params_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_EP_PARAM_FIELD_REMOTE_ADDRESS |
+ Address of remote peer + |
UCP_EP_PARAM_FIELD_ERR_HANDLING_MODE |
+ Error handling mode. ucp_err_handling_mode_t + |
UCP_EP_PARAM_FIELD_ERR_HANDLER |
+ Handler to process transport level errors + |
UCP_EP_PARAM_FIELD_USER_DATA |
+ User data pointer + |
UCP_EP_PARAM_FIELD_SOCK_ADDR |
+ Socket address field + |
UCP_EP_PARAM_FIELD_FLAGS |
+ Endpoint flags Connection request field + |
UCP_EP_PARAM_FIELD_CONN_REQUEST | + |
UCP_EP_PARAM_FIELD_NAME |
+ Endpoint name + |
UCP_EP_PARAM_FIELD_LOCAL_SOCK_ADDR |
+ Local socket Address + |
enum ucp_ep_params_flags_field | +
The enumeration list describes the endpoint's parameters flags supported by ucp_ep_create() function.
+Enumerator | |
---|---|
UCP_EP_PARAMS_FLAGS_CLIENT_SERVER |
+ Using a client-server connection establishment mechanism. ucs_sock_addr_t sockaddr field must be provided and contain the address of the remote peer + |
UCP_EP_PARAMS_FLAGS_NO_LOOPBACK |
+ Avoid connecting the endpoint to itself when connecting the endpoint to the same worker it was created on. Affects protocols which send to a particular remote endpoint, for example stream + |
UCP_EP_PARAMS_FLAGS_SEND_CLIENT_ID |
+ Send client id when connecting to remote socket address as part of the connection request payload. On the remote side value can be obtained from ucp_conn_request_h using ucp_conn_request_query + |
enum ucp_ep_close_flags_t | +
The enumeration is used to specify the behavior of ucp_ep_close_nbx.
+Enumerator | |
---|---|
UCP_EP_CLOSE_FLAG_FORCE |
+ ucp_ep_close_nbx releases the endpoint without any confirmation from the peer. All outstanding requests will be completed with UCS_ERR_CANCELED error. +
|
enum ucp_ep_perf_param_field | +
The enumeration allows specifying which fields in ucp_ep_evaluate_perf_param_t are present and operation flags are used. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_EP_PERF_PARAM_FIELD_MESSAGE_SIZE | + + |
enum ucp_ep_perf_attr_field | +
The enumeration allows specifying which fields in ucp_ep_evaluate_perf_attr_t are present and operation flags are used. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_EP_PERF_ATTR_FIELD_ESTIMATED_TIME | + + |
enum ucp_cb_param_flags | +
In a callback, if flags is set to UCP_CB_PARAM_FLAG_DATA in a callback then data was allocated, so if UCS_INPROGRESS is returned from the callback, the data parameter will persist and the user has to call ucp_am_data_release when data is no longer needed.
+Enumerator | |
---|---|
UCP_CB_PARAM_FLAG_DATA | + |
enum ucp_ep_attr_field | +
The enumeration allows specifying which fields in ucp_ep_attr_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_EP_ATTR_FIELD_NAME |
+ UCP endpoint name + |
UCP_EP_ATTR_FIELD_LOCAL_SOCKADDR |
+ Sockaddr used by the endpoint + |
UCP_EP_ATTR_FIELD_REMOTE_SOCKADDR |
+ Sockaddr the endpoint is connected to + |
UCP_EP_ATTR_FIELD_TRANSPORTS |
+ Transport and device used by endpoint + |
enum ucp_ep_close_mode | +
The enumeration is used to specify the behavior of ucp_ep_close_nb.
+Enumerator | |
---|---|
UCP_EP_CLOSE_MODE_FORCE |
+ ucp_ep_close_nb releases the endpoint without any confirmation from the peer. All outstanding requests will be completed with UCS_ERR_CANCELED error. +
|
UCP_EP_CLOSE_MODE_FLUSH |
+ ucp_ep_close_nb schedules flushes on all outstanding operations. + |
enum ucp_err_handling_mode_t | +
Specifies error handling mode for the UCP endpoint.
+Enumerator | |
---|---|
UCP_ERR_HANDLING_MODE_NONE |
+ No guarantees about error reporting, imposes minimal overhead from a performance perspective. +
|
UCP_ERR_HANDLING_MODE_PEER |
+ Guarantees that send requests are always completed (successfully or error) even in case of remote failure, disables protocols and APIs which may cause a hang or undefined behavior in case of peer failure, may affect performance and memory footprint + |
ucs_status_t ucp_ep_create | +( | +ucp_worker_h | +worker, | +
+ | + | const ucp_ep_params_t * | +params, | +
+ | + | ucp_ep_h * | +ep_p | +
+ | ) | ++ |
This routine creates and connects an endpoint on a local worker for a destination address that identifies the remote worker. This function is non-blocking, and communications may begin immediately after it returns. If the connection process is not completed, communications may be delayed. The created endpoint is associated with one and only one worker.
+[in] | worker | Handle to the worker; the endpoint is associated with the worker. |
[in] | params | User defined ucp_ep_params_t configurations for the UCP endpoint. |
[out] | ep_p | A handle to the created endpoint. |
ucs_status_ptr_t ucp_ep_close_nbx | +( | +ucp_ep_h | +ep, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
[in] | ep | Handle to the endpoint to close. |
[in] | param | Operation parameters, see ucp_request_param_t. This operation supports specific flags, which can be passed in param by ucp_request_param_t::flags. The exact set of flags is defined by ucp_ep_close_flags_t. |
void ucp_ep_print_info | +( | +ucp_ep_h | +ep, | +
+ | + | FILE * | +stream | +
+ | ) | ++ |
This routine prints information about the endpoint transport methods, their thresholds, and other useful information associated with the endpoint.
+[in] | ep | Endpoint object whose configuration to print. |
[in] | stream | Output stream to print the information to. |
ucs_status_ptr_t ucp_ep_flush_nbx | +( | +ucp_ep_h | +ep, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine flushes all outstanding AMO and RMA communications on the endpoint. All the AMO and RMA operations issued on the ep prior to this call are completed both at the origin and at the target endpoint when this call returns.
+[in] | ep | UCP endpoint. |
[in] | param | Operation parameters, see ucp_request_param_t. |
The following example demonstrates how blocking flush can be implemented using non-blocking flush:
+ucs_status_t ucp_ep_evaluate_perf | +( | +ucp_ep_h | +ep, | +
+ | + | const ucp_ep_evaluate_perf_param_t * | +param, | +
+ | + | ucp_ep_evaluate_perf_attr_t * | +attr | +
+ | ) | ++ |
This routine fetches information about the endpoint.
+[in] | ep | Endpoint to query. |
[in] | param | Filled by the user with request params. |
[out] | attr | Filled with performance estimation of the given operation on the endpoint. |
ucs_status_t ucp_ep_query | +( | +ucp_ep_h | +ep, | +
+ | + | ucp_ep_attr_t * | +attr | +
+ | ) | ++ |
This routine fetches information about the endpoint.
+[in] | ep | Endpoint object to query. |
[out] | attr | Filled with attributes of the endpoint. |
void ucp_request_release | +( | +void * | +request | ) | ++ |
void ucp_ep_destroy | +( | +ucp_ep_h | +ep | ) | ++ |
ucs_status_ptr_t ucp_disconnect_nb | +( | +ucp_ep_h | +ep | ) | ++ |
ucs_status_t ucp_request_test | +( | +void * | +request, | +
+ | + | ucp_tag_recv_info_t * | +info | +
+ | ) | ++ |
ucs_status_t ucp_ep_flush | +( | +ucp_ep_h | +ep | ) | ++ |
ucs_status_ptr_t ucp_ep_modify_nb | +( | +ucp_ep_h | +ep, | +
+ | + | const ucp_ep_params_t * | +params | +
+ | ) | ++ |
This routine modifies endpoint created by ucp_ep_create or ucp_listener_accept_callback_t. For example, this API can be used to setup custom parameters like ucp_ep_params_t::user_data or ucp_ep_params_t::err_handler to endpoint created by ucp_listener_accept_callback_t.
+[in] | ep | A handle to the endpoint. |
[in] | params | User defined ucp_ep_params_t configurations for the UCP endpoint. |
ucs_status_ptr_t ucp_ep_close_nb | +( | +ucp_ep_h | +ep, | +
+ | + | unsigned | +mode | +
+ | ) | ++ |
This routine releases the endpoint. The endpoint closure process depends on the selected mode.
+[in] | ep | Handle to the endpoint to close. |
[in] | mode | One from ucp_ep_close_mode value. |
ucs_status_ptr_t ucp_ep_flush_nb | +( | +ucp_ep_h | +ep, | +
+ | + | unsigned | +flags, | +
+ | + | ucp_send_callback_t | +cb | +
+ | ) | ++ |
This routine flushes all outstanding AMO and RMA communications on the endpoint. All the AMO and RMA operations issued on the ep prior to this call are completed both at the origin and at the target endpoint when this call returns.
+[in] | ep | UCP endpoint. |
[in] | flags | Flags for flush operation. Reserved for future use. |
[in] | cb | Callback which will be called when the flush operation completes. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | ucp_mem_map_params |
Tuning parameters for the UCP memory mapping. More... | |
struct | ucp_mem_advise_params |
Tuning parameters for the UCP memory advice. More... | |
struct | ucp_memh_pack_params |
Memory handle pack parameters passed to ucp_memh_pack. More... | |
struct | ucp_memh_buffer_release_params |
Memory handle release parameters passed to ucp_memh_buffer_release. More... | |
struct | ucp_mem_attr |
Attributes of the UCP Memory handle, filled by ucp_mem_query function. More... | |
+Typedefs | |
typedef struct ucp_mem_map_params | ucp_mem_map_params_t |
Tuning parameters for the UCP memory mapping. More... | |
typedef enum ucp_mem_advice | ucp_mem_advice_t |
list of UCP memory use advice. More... | |
typedef struct +ucp_mem_advise_params | ucp_mem_advise_params_t |
Tuning parameters for the UCP memory advice. More... | |
typedef struct ucp_memh_pack_params | ucp_memh_pack_params_t |
Memory handle pack parameters passed to ucp_memh_pack. More... | |
typedef struct +ucp_memh_buffer_release_params | ucp_memh_buffer_release_params_t |
Memory handle release parameters passed to ucp_memh_buffer_release. More... | |
typedef struct ucp_rkey * | ucp_rkey_h |
UCP Remote memory handle. More... | |
typedef struct ucp_mem * | ucp_mem_h |
UCP Memory handle. More... | |
typedef struct ucp_mem_attr | ucp_mem_attr_t |
Attributes of the UCP Memory handle, filled by ucp_mem_query function. More... | |
+Enumerations | |
enum | ucp_mem_map_params_field { + UCP_MEM_MAP_PARAM_FIELD_ADDRESS = UCS_BIT(0), +UCP_MEM_MAP_PARAM_FIELD_LENGTH = UCS_BIT(1), +UCP_MEM_MAP_PARAM_FIELD_FLAGS = UCS_BIT(2), +UCP_MEM_MAP_PARAM_FIELD_PROT = UCS_BIT(3), + + UCP_MEM_MAP_PARAM_FIELD_MEMORY_TYPE = UCS_BIT(4), +UCP_MEM_MAP_PARAM_FIELD_EXPORTED_MEMH_BUFFER = UCS_BIT(5) + + } |
UCP memory mapping parameters field mask. More... | |
enum | ucp_mem_advise_params_field { UCP_MEM_ADVISE_PARAM_FIELD_ADDRESS = UCS_BIT(0), +UCP_MEM_ADVISE_PARAM_FIELD_LENGTH = UCS_BIT(1), +UCP_MEM_ADVISE_PARAM_FIELD_ADVICE = UCS_BIT(2) + } |
UCP memory advice parameters field mask. More... | |
enum | { UCP_MEM_MAP_NONBLOCK = UCS_BIT(0), +UCP_MEM_MAP_ALLOCATE = UCS_BIT(1), +UCP_MEM_MAP_FIXED = UCS_BIT(2) + } |
UCP memory mapping flags. More... | |
enum | { UCP_MEM_MAP_PROT_LOCAL_READ = UCS_BIT(0), +UCP_MEM_MAP_PROT_LOCAL_WRITE = UCS_BIT(1), +UCP_MEM_MAP_PROT_REMOTE_READ = UCS_BIT(8), +UCP_MEM_MAP_PROT_REMOTE_WRITE = UCS_BIT(9) + } |
UCP memory mapping protection mode. More... | |
enum | ucp_mem_advice { UCP_MADV_NORMAL = 0, +UCP_MADV_WILLNEED + } |
list of UCP memory use advice. More... | |
enum | ucp_memh_pack_params_field { UCP_MEMH_PACK_PARAM_FIELD_FLAGS = UCS_BIT(0) + } |
UCP memory handle packing parameters field mask. More... | |
enum | ucp_memh_pack_flags { UCP_MEMH_PACK_FLAG_EXPORT = UCS_BIT(0) + } |
UCP memory handle flags. More... | |
enum | ucp_mem_attr_field { UCP_MEM_ATTR_FIELD_ADDRESS = UCS_BIT(0), +UCP_MEM_ATTR_FIELD_LENGTH = UCS_BIT(1), +UCP_MEM_ATTR_FIELD_MEM_TYPE = UCS_BIT(2) + } |
UCP Memory handle attributes field mask. More... | |
+Functions | |
ucs_status_t | ucp_mem_map (ucp_context_h context, const ucp_mem_map_params_t *params, ucp_mem_h *memh_p) |
Map or allocate memory for zero-copy operations. More... | |
ucs_status_t | ucp_mem_unmap (ucp_context_h context, ucp_mem_h memh) |
Unmap memory segment. More... | |
ucs_status_t | ucp_mem_query (const ucp_mem_h memh, ucp_mem_attr_t *attr) |
query mapped memory segment More... | |
void | ucp_mem_print_info (const char *mem_spec, ucp_context_h context, FILE *stream) |
Print memory mapping information. More... | |
ucs_status_t | ucp_mem_advise (ucp_context_h context, ucp_mem_h memh, ucp_mem_advise_params_t *params) |
give advice about the use of memory More... | |
ucs_status_t | ucp_memh_pack (ucp_mem_h memh, const ucp_memh_pack_params_t *params, void **buffer_p, size_t *buffer_size_p) |
Pack a memory handle to a buffer specified by the user. More... | |
void | ucp_memh_buffer_release (void *buffer, const ucp_memh_buffer_release_params_t *params) |
Release packed memory handle buffer. More... | |
ucs_status_t | ucp_ep_rkey_unpack (ucp_ep_h ep, const void *rkey_buffer, ucp_rkey_h *rkey_p) |
Create remote access key from packed buffer. More... | |
ucs_status_t | ucp_rkey_ptr (ucp_rkey_h rkey, uint64_t raddr, void **addr_p) |
Get a local pointer to remote memory. More... | |
void | ucp_rkey_destroy (ucp_rkey_h rkey) |
Destroy the remote key. More... | |
ucs_status_t | ucp_rkey_pack (ucp_context_h context, ucp_mem_h memh, void **rkey_buffer_p, size_t *size_p) |
Pack memory region remote access key. More... | |
void | ucp_rkey_buffer_release (void *rkey_buffer) |
Release packed remote key buffer. More... | |
UCP Memory routines
+struct ucp_mem_map_params | +
The structure defines the parameters that are used for the UCP memory mapping tuning during the ucp_mem_map routine.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_mem_map_params_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+void * | ++address | +
+ If the address is not NULL, the routine maps (registers) the memory segment pointed to by this address. If the pointer is NULL, the library allocates mapped (registered) memory segment and returns its address in this argument. Therefore, this value is optional. If it's not set (along with its corresponding bit in the field_mask - UCP_MEM_MAP_PARAM_FIELD_ADDRESS), the ucp_mem_map routine will consider address as set to NULL and will allocate memory. + |
+size_t | ++length | +
+ Length (in bytes) to allocate or map (register). This field is mandatory for filling (along with its corresponding bit in the field_mask - UCP_MEM_MAP_PARAM_FIELD_LENGTH). The ucp_mem_map routine will return with an error if the length isn't specified. + |
+unsigned | ++flags | +
+ Allocation flags, e.g. UCP_MEM_MAP_NONBLOCK. This value is optional. If it's not set (along with its corresponding bit in the field_mask - UCP_MEM_MAP_PARAM_FIELD_FLAGS), the ucp_mem_map routine will consider the flags as set to zero. + |
+unsigned | ++prot | +
+ Memory protection mode, e.g. UCP_MEM_MAP_PROT_LOCAL_READ. This value is optional. If it's not set, the ucp_mem_map routine will consider the flags as set to UCP_MEM_MAP_PROT_LOCAL_READ|UCP_MEM_MAP_PROT_LOCAL_WRITE| UCP_MEM_MAP_PROT_REMOTE_READ|UCP_MEM_MAP_PROT_REMOTE_WRITE. + |
+ucs_memory_type_t | ++memory_type | ++ |
+const void * | ++exported_memh_buffer | +
+ Exported memory handle buffer as returned by ucp_mem_map function for a memory handle created and packed by ucp_memh_pack with UCP_MEMH_PACK_FLAG_EXPORT flag. If this field is specified for ucp_mem_map function, a resulting memory handle will be a mapping of peer memory instead of local memory. If the field is not set (along with its corresponding bit in the field_mask - UCP_MEM_MAP_PARAM_FIELD_EXPORTED_MEMH_BUFFER), the ucp_mem_map routine will consider the memory handle buffer to be set to NULL by default. + |
struct ucp_mem_advise_params | +
This structure defines the parameters that are used for the UCP memory advice tuning during the ucp_mem_advise routine.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_mem_advise_params_field. All fields are mandatory. Provides ABI compatibility with respect to adding new fields. + |
+void * | ++address | +
+ Memory base address. + |
+size_t | ++length | +
+ Length (in bytes) to allocate or map (register). + |
+ucp_mem_advice_t | ++advice | +
+ Memory use advice ucp_mem_advice + |
struct ucp_memh_pack_params | +
This structure defines the parameters that are used for packing the UCP memory handle during the ucp_memh_pack routine.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+uint64_t | ++flags | +
+ Flags to control packing of a memory handle. + |
struct ucp_memh_buffer_release_params | +
This structure defines the parameters that are used for releasing the UCP memory handle buffer during the ucp_memh_buffer_release routine.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure. All fields are mandatory. Provides ABI compatibility with respect to adding new fields. + |
struct ucp_mem_attr | +
Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_mem_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+void * | ++address | +
+ Address of the memory segment. + |
+size_t | ++length | +
+ Size of the memory segment. + |
+ucs_memory_type_t | ++mem_type | +
+ Type of allocated or registered memory + |
typedef struct ucp_mem_map_params ucp_mem_map_params_t | +
The structure defines the parameters that are used for the UCP memory mapping tuning during the ucp_mem_map routine.
+ +typedef enum ucp_mem_advice ucp_mem_advice_t | +
The enumeration list describes memory advice supported by ucp_mem_advise() function.
+ +typedef struct ucp_mem_advise_params ucp_mem_advise_params_t | +
This structure defines the parameters that are used for the UCP memory advice tuning during the ucp_mem_advise routine.
+ +typedef struct ucp_memh_pack_params ucp_memh_pack_params_t | +
This structure defines the parameters that are used for packing the UCP memory handle during the ucp_memh_pack routine.
+ +typedef struct ucp_memh_buffer_release_params ucp_memh_buffer_release_params_t | +
This structure defines the parameters that are used for releasing the UCP memory handle buffer during the ucp_memh_buffer_release routine.
+ +typedef struct ucp_rkey* ucp_rkey_h | +
Remote memory handle is an opaque object representing remote memory access information. Typically, the handle includes a memory access key and other network hardware specific information, which are input to remote memory access operations, such as PUT, GET, and ATOMIC. The object is communicated to remote peers to enable an access to the memory region.
+ +typedef struct ucp_mem* ucp_mem_h | +
Memory handle is an opaque object representing a memory region allocated through UCP library, which is optimized for remote memory access operations (zero-copy operations). The memory handle is a self-contained object, which includes the information required to access the memory region locally, while remote key is used to access it remotely. The memory could be registered to one or multiple network resources that are supported by UCP, such as InfiniBand, Gemini, and others.
+ +typedef struct ucp_mem_attr ucp_mem_attr_t | +
enum ucp_mem_map_params_field | +
The enumeration allows specifying which fields in ucp_mem_map_params_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_MEM_MAP_PARAM_FIELD_ADDRESS |
+ Address of the memory that will be used in the ucp_mem_map routine. + |
UCP_MEM_MAP_PARAM_FIELD_LENGTH |
+ The size of memory that will be allocated or registered in the ucp_mem_map routine. + |
UCP_MEM_MAP_PARAM_FIELD_FLAGS |
+ Allocation flags. + |
UCP_MEM_MAP_PARAM_FIELD_PROT |
+ Memory protection mode. + |
UCP_MEM_MAP_PARAM_FIELD_MEMORY_TYPE |
+ Memory type. + |
UCP_MEM_MAP_PARAM_FIELD_EXPORTED_MEMH_BUFFER |
+ Exported memory handle buffer. + |
enum ucp_mem_advise_params_field | +
The enumeration allows specifying which fields in ucp_mem_advise_params_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_MEM_ADVISE_PARAM_FIELD_ADDRESS |
+ Address of the memory + |
UCP_MEM_ADVISE_PARAM_FIELD_LENGTH |
+ The size of memory + |
UCP_MEM_ADVISE_PARAM_FIELD_ADVICE |
+ Advice on memory usage + |
anonymous enum | +
The enumeration list describes the memory mapping flags supported by ucp_mem_map() function.
+Enumerator | |
---|---|
UCP_MEM_MAP_NONBLOCK |
+ Complete the mapping faster, possibly by not populating the pages in the mapping up-front, and mapping them later when they are accessed by communication routines. + |
UCP_MEM_MAP_ALLOCATE |
+ Identify requirement for allocation, if passed address is not a null-pointer, then it will be used as a hint or direct address for allocation. + |
UCP_MEM_MAP_FIXED |
+ Don't interpret address as a hint: place the mapping at exactly that address. The address must be a multiple of the page size. + |
anonymous enum | +
The enumeration list describes the memory mapping protections supported by the ucp_mem_map() function.
+Enumerator | |
---|---|
UCP_MEM_MAP_PROT_LOCAL_READ |
+ Enable local read access. + |
UCP_MEM_MAP_PROT_LOCAL_WRITE |
+ Enable local write access. + |
UCP_MEM_MAP_PROT_REMOTE_READ |
+ Enable remote read access. + |
UCP_MEM_MAP_PROT_REMOTE_WRITE |
+ Enable remote write access. + |
enum ucp_mem_advice | +
The enumeration list describes memory advice supported by ucp_mem_advise() function.
+Enumerator | |
---|---|
UCP_MADV_NORMAL |
+ No special treatment + |
UCP_MADV_WILLNEED |
+ can be used on the memory mapped with UCP_MEM_MAP_NONBLOCK to speed up memory mapping and to avoid page faults when the memory is accessed for the first time. + |
enum ucp_memh_pack_params_field | +
The enumeration allows specifying which fields in ucp_memh_pack_params_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_MEMH_PACK_PARAM_FIELD_FLAGS |
+ Memory handle packing field that will be used in the ucp_memh_pack routine. + |
enum ucp_memh_pack_flags | +
The enumeration list describes the memory handle packing flags supported by ucp_memh_pack() function.
+Enumerator | |
---|---|
UCP_MEMH_PACK_FLAG_EXPORT |
+ Pack a memory handle to be exported and used by peers for their local operations on a memory buffer allocated from same or another virtual memory space, but physically registered on the same network device. A peer should call ucp_mem_map with the flag UCP_MEM_MAP_PARAM_FIELD_EXPORTED_MEMH_BUFFER in order to import and use a memory handle buffer obtained from ucp_memh_pack. + |
enum ucp_mem_attr_field | +
The enumeration allows specifying which fields in ucp_mem_attr_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_MEM_ATTR_FIELD_ADDRESS |
+ Virtual address + |
UCP_MEM_ATTR_FIELD_LENGTH |
+ The size of memory region + |
UCP_MEM_ATTR_FIELD_MEM_TYPE |
+ Type of allocated or registered memory + |
ucs_status_t ucp_mem_map | +( | +ucp_context_h | +context, | +
+ | + | const ucp_mem_map_params_t * | +params, | +
+ | + | ucp_mem_h * | +memh_p | +
+ | ) | ++ |
This routine maps or/and allocates a user-specified memory segment with UCP application context and the network resources associated with it. If the application specifies NULL as an address for the memory segment, the routine allocates a mapped memory segment and returns its address in the address_p argument. The network stack associated with an application context can typically send and receive data from the mapped memory without CPU intervention; some devices and associated network stacks require the memory to be mapped to send and receive data. The memory handle includes all information required to access the memory locally using UCP routines, while remote registration handle provides an information that is necessary for remote memory access.
+Memory mapping assumptions:
+parameter/flag | NONBLOCK | ALLOCATE | FIXED | address | result |
---|---|---|---|---|---|
value | 0/1 - the value + only affects the + register/map + phase | 0 | 0 | 0 | error if length > 0 |
1 | 0 | 0 | alloc+register | ||
0 | 1 | 0 | error | ||
0 | 0 | defined | register | ||
1 | 1 | 0 | error | ||
1 | 0 | defined | alloc+register,hint | ||
0 | 1 | defined | error | ||
1 | 1 | defined | alloc+register,fixed |
[in] | context | Application context to map (register) and allocate the memory on. |
[in] | params | User defined ucp_mem_map_params_t configurations for the UCP memory handle. |
[out] | memh_p | UCP handle for the allocated segment. |
ucs_status_t ucp_mem_unmap | +( | +ucp_context_h | +context, | +
+ | + | ucp_mem_h | +memh | +
+ | ) | ++ |
This routine unmaps a user specified memory segment, that was previously mapped using the ucp_mem_map() routine. The unmap routine will also release the resources associated with the memory handle. When the function returns, the ucp_mem_h and associated remote key will be invalid and cannot be used with any UCP routine.
+Error cases:
+[in] | context | Application context which was used to allocate/map the memory. |
[in] | memh | Handle to memory region. |
ucs_status_t ucp_mem_query | +( | +const ucp_mem_h | +memh, | +
+ | + | ucp_mem_attr_t * | +attr | +
+ | ) | ++ |
This routine returns address and length of memory segment mapped with ucp_mem_map() routine.
+[in] | memh | Handle to memory region. |
[out] | attr | Filled with attributes of the UCP memory handle. |
void ucp_mem_print_info | +( | +const char * | +mem_spec, | +
+ | + | ucp_context_h | +context, | +
+ | + | FILE * | +stream | +
+ | ) | ++ |
This routine maps memory and prints information about the created memory handle: including the mapped memory length, the allocation method, and other useful information associated with the memory handle.
+[in] | mem_spec | Size and optional type of the memory to map. The format of the string is: "<size>[,<type>]". For example:
|
[in] | context | The context on which the memory is mapped. |
[in] | stream | Output stream on which to print the information. |
ucs_status_t ucp_mem_advise | +( | +ucp_context_h | +context, | +
+ | + | ucp_mem_h | +memh, | +
+ | + | ucp_mem_advise_params_t * | +params | +
+ | ) | ++ |
This routine advises the UCP about how to handle memory range beginning at address and size of length bytes. This call does not influence the semantics of the application, but may influence its performance. The UCP may ignore the advice.
+[in] | context | Application context which was used to allocate/map the memory. |
[in] | memh | Handle to memory region. |
[in] | params | Memory base address and length. The advice field is used to pass memory use advice as defined in the ucp_mem_advice list The memory range must belong to the memh |
ucs_status_t ucp_memh_pack | +( | +ucp_mem_h | +memh, | +
+ | + | const ucp_memh_pack_params_t * | +params, | +
+ | + | void ** | +buffer_p, | +
+ | + | size_t * | +buffer_size_p | +
+ | ) | ++ |
This routine allocates a memory buffer and packs a memory handle into the buffer. A packed memory key is an opaque object that provides the information that is necessary for a peer. This routine packs the memory handle in a portable format such that the object can be unpacked on any platform supported by the UCP library, e.g. if the memory handle was packed as a remote memory key (RKEY), it should be unpacked by ucp_ep_rkey_unpack(). In order to release the memory buffer allocated by this routine, the application is responsible for calling the ucp_memh_buffer_release() routine.
+[in] | memh | Handle to memory region. |
[in] | params | Memory handle packing parameters, as defined by ucp_memh_pack_params_t. |
[out] | buffer_p | Memory buffer allocated by the library. The buffer contains the packed memory handle. |
[out] | buffer_size_p | Size (in bytes) of the buffer which contains packed memory handle. |
void ucp_memh_buffer_release | +( | +void * | +buffer, | +
+ | + | const ucp_memh_buffer_release_params_t * | +params | +
+ | ) | ++ |
This routine releases the buffer that was allocated using ucp_memh_pack().
+[in] | buffer | Buffer to release. |
[in] | params | Memory handle buffer release parameters, as defined by ucp_memh_buffer_release_params_t. |
ucs_status_t ucp_ep_rkey_unpack | +( | +ucp_ep_h | +ep, | +
+ | + | const void * | +rkey_buffer, | +
+ | + | ucp_rkey_h * | +rkey_p | +
+ | ) | ++ |
This routine unpacks the remote key (RKEY) object into the local memory such that it can be accessed and used by UCP routines. The RKEY object has to be packed using the ucp_rkey_pack() routine. Application code should not make any changes to the content of the RKEY buffer.
+[in] | ep | Endpoint to access using the remote key. |
[in] | rkey_buffer | Packed rkey. |
[out] | rkey_p | Remote key handle. |
ucs_status_t ucp_rkey_ptr | +( | +ucp_rkey_h | +rkey, | +
+ | + | uint64_t | +raddr, | +
+ | + | void ** | +addr_p | +
+ | ) | ++ |
This routine returns a local pointer to the remote memory described by the rkey.
+[in] | rkey | A remote key handle. |
[in] | raddr | A remote memory address within the memory area described by the rkey. |
[out] | addr_p | A pointer that can be used for direct access to the remote memory. |
void ucp_rkey_destroy | +( | +ucp_rkey_h | +rkey | ) | ++ |
This routine destroys the RKEY object and the memory that was allocated using the ucp_ep_rkey_unpack() routine. This routine also releases any resources that are associated with the RKEY object.
+[in] | rkey | Remote key to destroy. |
ucs_status_t ucp_rkey_pack | +( | +ucp_context_h | +context, | +
+ | + | ucp_mem_h | +memh, | +
+ | + | void ** | +rkey_buffer_p, | +
+ | + | size_t * | +size_p | +
+ | ) | ++ |
This routine allocates a memory buffer and packs a remote access key (RKEY) object into it. RKEY is an opaque object that provides the information that is necessary for remote memory access. This routine packs the RKEY object in a portable format such that the object can be unpacked on any platform supported by the UCP library. In order to release the memory buffer allocated by this routine, the application is responsible for calling the ucp_rkey_buffer_release() routine.
+[in] | context | Application context which was used to allocate/map the memory. |
[in] | memh | Handle to the memory region. |
[out] | rkey_buffer_p | Memory buffer allocated by the library. The buffer contains the packed RKEY. |
[out] | size_p | Size (in bytes) of the packed RKEY. |
void ucp_rkey_buffer_release | +( | +void * | +rkey_buffer | ) | ++ |
This routine releases the buffer that was allocated using ucp_rkey_pack().
+[in] | rkey_buffer | Buffer to release. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Functions | |
ucs_status_t | ucp_worker_get_efd (ucp_worker_h worker, int *fd) |
Obtain an event file descriptor for event notification. More... | |
ucs_status_t | ucp_worker_wait (ucp_worker_h worker) |
Wait for an event of the worker. More... | |
void | ucp_worker_wait_mem (ucp_worker_h worker, void *address) |
Wait for memory update on the address. More... | |
ucs_status_t | ucp_worker_arm (ucp_worker_h worker) |
Turn on event notification for the next event. More... | |
ucs_status_t | ucp_worker_signal (ucp_worker_h worker) |
Cause an event of the worker. More... | |
UCP Wake-up routines
+ucs_status_t ucp_worker_get_efd | +( | +ucp_worker_h | +worker, | +
+ | + | int * | +fd | +
+ | ) | ++ |
This routine returns a valid file descriptor for polling functions. The file descriptor will get signaled when an event occurs, as part of the wake-up mechanism. Signaling means a call to poll() or select() with this file descriptor will return at this point, with this descriptor marked as the reason (or one of the reasons) the function has returned. The user does not need to release the obtained file descriptor.
+The wake-up mechanism exists to allow for the user process to register for notifications on events of the underlying interfaces, and wait until such occur. This is an alternative to repeated polling for request completion. The goal is to allow for waiting while consuming minimal resources from the system. This is recommended for cases where traffic is infrequent, and latency can be traded for lower resource consumption while waiting for it.
+There are two alternative ways to use the wakeup mechanism: the first is the file descriptor obtained per worker (this function) and the second is the ucp_worker_wait function for waiting on the next event internally.
+[in] | worker | Worker of notified events. |
[out] | fd | File descriptor. |
ucs_status_t ucp_worker_wait | +( | +ucp_worker_h | +worker | ) | ++ |
This routine waits (blocking) until an event has happened, as part of the wake-up mechanism.
+This function is guaranteed to return only if new communication events occur on the worker. Therefore one must drain all existing events before waiting on the file descriptor. This can be achieved by calling ucp_worker_progress repeatedly until it returns 0.
+There are two alternative ways to use the wakeup mechanism. The first is by polling on a per-worker file descriptor obtained from ucp_worker_get_efd. The second is by using this function to perform an internal wait for the next event associated with the specified worker.
+[in] | worker | Worker to wait for events on. |
void ucp_worker_wait_mem | +( | +ucp_worker_h | +worker, | +
+ | + | void * | +address | +
+ | ) | ++ |
This routine waits for a memory update at the local memory address. This is a blocking routine. The routine returns when the memory address is updated ("write") or an event occurs in the system.
+This function is guaranteed to return only if new communication events occur on the worker or address is modified. Therefore one must drain all existing events before waiting on the file descriptor. This can be achieved by calling ucp_worker_progress repeatedly until it returns 0.
+[in] | worker | Worker to wait for updates on. |
[in] | address | Local memory address |
ucs_status_t ucp_worker_arm | +( | +ucp_worker_h | +worker | ) | ++ |
This routine needs to be called before waiting on each notification on this worker, so will typically be called once the processing of the previous event is over, as part of the wake-up mechanism.
+The worker must be armed before waiting on an event (must be re-armed after it has been signaled for re-use) with ucp_worker_arm. The events triggering a signal of the file descriptor from ucp_worker_get_efd depend on the interfaces used by the worker and defined in the transport layer, and typically represent a request completion or newly available resources. It can also be triggered by calling ucp_worker_signal .
+The file descriptor is guaranteed to become signaled only if new communication events occur on the worker. Therefore one must drain all existing events before waiting on the file descriptor. This can be achieved by calling ucp_worker_progress repeatedly until it returns 0.
+[in] | worker | Worker of notified events. |
ucs_status_t ucp_worker_signal | +( | +ucp_worker_h | +worker | ) | ++ |
This routine signals that the event has happened, as part of the wake-up mechanism. This function causes a blocking call to ucp_worker_wait or waiting on a file descriptor from ucp_worker_get_efd to return, even if no event from the underlying interfaces has taken place.
+[in] | worker | Worker to wait for events on. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | ucp_worker_attr |
UCP worker attributes. More... | |
struct | ucp_worker_params |
Tuning parameters for the UCP worker. More... | |
struct | ucp_worker_address_attr |
UCP worker address attributes. More... | |
struct | ucp_listener_attr |
UCP listener attributes. More... | |
struct | ucp_conn_request_attr |
UCP listener's connection request attributes. More... | |
struct | ucp_listener_params |
Parameters for a UCP listener object. More... | |
struct | ucp_am_handler_param |
Active Message handler parameters passed to ucp_worker_set_am_recv_handler routine. More... | |
struct | ucp_am_recv_param |
Operation parameters provided in ucp_am_recv_callback_t callback. More... | |
struct | ucp_listener_accept_handler |
struct | ucp_listener_conn_handler |
UCP callback to handle the connection request in a client-server connection establishment flow. More... | |
+Typedefs | |
typedef struct ucp_worker_attr | ucp_worker_attr_t |
UCP worker attributes. More... | |
typedef struct ucp_worker_params | ucp_worker_params_t |
Tuning parameters for the UCP worker. More... | |
typedef struct +ucp_worker_address_attr | ucp_worker_address_attr_t |
UCP worker address attributes. More... | |
typedef struct ucp_listener_attr | ucp_listener_attr_t |
UCP listener attributes. More... | |
typedef struct +ucp_conn_request_attr | ucp_conn_request_attr_t |
UCP listener's connection request attributes. More... | |
typedef struct ucp_listener_params | ucp_listener_params_t |
Parameters for a UCP listener object. More... | |
typedef struct ucp_am_handler_param | ucp_am_handler_param_t |
Active Message handler parameters passed to ucp_worker_set_am_recv_handler routine. More... | |
typedef struct +ucp_listener_accept_handler | ucp_listener_accept_handler_t |
typedef struct ucp_am_recv_param | ucp_am_recv_param_t |
Operation parameters provided in ucp_am_recv_callback_t callback. More... | |
typedef struct ucp_address | ucp_address_t |
UCP worker address. More... | |
typedef struct ucp_listener * | ucp_listener_h |
UCP listen handle. More... | |
typedef struct ucp_worker * | ucp_worker_h |
UCP Worker. More... | |
typedef void(* | ucp_listener_accept_callback_t )(ucp_ep_h ep, void *arg) |
A callback for accepting client/server connections on a listener ucp_listener_h. More... | |
typedef void(* | ucp_listener_conn_callback_t )(ucp_conn_request_h conn_request, void *arg) |
A callback for handling of incoming connection request conn_request from a client. More... | |
typedef struct +ucp_listener_conn_handler | ucp_listener_conn_handler_t |
UCP callback to handle the connection request in a client-server connection establishment flow. More... | |
typedef enum ucp_wakeup_event_types | ucp_wakeup_event_t |
UCP worker wakeup events mask. More... | |
+Enumerations | |
enum | ucp_worker_params_field { + UCP_WORKER_PARAM_FIELD_THREAD_MODE = UCS_BIT(0), +UCP_WORKER_PARAM_FIELD_CPU_MASK = UCS_BIT(1), +UCP_WORKER_PARAM_FIELD_EVENTS = UCS_BIT(2), +UCP_WORKER_PARAM_FIELD_USER_DATA = UCS_BIT(3), + + UCP_WORKER_PARAM_FIELD_EVENT_FD = UCS_BIT(4), +UCP_WORKER_PARAM_FIELD_FLAGS = UCS_BIT(5), +UCP_WORKER_PARAM_FIELD_NAME = UCS_BIT(6), +UCP_WORKER_PARAM_FIELD_AM_ALIGNMENT = UCS_BIT(7), + + UCP_WORKER_PARAM_FIELD_CLIENT_ID = UCS_BIT(8) + + } |
UCP worker parameters field mask. More... | |
enum | ucp_worker_flags_t { UCP_WORKER_FLAG_IGNORE_REQUEST_LEAK = UCS_BIT(0) + } |
UCP worker flags. More... | |
enum | ucp_listener_params_field { UCP_LISTENER_PARAM_FIELD_SOCK_ADDR = UCS_BIT(0), +UCP_LISTENER_PARAM_FIELD_ACCEPT_HANDLER = UCS_BIT(1), +UCP_LISTENER_PARAM_FIELD_CONN_HANDLER = UCS_BIT(2) + } |
UCP listener parameters field mask. More... | |
enum | ucp_worker_address_flags_t { UCP_WORKER_ADDRESS_FLAG_NET_ONLY = UCS_BIT(0) + } |
UCP worker address flags. More... | |
enum | ucp_worker_attr_field { + UCP_WORKER_ATTR_FIELD_THREAD_MODE = UCS_BIT(0), +UCP_WORKER_ATTR_FIELD_ADDRESS = UCS_BIT(1), +UCP_WORKER_ATTR_FIELD_ADDRESS_FLAGS = UCS_BIT(2), +UCP_WORKER_ATTR_FIELD_MAX_AM_HEADER = UCS_BIT(3), + + UCP_WORKER_ATTR_FIELD_NAME = UCS_BIT(4), +UCP_WORKER_ATTR_FIELD_MAX_INFO_STRING = UCS_BIT(5) + + } |
UCP worker attributes field mask. More... | |
enum | ucp_worker_address_attr_field { UCP_WORKER_ADDRESS_ATTR_FIELD_UID = UCS_BIT(0) + } |
UCP worker address attributes field mask. More... | |
enum | ucp_listener_attr_field { UCP_LISTENER_ATTR_FIELD_SOCKADDR = UCS_BIT(0) + } |
UCP listener attributes field mask. More... | |
enum | ucp_conn_request_attr_field { UCP_CONN_REQUEST_ATTR_FIELD_CLIENT_ADDR = UCS_BIT(0), +UCP_CONN_REQUEST_ATTR_FIELD_CLIENT_ID = UCS_BIT(1) + } |
UCP listener's connection request attributes field mask. More... | |
enum | ucp_am_cb_flags { UCP_AM_FLAG_WHOLE_MSG = UCS_BIT(0), +UCP_AM_FLAG_PERSISTENT_DATA = UCS_BIT(1) + } |
Flags for a UCP Active Message callback. More... | |
enum | ucp_send_am_flags { + UCP_AM_SEND_FLAG_REPLY = UCS_BIT(0), +UCP_AM_SEND_FLAG_EAGER = UCS_BIT(1), +UCP_AM_SEND_FLAG_RNDV = UCS_BIT(2), +UCP_AM_SEND_FLAG_COPY_HEADER = UCS_BIT(3), + + UCP_AM_SEND_REPLY = UCP_AM_SEND_FLAG_REPLY + + } |
Flags for sending a UCP Active Message. More... | |
enum | ucp_wakeup_event_types { + UCP_WAKEUP_RMA = UCS_BIT(0), +UCP_WAKEUP_AMO = UCS_BIT(1), +UCP_WAKEUP_TAG_SEND = UCS_BIT(2), +UCP_WAKEUP_TAG_RECV = UCS_BIT(3), + + UCP_WAKEUP_TX = UCS_BIT(10), +UCP_WAKEUP_RX = UCS_BIT(11), +UCP_WAKEUP_EDGE = UCS_BIT(16) + + } |
UCP worker wakeup events mask. More... | |
+Functions | |
ucs_status_t | ucp_worker_create (ucp_context_h context, const ucp_worker_params_t *params, ucp_worker_h *worker_p) |
Create a worker object. More... | |
void | ucp_worker_destroy (ucp_worker_h worker) |
Destroy a worker object. More... | |
ucs_status_t | ucp_worker_query (ucp_worker_h worker, ucp_worker_attr_t *attr) |
Get attributes specific to a particular worker. More... | |
void | ucp_worker_print_info (ucp_worker_h worker, FILE *stream) |
Print information about the worker. More... | |
void | ucp_worker_release_address (ucp_worker_h worker, ucp_address_t *address) |
Release an address of the worker object. More... | |
ucs_status_t | ucp_worker_address_query (ucp_address_t *address, ucp_worker_address_attr_t *attr) |
Get attributes of the particular worker address. More... | |
unsigned | ucp_worker_progress (ucp_worker_h worker) |
Progress all communications on a specific worker. More... | |
ssize_t | ucp_stream_worker_poll (ucp_worker_h worker, ucp_stream_poll_ep_t *poll_eps, size_t max_eps, unsigned flags) |
Poll for endpoints that are ready to consume streaming data. More... | |
ucs_status_t | ucp_listener_create (ucp_worker_h worker, const ucp_listener_params_t *params, ucp_listener_h *listener_p) |
Create a listener to accept connections on. Connection requests on the listener will arrive at a local address specified by the user. More... | |
void | ucp_listener_destroy (ucp_listener_h listener) |
Stop accepting connections on a local address of the worker object. More... | |
ucs_status_t | ucp_listener_query (ucp_listener_h listener, ucp_listener_attr_t *attr) |
Get attributes specific to a particular listener. More... | |
ucs_status_t | ucp_conn_request_query (ucp_conn_request_h conn_request, ucp_conn_request_attr_t *attr) |
Get attributes specific to a particular connection request received on the server side. More... | |
ucs_status_t | ucp_listener_reject (ucp_listener_h listener, ucp_conn_request_h conn_request) |
Reject an incoming connection request. More... | |
ucs_status_t | ucp_worker_set_am_recv_handler (ucp_worker_h worker, const ucp_am_handler_param_t *param) |
Add user defined callback for Active Message. More... | |
ucs_status_t | ucp_worker_fence (ucp_worker_h worker) |
Assures ordering between non-blocking operations. More... | |
ucs_status_ptr_t | ucp_worker_flush_nbx (ucp_worker_h worker, const ucp_request_param_t *param) |
Flush outstanding AMO and RMA operations on the worker. More... | |
ucs_status_t | ucp_worker_flush (ucp_worker_h worker) |
Flush outstanding AMO and RMA operations on the worker. More... | |
ucs_status_t | ucp_worker_get_address (ucp_worker_h worker, ucp_address_t **address_p, size_t *address_length_p) |
Get the address of the worker object. More... | |
ucs_status_t | ucp_worker_set_am_handler (ucp_worker_h worker, uint16_t id, ucp_am_callback_t cb, void *arg, uint32_t flags) |
Add user defined callback for Active Message. More... | |
ucs_status_ptr_t | ucp_worker_flush_nb (ucp_worker_h worker, unsigned flags, ucp_send_callback_t cb) |
Flush outstanding AMO and RMA operations on the worker. More... | |
UCP Worker routines
+struct ucp_worker_attr | +
The structure defines the attributes which characterize the particular worker.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_worker_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+ucs_thread_mode_t | ++thread_mode | +
+ Thread safe level of the worker. + |
+uint32_t | ++address_flags | +
+ Flags indicating requested details of the worker address. If UCP_WORKER_ATTR_FIELD_ADDRESS_FLAGS bit is set in the field_mask, this value should be set as well. Possible flags are specified in ucp_worker_address_flags_t. +
|
+ucp_address_t * | ++address | +
+ Worker address, which can be passed to remote instances of the UCP library in order to connect to this worker. The memory for the address handle is allocated by ucp_worker_query() routine, and must be released by using ucp_worker_release_address() routine. + |
+size_t | ++address_length | +
+ Size of worker address in bytes. + |
+size_t | ++max_am_header | +
+ Maximum allowed header size for ucp_am_send_nbx routine. + |
+char | ++name[UCP_ENTITY_NAME_MAX] | +
+ Tracing and analysis tools can identify the worker using this name. + |
+size_t | ++max_debug_string | +
+ Maximum debug string size that can be filled with ucp_request_query. + |
struct ucp_worker_params | +
The structure defines the parameters that are used for the UCP worker tuning during the UCP worker creation.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_worker_params_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+ucs_thread_mode_t | ++thread_mode | +
+ The parameter thread_mode suggests the thread safety mode which worker and the associated resources should be created with. This is an optional parameter. The default value is UCS_THREAD_MODE_SINGLE and it is used when the value of the parameter is not set. When this parameter along with its corresponding bit in the field_mask - UCP_WORKER_PARAM_FIELD_THREAD_MODE is set, the ucp_worker_create attempts to create worker with this thread mode. The thread mode with which worker is created can differ from the suggested mode. The actual thread mode of the worker should be obtained using the query interface ucp_worker_query. + |
+ucs_cpu_set_t | ++cpu_mask | +
+ Mask of which CPUs worker resources should preferably be allocated on. This value is optional. If it's not set (along with its corresponding bit in the field_mask - UCP_WORKER_PARAM_FIELD_CPU_MASK), resources are allocated according to system's default policy. + |
+unsigned | ++events | +
+ Mask of events (ucp_wakeup_event_t) which are expected on wakeup. This value is optional. If it's not set (along with its corresponding bit in the field_mask - UCP_WORKER_PARAM_FIELD_EVENTS), all types of events will trigger on wakeup. + |
+void * | ++user_data | +
+ User data associated with the current worker. This value is optional. If it's not set (along with its corresponding bit in the field_mask - UCP_WORKER_PARAM_FIELD_USER_DATA), it will default to NULL. + |
+int | ++event_fd | +
+ External event file descriptor. This value is optional. If UCP_WORKER_PARAM_FIELD_EVENT_FD is set in the field_mask, events on the worker will be reported on the provided event file descriptor. In this case, calling ucp_worker_get_efd will result in an error. The provided file descriptor must be capable of aggregating notifications for arbitrary events, for example Otherwise, events will be reported to the event file descriptor returned from ucp_worker_get_efd(). + |
+uint64_t | ++flags | +
+ Worker flags. This value is optional. If UCP_WORKER_PARAM_FIELD_FLAGS is not set in the field_mask, the value of this field will default to 0. + |
+const char * | ++name | +
+ Tracing and analysis tools can identify the worker using this name. To retrieve the worker's name, use ucp_worker_query, as the name you supply may be changed by UCX under some circumstances, e.g. a name conflict. This field is only assigned if you set UCP_WORKER_PARAM_FIELD_NAME in the field mask. If not, then a default unique name will be created for you. + |
+size_t | ++am_alignment | +
+ Minimal address alignment of the active message data pointer as passed in argument data to the active message handler, defined as ucp_am_recv_callback_t. + |
+uint64_t | ++client_id | +
+ Client id that is sent as part of the connection request payload when connecting to a remote socket address. On the remote side, this value can be obtained from ucp_conn_request_h using ucp_conn_request_query. + |
struct ucp_worker_address_attr | +
The structure defines the attributes of the particular worker address.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_worker_address_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+uint64_t | ++worker_uid | +
+ Unique id of the worker this address belongs to. + |
struct ucp_listener_attr | +
The structure defines the attributes which characterize the particular listener.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_listener_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+struct sockaddr_storage | ++sockaddr | +
+ Sockaddr on which this listener is listening for incoming connection requests. + |
struct ucp_conn_request_attr | +
The structure defines the attributes that characterize the particular connection request received on the server side.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_conn_request_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+struct sockaddr_storage | ++client_address | +
+ The address of the remote client that sent the connection request to the server. + |
+uint64_t | ++client_id | +
+ Remote client id if remote endpoint's flag UCP_EP_PARAMS_FLAGS_SEND_CLIENT_ID is set. + |
struct ucp_listener_params | +
This structure defines parameters for ucp_listener_create, which is used to listen for incoming client/server connections.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_listener_params_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+ucs_sock_addr_t | ++sockaddr | +
+ An address in the form of a sockaddr. This field is mandatory for filling (along with its corresponding bit in the field_mask - UCP_LISTENER_PARAM_FIELD_SOCK_ADDR). The ucp_listener_create routine will return with an error if sockaddr is not specified. + |
+ucp_listener_accept_handler_t | ++accept_handler | +
+ Handler to endpoint creation in a client-server connection flow. In order for the callback inside this handler to be invoked, the UCP_LISTENER_PARAM_FIELD_ACCEPT_HANDLER needs to be set in the field_mask. + |
+ucp_listener_conn_handler_t | ++conn_handler | +
+ Handler of an incoming connection request in a client-server connection flow. In order for the callback inside this handler to be invoked, the UCP_LISTENER_PARAM_FIELD_CONN_HANDLER needs to be set in the field_mask. +
|
struct ucp_am_handler_param | +
Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from ucp_am_handler_param_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+unsigned | ++id | +
+ Active Message id. + |
+uint32_t | ++flags | +
+ Handler flags as defined by ucp_am_cb_flags. + |
+ucp_am_recv_callback_t | ++cb | +
+ Active Message callback. To clear the already set callback, this value should be set to NULL. + |
+void * | ++arg | +
+ Active Message argument, which will be passed in to every invocation of ucp_am_recv_callback_t function as the arg argument. + |
struct ucp_am_recv_param | +
Data Fields | ||
---|---|---|
+uint64_t | ++recv_attr | +
+ Mask of valid fields in this structure and receive operation flags, using bits from ucp_am_recv_attr_t. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+ucp_ep_h | ++reply_ep | +
+ Endpoint, which can be used for the reply to this message. + |
struct ucp_listener_accept_handler | +
Data Fields | ||
---|---|---|
+ucp_listener_accept_callback_t | ++cb | +
+ Endpoint creation callback + |
+void * | ++arg | +
+ User defined argument for the callback + |
struct ucp_listener_conn_handler | +
This structure is used for handling an incoming connection request on the listener. Setting this type of handler allows creating an endpoint on any other worker and not limited to the worker on which the listener was created.
+Data Fields | ||
---|---|---|
+ucp_listener_conn_callback_t | ++cb | +
+ Connection request callback + |
+void * | ++arg | +
+ User defined argument for the callback + |
typedef struct ucp_worker_attr ucp_worker_attr_t | +
The structure defines the attributes which characterize the particular worker.
+ +typedef struct ucp_worker_params ucp_worker_params_t | +
The structure defines the parameters that are used for the UCP worker tuning during the UCP worker creation.
+ +typedef struct ucp_worker_address_attr ucp_worker_address_attr_t | +
The structure defines the attributes of the particular worker address.
+ +typedef struct ucp_listener_attr ucp_listener_attr_t | +
The structure defines the attributes which characterize the particular listener.
+ +typedef struct ucp_conn_request_attr ucp_conn_request_attr_t | +
The structure defines the attributes that characterize the particular connection request received on the server side.
+ +typedef struct ucp_listener_params ucp_listener_params_t | +
This structure defines parameters for ucp_listener_create, which is used to listen for incoming client/server connections.
+ +typedef struct ucp_am_handler_param ucp_am_handler_param_t | +
typedef struct ucp_listener_accept_handler ucp_listener_accept_handler_t | +
typedef struct ucp_am_recv_param ucp_am_recv_param_t | +
typedef struct ucp_address ucp_address_t | +
The address handle is an opaque object that is used as an identifier for a worker instance.
+ +typedef struct ucp_listener* ucp_listener_h | +
The listener handle is an opaque object that is used for listening on a specific address and accepting connections from clients.
+ +typedef struct ucp_worker* ucp_worker_h | +
UCP worker is an opaque object representing the communication context. The worker represents an instance of a local communication resource and the progress engine associated with it. The progress engine is a construct that is responsible for asynchronous and independent progress of communication directives. The progress engine could be implemented in hardware or software. The worker object abstracts an instance of network resources such as a host channel adapter port, network interface, or multiple resources such as multiple network interfaces or communication ports. It could also represent virtual communication resources that are defined across multiple devices. Although the worker can represent multiple network resources, it is associated with a single UCX application context. All communication functions require a context to perform the operation on the dedicated hardware resource(s) and an endpoint to address the destination.
+typedef void(* ucp_listener_accept_callback_t)(ucp_ep_h ep, void *arg) | +
This callback routine is invoked on the server side upon creating a connection to a remote client. The user can pass an argument to this callback. The user is responsible for releasing the ep handle using the ucp_ep_destroy() routine.
+[in] | ep | Handle to a newly created endpoint which is connected to the remote peer which has initiated the connection. |
[in] | arg | User's argument for the callback. |
typedef void(* ucp_listener_conn_callback_t)(ucp_conn_request_h conn_request, void *arg) | +
This callback routine is invoked on the server side to handle incoming connections from remote clients. The user can pass an argument to this callback. The conn_request handle has to be released, either by ucp_ep_create or ucp_listener_reject routine.
+[in] | conn_request | Connection request handle. |
[in] | arg | User's argument for the callback. |
typedef struct ucp_listener_conn_handler ucp_listener_conn_handler_t | +
This structure is used for handling an incoming connection request on the listener. Setting this type of handler allows creating an endpoint on any other worker and not limited to the worker on which the listener was created.
+typedef enum ucp_wakeup_event_types ucp_wakeup_event_t | +
The enumeration allows specifying which events are expected on wakeup. Empty events are possible for any type of event except for UCP_WAKEUP_TX and UCP_WAKEUP_RX.
+enum ucp_worker_params_field | +
The enumeration allows specifying which fields in ucp_worker_params_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_WORKER_PARAM_FIELD_THREAD_MODE |
+ UCP thread mode + |
UCP_WORKER_PARAM_FIELD_CPU_MASK |
+ Worker's CPU bitmap + |
UCP_WORKER_PARAM_FIELD_EVENTS |
+ Worker's events bitmap + |
UCP_WORKER_PARAM_FIELD_USER_DATA |
+ User data + |
UCP_WORKER_PARAM_FIELD_EVENT_FD |
+ External event file descriptor + |
UCP_WORKER_PARAM_FIELD_FLAGS |
+ Worker flags + |
UCP_WORKER_PARAM_FIELD_NAME |
+ Worker name + |
UCP_WORKER_PARAM_FIELD_AM_ALIGNMENT |
+ Alignment of active messages on the receiver + |
UCP_WORKER_PARAM_FIELD_CLIENT_ID |
+ Client id + |
enum ucp_worker_flags_t | +
This enumeration allows specifying flags for ucp_worker_params_t::flags, which is used as parameter for ucp_worker_create.
+Enumerator | |
---|---|
UCP_WORKER_FLAG_IGNORE_REQUEST_LEAK |
+ Do not print warnings about request leaks + |
enum ucp_listener_params_field | +
The enumeration allows specifying which fields in ucp_listener_params_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_LISTENER_PARAM_FIELD_SOCK_ADDR |
+ Sock address and length. + |
UCP_LISTENER_PARAM_FIELD_ACCEPT_HANDLER |
+ User's callback and argument for handling the creation of an endpoint.User's callback and argument for handling the incoming connection request. + |
UCP_LISTENER_PARAM_FIELD_CONN_HANDLER | + |
enum ucp_worker_address_flags_t | +
The enumeration list describes possible UCP worker address flags, indicating what needs to be included to the worker address returned by ucp_worker_query() routine.
+Enumerator | |
---|---|
UCP_WORKER_ADDRESS_FLAG_NET_ONLY |
+ Pack addresses of network devices only. Using such shortened addresses for the remote node peers will reduce the amount of wireup data being exchanged during connection establishment phase. + |
enum ucp_worker_attr_field | +
The enumeration allows specifying which fields in ucp_worker_attr_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_WORKER_ATTR_FIELD_THREAD_MODE |
+ UCP thread mode + |
UCP_WORKER_ATTR_FIELD_ADDRESS |
+ UCP address + |
UCP_WORKER_ATTR_FIELD_ADDRESS_FLAGS |
+ UCP address flags + |
UCP_WORKER_ATTR_FIELD_MAX_AM_HEADER |
+ Maximum header size used by UCP AM API + |
UCP_WORKER_ATTR_FIELD_NAME |
+ UCP worker name + |
UCP_WORKER_ATTR_FIELD_MAX_INFO_STRING |
+ Maximum size of info string + |
enum ucp_worker_address_attr_field | +
The enumeration allows specifying which fields in ucp_worker_address_attr_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_WORKER_ADDRESS_ATTR_FIELD_UID |
+ Unique id of the worker + |
enum ucp_listener_attr_field | +
The enumeration allows specifying which fields in ucp_listener_attr_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_LISTENER_ATTR_FIELD_SOCKADDR |
+ Sockaddr used for listening + |
enum ucp_conn_request_attr_field | +
The enumeration allows specifying which fields in ucp_conn_request_attr_t are present. It is used to enable backward compatibility support.
+Enumerator | |
---|---|
UCP_CONN_REQUEST_ATTR_FIELD_CLIENT_ADDR |
+ Client's address + |
UCP_CONN_REQUEST_ATTR_FIELD_CLIENT_ID |
+ Remote client id + |
enum ucp_am_cb_flags | +
Flags that indicate how to handle UCP Active Messages.
+Enumerator | |
---|---|
UCP_AM_FLAG_WHOLE_MSG |
+ Indicates that the entire message will be handled in one callback. + |
UCP_AM_FLAG_PERSISTENT_DATA |
+ Guarantees that the specified ucp_am_recv_callback_t callback, will always be called with UCP_AM_RECV_ATTR_FLAG_DATA flag set, so the data will be accessible outside the callback, until ucp_am_data_release is called. + |
enum ucp_send_am_flags | +
Flags dictate the behavior of ucp_am_send_nb and ucp_am_send_nbx routines.
+Enumerator | |
---|---|
UCP_AM_SEND_FLAG_REPLY |
+ Force relevant reply endpoint to be passed to the data callback on the receiver. + |
UCP_AM_SEND_FLAG_EAGER |
+ Force UCP to use only eager protocol for AM sends. + |
UCP_AM_SEND_FLAG_RNDV |
+ Force UCP to use only rendezvous protocol for AM sends. + |
UCP_AM_SEND_FLAG_COPY_HEADER |
+ The flag indicates that the header should be copied to an internal buffer in case it's needed after the send function returns. If this flag is specified, the header can be released immediately after the send function returns, even if the non-blocking send request is not completed. + |
UCP_AM_SEND_REPLY |
+ Backward compatibility. + |
enum ucp_wakeup_event_types | +
The enumeration allows specifying which events are expected on wakeup. Empty events are possible for any type of event except for UCP_WAKEUP_TX and UCP_WAKEUP_RX.
+Enumerator | |
---|---|
UCP_WAKEUP_RMA |
+ Remote memory access send completion + |
UCP_WAKEUP_AMO |
+ Atomic operation send completion + |
UCP_WAKEUP_TAG_SEND |
+ Tag send completion + |
UCP_WAKEUP_TAG_RECV |
+ Tag receive completion + |
UCP_WAKEUP_TX |
+ This event type will generate an event on completion of any outgoing operation (complete or partial, according to the underlying protocol) for any type of transfer (send, atomic, or RMA). + |
UCP_WAKEUP_RX |
+ This event type will generate an event on completion of any receive operation (complete or partial, according to the underlying protocol). + |
UCP_WAKEUP_EDGE |
+ Use edge-triggered wakeup. The event file descriptor will be signaled only for new events, rather than existing ones. + |
ucs_status_t ucp_worker_create | +( | +ucp_context_h | +context, | +
+ | + | const ucp_worker_params_t * | +params, | +
+ | + | ucp_worker_h * | +worker_p | +
+ | ) | ++ |
This routine allocates and initializes a worker object. Each worker is associated with one and only one application context. In the same time, an application context can create multiple workers in order to enable concurrent access to communication resources. For example, application can allocate a dedicated worker for each application thread, where every worker can be progressed independently of others.
+[in] | context | Handle to UCP application context. |
[in] | params | User defined ucp_worker_params_t configurations for the UCP worker. |
[out] | worker_p | A pointer to the worker object allocated by the UCP library |
void ucp_worker_destroy | +( | +ucp_worker_h | +worker | ) | ++ |
This routine releases the resources associated with a UCP worker.
+The destroy process releases and shuts down all resources associated with the worker.
+[in] | worker | Worker object to destroy. |
ucs_status_t ucp_worker_query | +( | +ucp_worker_h | +worker, | +
+ | + | ucp_worker_attr_t * | +attr | +
+ | ) | ++ |
This routine fetches information about the worker.
+[in] | worker | Worker object to query. |
[out] | attr | Filled with attributes of worker. |
void ucp_worker_print_info | +( | +ucp_worker_h | +worker, | +
+ | + | FILE * | +stream | +
+ | ) | ++ |
This routine prints information about the protocols being used, thresholds, UCT transport methods, and other useful information associated with the worker.
+[in] | worker | Worker object to print information for. |
[in] | stream | Output stream to print the information to. |
void ucp_worker_release_address | +( | +ucp_worker_h | +worker, | +
+ | + | ucp_address_t * | +address | +
+ | ) | ++ |
This routine release an address handle associated within the worker object.
+[in] | worker | Worker object that is associated with the address object. |
[in] | address | Address to release; the address object has to be allocated using ucp_worker_query() routine. |
ucs_status_t ucp_worker_address_query | +( | +ucp_address_t * | +address, | +
+ | + | ucp_worker_address_attr_t * | +attr | +
+ | ) | ++ |
This routine fetches information about the worker address. The address can be either of local or remote worker.
+[in] | address | Worker address to query. |
[out] | attr | Filled with attributes of the worker address. |
unsigned ucp_worker_progress | +( | +ucp_worker_h | +worker | ) | ++ |
This routine explicitly progresses all communication operations on a worker.
+[in] | worker | Worker to progress. |
ssize_t ucp_stream_worker_poll | +( | +ucp_worker_h | +worker, | +
+ | + | ucp_stream_poll_ep_t * | +poll_eps, | +
+ | + | size_t | +max_eps, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
This non-blocking routine returns endpoints on a worker which are ready to consume streaming data. The ready endpoints are placed in poll_eps array, and the function return value indicates how many are there.
+[in] | worker | Worker to poll. |
[out] | poll_eps | Pointer to array of endpoints, should be allocated by user. |
[in] | max_eps | Maximum number of endpoints that should be filled in poll_eps. |
[in] | flags | Reserved for future use. |
ucs_status_t ucp_listener_create | +( | +ucp_worker_h | +worker, | +
+ | + | const ucp_listener_params_t * | +params, | +
+ | + | ucp_listener_h * | +listener_p | +
+ | ) | ++ |
This routine creates a new listener object that is bound to a specific local address. The listener will listen to incoming connection requests. After receiving a request from the remote peer, an endpoint to this peer will be created - either right away or by calling ucp_ep_create, as specified by the callback type in ucp_listener_params_t. The user's callback will be invoked once the endpoint is created.
+[in] | worker | Worker object to create the listener on. |
[in] | params | User defined ucp_listener_params_t configurations for the ucp_listener_h. |
[out] | listener_p | A handle to the created listener, can be released by calling ucp_listener_destroy |
void ucp_listener_destroy | +( | +ucp_listener_h | +listener | ) | ++ |
This routine unbinds the worker from the given handle and stops listening for incoming connection requests on it.
+[in] | listener | A handle to the listener to stop listening on. |
ucs_status_t ucp_listener_query | +( | +ucp_listener_h | +listener, | +
+ | + | ucp_listener_attr_t * | +attr | +
+ | ) | ++ |
This routine fetches information about the listener.
+[in] | listener | listener object to query. |
[out] | attr | Filled with attributes of the listener. |
ucs_status_t ucp_conn_request_query | +( | +ucp_conn_request_h | +conn_request, | +
+ | + | ucp_conn_request_attr_t * | +attr | +
+ | ) | ++ |
This routine fetches information about the connection request.
+[in] | conn_request | connection request object to query. |
[out] | attr | Filled with attributes of the connection request. |
ucs_status_t ucp_listener_reject | +( | +ucp_listener_h | +listener, | +
+ | + | ucp_conn_request_h | +conn_request | +
+ | ) | ++ |
Reject the incoming connection request and release associated resources. If the remote initiator endpoint has set an ucp_ep_params_t::err_handler, it will be invoked with status UCS_ERR_REJECTED.
+[in] | listener | Handle to the listener on which the connection request was received. |
[in] | conn_request | Handle to the connection request to reject. |
ucs_status_t ucp_worker_set_am_recv_handler | +( | +ucp_worker_h | +worker, | +
+ | + | const ucp_am_handler_param_t * | +param | +
+ | ) | ++ |
This routine installs a user defined callback to handle incoming Active Messages with a specific id. This callback is called whenever an Active Message that was sent from the remote peer by ucp_am_send_nbx is received on this worker.
+[in] | worker | UCP worker on which to set the Active Message handler. |
[in] | param | Active Message handler parameters, as defined by ucp_am_handler_param_t. |
ucs_status_t ucp_worker_fence | +( | +ucp_worker_h | +worker | ) | ++ |
This routine ensures ordering of non-blocking communication operations on the UCP worker. Communication operations issued on a particular endpoint created on the worker prior to this call are guaranteed to be completed before any communication operations issued on the same endpoint after this call.
+[in] | worker | UCP worker. |
ucs_status_ptr_t ucp_worker_flush_nbx | +( | +ucp_worker_h | +worker, | +
+ | + | const ucp_request_param_t * | +param | +
+ | ) | ++ |
This routine flushes all outstanding AMO and RMA communications on the worker. All the AMO and RMA operations issued on the worker prior to this call are completed both at the origin and at the target when this call returns.
+[in] | worker | UCP worker. |
[in] | param | Operation parameters, see ucp_request_param_t |
ucs_status_t ucp_worker_flush | +( | +ucp_worker_h | +worker | ) | ++ |
This routine flushes all outstanding AMO and RMA communications on the worker. All the AMO and RMA operations issued on the worker prior to this call are completed both at the origin and at the target when this call returns.
+[in] | worker | UCP worker. |
ucs_status_t ucp_worker_get_address | +( | +ucp_worker_h | +worker, | +
+ | + | ucp_address_t ** | +address_p, | +
+ | + | size_t * | +address_length_p | +
+ | ) | ++ |
This routine returns the address of the worker object. This address can be passed to remote instances of the UCP library in order to connect to this worker. The memory for the address handle is allocated by this function, and must be released by using ucp_worker_release_address() routine.
+[in] | worker | Worker object whose address to return. |
[out] | address_p | A pointer to the worker address. |
[out] | address_length_p | The size in bytes of the address. |
ucs_status_t ucp_worker_set_am_handler | +( | +ucp_worker_h | +worker, | +
+ | + | uint16_t | +id, | +
+ | + | ucp_am_callback_t | +cb, | +
+ | + | void * | +arg, | +
+ | + | uint32_t | +flags | +
+ | ) | ++ |
This routine installs a user defined callback to handle incoming Active Messages with a specific id. This callback is called whenever an Active Message that was sent from the remote peer by ucp_am_send_nb is received on this worker.
+[in] | worker | UCP worker on which to set the Active Message handler. |
[in] | id | Active Message id. |
[in] | cb | Active Message callback. NULL to clear. |
[in] | arg | Active Message argument, which will be passed in to every invocation of the callback as the arg argument. |
[in] | flags | Dictates how an Active Message is handled on the remote endpoint. Currently only UCP_AM_FLAG_WHOLE_MSG is supported, which indicates the callback will not be invoked until all data has arrived. |
ucs_status_ptr_t ucp_worker_flush_nb | +( | +ucp_worker_h | +worker, | +
+ | + | unsigned | +flags, | +
+ | + | ucp_send_callback_t | +cb | +
+ | ) | ++ |
This routine flushes all outstanding AMO and RMA communications on the worker. All the AMO and RMA operations issued on the worker prior to this call are completed both at the origin and at the target when this call returns.
+[in] | worker | UCP worker. |
[in] | flags | Flags for flush operation. Reserved for future use. |
[in] | cb | Callback which will be called when the flush operation completes. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Modules | |
UCS Communication Resource | |
This section describes UCS API.
++ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | ucs_sock_addr |
+Typedefs | |
typedef void(* | ucs_async_event_cb_t )(int id, ucs_event_set_types_t events, void *arg) |
typedef struct ucs_sock_addr | ucs_sock_addr_t |
typedef enum ucs_memory_type | ucs_memory_type_t |
Memory types. More... | |
typedef unsigned long | ucs_time_t |
typedef void * | ucs_status_ptr_t |
Status pointer. More... | |
+Enumerations | |
enum | ucs_callbackq_flags { UCS_CALLBACKQ_FLAG_FAST = UCS_BIT(0), +UCS_CALLBACKQ_FLAG_ONESHOT = UCS_BIT(1) + } |
enum | ucs_memory_type { + UCS_MEMORY_TYPE_HOST, +UCS_MEMORY_TYPE_CUDA, +UCS_MEMORY_TYPE_CUDA_MANAGED, +UCS_MEMORY_TYPE_ROCM, + + UCS_MEMORY_TYPE_ROCM_MANAGED, +UCS_MEMORY_TYPE_LAST, +UCS_MEMORY_TYPE_UNKNOWN = UCS_MEMORY_TYPE_LAST + + } |
Memory types. More... | |
enum | ucs_status_t { + UCS_OK = 0, +UCS_INPROGRESS = 1, +UCS_ERR_NO_MESSAGE = -1, +UCS_ERR_NO_RESOURCE = -2, + + UCS_ERR_IO_ERROR = -3, +UCS_ERR_NO_MEMORY = -4, +UCS_ERR_INVALID_PARAM = -5, +UCS_ERR_UNREACHABLE = -6, + + UCS_ERR_INVALID_ADDR = -7, +UCS_ERR_NOT_IMPLEMENTED = -8, +UCS_ERR_MESSAGE_TRUNCATED = -9, +UCS_ERR_NO_PROGRESS = -10, + + UCS_ERR_BUFFER_TOO_SMALL = -11, +UCS_ERR_NO_ELEM = -12, +UCS_ERR_SOME_CONNECTS_FAILED = -13, +UCS_ERR_NO_DEVICE = -14, + + UCS_ERR_BUSY = -15, +UCS_ERR_CANCELED = -16, +UCS_ERR_SHMEM_SEGMENT = -17, +UCS_ERR_ALREADY_EXISTS = -18, + + UCS_ERR_OUT_OF_RANGE = -19, +UCS_ERR_TIMED_OUT = -20, +UCS_ERR_EXCEEDS_LIMIT = -21, +UCS_ERR_UNSUPPORTED = -22, + + UCS_ERR_REJECTED = -23, +UCS_ERR_NOT_CONNECTED = -24, +UCS_ERR_CONNECTION_RESET = -25, +UCS_ERR_FIRST_LINK_FAILURE = -40, + + UCS_ERR_LAST_LINK_FAILURE = -59, +UCS_ERR_FIRST_ENDPOINT_FAILURE = -60, +UCS_ERR_ENDPOINT_TIMEOUT = -80, +UCS_ERR_LAST_ENDPOINT_FAILURE = -89, + + UCS_ERR_LAST = -100 + + } |
Status codes. More... | |
enum | ucs_thread_mode_t { UCS_THREAD_MODE_SINGLE, +UCS_THREAD_MODE_SERIALIZED, +UCS_THREAD_MODE_MULTI, +UCS_THREAD_MODE_LAST + } |
Thread sharing mode. More... | |
+Functions | |
ucs_status_t | ucs_async_set_event_handler (ucs_async_mode_t mode, int event_fd, ucs_event_set_types_t events, ucs_async_event_cb_t cb, void *arg, ucs_async_context_t *async) |
ucs_status_t | ucs_async_add_timer (ucs_async_mode_t mode, ucs_time_t interval, ucs_async_event_cb_t cb, void *arg, ucs_async_context_t *async, int *timer_id_p) |
ucs_status_t | ucs_async_remove_handler (int id, int sync) |
ucs_status_t | ucs_async_modify_handler (int fd, ucs_event_set_types_t events) |
ucs_status_t | ucs_async_context_create (ucs_async_mode_t mode, ucs_async_context_t **async_p) |
Create an asynchronous execution context. More... | |
void | ucs_async_context_destroy (ucs_async_context_t *async) |
Destroy the asynchronous execution context. More... | |
void | ucs_async_poll (ucs_async_context_t *async) |
This section describes a concept of the Communication Resource and routines associated with the concept.
+struct ucs_sock_addr | +
typedef void(* ucs_async_event_cb_t)(int id, ucs_event_set_types_t events, void *arg) | +
Async event callback.
+id | Event id (timer or file descriptor). |
events | The events that triggered the callback. |
arg | User-defined argument. |
typedef struct ucs_sock_addr ucs_sock_addr_t | +
BSD socket address specification.
+ +typedef enum ucs_memory_type ucs_memory_type_t | +
List of supported memory types.
+ +typedef unsigned long ucs_time_t | +
UCS time units. These are not necessarily aligned with metric time units. MUST compare short time values with UCS_SHORT_TIME_CMP to handle wrap-around.
+ +typedef void* ucs_status_ptr_t | +
A pointer can represent one of these values:
+enum ucs_callbackq_flags | +
enum ucs_memory_type | +
List of supported memory types.
+Enumerator | |
---|---|
UCS_MEMORY_TYPE_HOST |
+ Default system memory + |
UCS_MEMORY_TYPE_CUDA |
+ NVIDIA CUDA memory + |
UCS_MEMORY_TYPE_CUDA_MANAGED |
+ NVIDIA CUDA managed (or unified) memory + |
UCS_MEMORY_TYPE_ROCM |
+ AMD ROCM memory + |
UCS_MEMORY_TYPE_ROCM_MANAGED |
+ AMD ROCM managed system memory + |
UCS_MEMORY_TYPE_LAST | + |
UCS_MEMORY_TYPE_UNKNOWN | + |
enum ucs_status_t | +
Enumerator | |
---|---|
UCS_OK | + |
UCS_INPROGRESS | + |
UCS_ERR_NO_MESSAGE | + |
UCS_ERR_NO_RESOURCE | + |
UCS_ERR_IO_ERROR | + |
UCS_ERR_NO_MEMORY | + |
UCS_ERR_INVALID_PARAM | + |
UCS_ERR_UNREACHABLE | + |
UCS_ERR_INVALID_ADDR | + |
UCS_ERR_NOT_IMPLEMENTED | + |
UCS_ERR_MESSAGE_TRUNCATED | + |
UCS_ERR_NO_PROGRESS | + |
UCS_ERR_BUFFER_TOO_SMALL | + |
UCS_ERR_NO_ELEM | + |
UCS_ERR_SOME_CONNECTS_FAILED | + |
UCS_ERR_NO_DEVICE | + |
UCS_ERR_BUSY | + |
UCS_ERR_CANCELED | + |
UCS_ERR_SHMEM_SEGMENT | + |
UCS_ERR_ALREADY_EXISTS | + |
UCS_ERR_OUT_OF_RANGE | + |
UCS_ERR_TIMED_OUT | + |
UCS_ERR_EXCEEDS_LIMIT | + |
UCS_ERR_UNSUPPORTED | + |
UCS_ERR_REJECTED | + |
UCS_ERR_NOT_CONNECTED | + |
UCS_ERR_CONNECTION_RESET | + |
UCS_ERR_FIRST_LINK_FAILURE | + |
UCS_ERR_LAST_LINK_FAILURE | + |
UCS_ERR_FIRST_ENDPOINT_FAILURE | + |
UCS_ERR_ENDPOINT_TIMEOUT | + |
UCS_ERR_LAST_ENDPOINT_FAILURE | + |
UCS_ERR_LAST | + |
enum ucs_thread_mode_t | +
Specifies thread sharing mode of an object.
+Enumerator | |
---|---|
UCS_THREAD_MODE_SINGLE |
+ Only the master thread can access (i.e. the thread that initialized the context; multiple threads may exist and never access) + |
UCS_THREAD_MODE_SERIALIZED |
+ Multiple threads can access, but only one at a time + |
UCS_THREAD_MODE_MULTI |
+ Multiple threads can access concurrently + |
UCS_THREAD_MODE_LAST | + |
ucs_status_t ucs_async_set_event_handler | +( | +ucs_async_mode_t | +mode, | +
+ | + | int | +event_fd, | +
+ | + | ucs_event_set_types_t | +events, | +
+ | + | ucs_async_event_cb_t | +cb, | +
+ | + | void * | +arg, | +
+ | + | ucs_async_context_t * | +async | +
+ | ) | ++ |
Register a file descriptor for monitoring (call handler upon events). Every fd can have only one handler.
+mode | Thread or signal. |
event_fd | File descriptor to set handler for. |
events | Events to wait on (UCS_EVENT_SET_EVxxx bits). |
cb | Callback function to execute. |
arg | Argument to callback. |
async | Async context to which events are delivered. If NULL, safety is up to the user. |
ucs_status_t ucs_async_add_timer | +( | +ucs_async_mode_t | +mode, | +
+ | + | ucs_time_t | +interval, | +
+ | + | ucs_async_event_cb_t | +cb, | +
+ | + | void * | +arg, | +
+ | + | ucs_async_context_t * | +async, | +
+ | + | int * | +timer_id_p | +
+ | ) | ++ |
Add timer handler.
+mode | Thread or signal. |
interval | Timer interval. |
cb | Callback function to execute. |
arg | Argument to callback. |
async | Async context to which events are delivered. If NULL, safety is up to the user. |
timer_id_p | Filled with timer id. |
ucs_status_t ucs_async_remove_handler | +( | +int | +id, | +
+ | + | int | +sync | +
+ | ) | ++ |
Remove an event handler (Timer or event file).
+id | Timer/FD to remove. |
sync | If nonzero, wait until the handler for this event is not running anymore. If called from the context of the callback, the handler will be removed immediately after the current callback returns. |
ucs_status_t ucs_async_modify_handler | +( | +int | +fd, | +
+ | + | ucs_event_set_types_t | +events | +
+ | ) | ++ |
Modify events mask for an existing event handler (event file).
+fd | File descriptor modify events for. |
events | New set of events to wait on (UCS_EVENT_SET_EVxxx bits). |
ucs_status_t ucs_async_context_create | +( | +ucs_async_mode_t | +mode, | +
+ | + | ucs_async_context_t ** | +async_p | +
+ | ) | ++ |
Allocate and initialize an asynchronous execution context. This can be used to ensure safe event delivery.
+mode | Indicates whether to use signals or polling threads for waiting. |
async_p | Event context pointer to initialize. |
void ucs_async_context_destroy | +( | +ucs_async_context_t * | +async | ) | ++ |
Clean up the async context, and release system resources if possible. The context memory released.
+async | Asynchronous context to clean up. |
void ucs_async_poll | +( | +ucs_async_context_t * | +async | ) | ++ |
Poll on async context.
+async | Async context to poll on. NULL polls on all. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Typedefs | |
typedef ucs_status_t(* | uct_am_callback_t )(void *arg, void *data, size_t length, unsigned flags) |
Callback to process incoming active message. More... | |
typedef void(* | uct_am_tracer_t )(void *arg, uct_am_trace_type_t type, uint8_t id, const void *data, size_t length, char *buffer, size_t max) |
Callback to trace active messages. More... | |
+Enumerations | |
enum | uct_msg_flags { UCT_SEND_FLAG_SIGNALED = UCS_BIT(0), +UCT_SEND_FLAG_PEER_CHECK = UCS_BIT(1) + } |
Flags for active message send operation. More... | |
enum | uct_am_trace_type { + UCT_AM_TRACE_TYPE_SEND, +UCT_AM_TRACE_TYPE_RECV, +UCT_AM_TRACE_TYPE_SEND_DROP, +UCT_AM_TRACE_TYPE_RECV_DROP, + + UCT_AM_TRACE_TYPE_LAST + + } |
Trace types for active message tracer. More... | |
+Functions | |
ucs_status_t | uct_iface_set_am_handler (uct_iface_h iface, uint8_t id, uct_am_callback_t cb, void *arg, uint32_t flags) |
Set active message handler for the interface. More... | |
ucs_status_t | uct_iface_set_am_tracer (uct_iface_h iface, uct_am_tracer_t tracer, void *arg) |
Set active message tracer for the interface. More... | |
void | uct_iface_release_desc (void *desc) |
Release AM descriptor. More... | |
ucs_status_t | uct_ep_am_short (uct_ep_h ep, uint8_t id, uint64_t header, const void *payload, unsigned length) |
ucs_status_t | uct_ep_am_short_iov (uct_ep_h ep, uint8_t id, const uct_iov_t *iov, size_t iovcnt) |
Short io-vector send operation. More... | |
ssize_t | uct_ep_am_bcopy (uct_ep_h ep, uint8_t id, uct_pack_callback_t pack_cb, void *arg, unsigned flags) |
ucs_status_t | uct_ep_am_zcopy (uct_ep_h ep, uint8_t id, const void *header, unsigned header_length, const uct_iov_t *iov, size_t iovcnt, unsigned flags, uct_completion_t *comp) |
Send active message while avoiding local memory copy. More... | |
Defines active message functions.
+typedef ucs_status_t(* uct_am_callback_t)(void *arg, void *data, size_t length, unsigned flags) | +
When the callback is called, flags indicates how data should be handled. If flags contain UCT_CB_PARAM_FLAG_DESC value, it means data is part of a descriptor which must be released later by uct_iface_release_desc by the user if the callback returns UCS_INPROGRESS.
+[in] | arg | User-defined argument. |
[in] | data | Points to the received data. This may be a part of a descriptor which may be released later. |
[in] | length | Length of data. |
[in] | flags | Mask with uct_cb_param_flags |
UCS_OK | - descriptor was consumed, and can be released by the caller. |
UCS_INPROGRESS | - descriptor is owned by the callee, and would be released later. Supported only if flags contain UCT_CB_PARAM_FLAG_DESC value. Otherwise, this is an error. |
typedef void(* uct_am_tracer_t)(void *arg, uct_am_trace_type_t type, uint8_t id, const void *data, size_t length, char *buffer, size_t max) | +
Writes a string which represents active message contents into 'buffer'.
+[in] | arg | User-defined argument. |
[in] | type | Message type. |
[in] | id | Active message id. |
[in] | data | Points to the received data. |
[in] | length | Length of data. |
[out] | buffer | Filled with a debug information string. |
[in] | max | Maximal length of the string. |
enum uct_msg_flags | +
Enumerator | |
---|---|
UCT_SEND_FLAG_SIGNALED |
+ Trigger UCT_EVENT_RECV_SIG event on remote side. Make best effort attempt to avoid triggering UCT_EVENT_RECV event. Ignored if not supported by interface. + |
UCT_SEND_FLAG_PEER_CHECK |
+ Forces checking connectivity to a peer. If the connection is not alive, an error callback will be invoked. If the flag is not set, there is no guarantee that a connectivity error could be detected. + |
enum uct_am_trace_type | +
ucs_status_t uct_iface_set_am_handler | +( | +uct_iface_h | +iface, | +
+ | + | uint8_t | +id, | +
+ | + | uct_am_callback_t | +cb, | +
+ | + | void * | +arg, | +
+ | + | uint32_t | +flags | +
+ | ) | ++ |
Only one handler can be set of each active message ID, and setting a handler replaces the previous value. If cb == NULL, the current handler is removed.
+[in] | iface | Interface to set the active message handler for. |
[in] | id | Active message id. Must be 0..UCT_AM_ID_MAX-1. |
[in] | cb | Active message callback. NULL to clear. |
[in] | arg | Active message argument. |
[in] | flags | Required callback flags |
ucs_status_t uct_iface_set_am_tracer | +( | +uct_iface_h | +iface, | +
+ | + | uct_am_tracer_t | +tracer, | +
+ | + | void * | +arg | +
+ | ) | ++ |
Sets a function which dumps active message debug information to a buffer, which is printed every time an active message is sent or received, when data tracing is on. Without the tracer, only transport-level information is printed.
+[in] | iface | Interface to set the active message tracer for. |
[in] | tracer | Active message tracer. NULL to clear. |
[in] | arg | Tracer custom argument. |
void uct_iface_release_desc | +( | +void * | +desc | ) | ++ |
Release active message descriptor desc, which was passed to the active message callback, and owned by the callee.
+[in] | desc | Descriptor to release. |
ucs_status_t uct_ep_am_short | +( | +uct_ep_h | +ep, | +
+ | + | uint8_t | +id, | +
+ | + | uint64_t | +header, | +
+ | + | const void * | +payload, | +
+ | + | unsigned | +length | +
+ | ) | ++ |
ucs_status_t uct_ep_am_short_iov | +( | +uct_ep_h | +ep, | +
+ | + | uint8_t | +id, | +
+ | + | const uct_iov_t * | +iov, | +
+ | + | size_t | +iovcnt | +
+ | ) | ++ |
This routine sends a message using short protocol. The input data in iov array of uct_iov_t structures is sent to remote side to contiguous buffer keeping the order of the data in the array.
+[in] | ep | Destination endpoint handle. |
[in] | id | Active message id. Must be in range 0..UCT_AM_ID_MAX-1. |
[in] | iov | Points to an array of uct_iov_t structures. The iov pointer must be a valid address of an array of uct_iov_t structures. A particular structure pointer must be a valid address. A NULL terminated array is not required. stride and count fields in uct_iov_t structure are ignored in current implementation. The total size of the data buffers in the array is limited by uct_iface_attr::cap::am::max_short. |
[in] | iovcnt | Size of the iov data uct_iov_t structures array. If iovcnt is zero, the data is considered empty. iovcnt is limited by uct_iface_attr::cap::am::max_iov. |
ssize_t uct_ep_am_bcopy | +( | +uct_ep_h | +ep, | +
+ | + | uint8_t | +id, | +
+ | + | uct_pack_callback_t | +pack_cb, | +
+ | + | void * | +arg, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
ucs_status_t uct_ep_am_zcopy | +( | +uct_ep_h | +ep, | +
+ | + | uint8_t | +id, | +
+ | + | const void * | +header, | +
+ | + | unsigned | +header_length, | +
+ | + | const uct_iov_t * | +iov, | +
+ | + | size_t | +iovcnt, | +
+ | + | unsigned | +flags, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
The input data in iov array of uct_iov_t structures sent to remote side ("gather output"). Buffers in iov are processed in array order. This means that the function complete iov[0] before proceeding to iov[1], and so on.
+[in] | ep | Destination endpoint handle. |
[in] | id | Active message id. Must be in range 0..UCT_AM_ID_MAX-1. |
[in] | header | Active message header. |
[in] | header_length | Active message header length in bytes. |
[in] | iov | Points to an array of uct_iov_t structures. The iov pointer must be a valid address of an array of uct_iov_t structures. A particular structure pointer must be a valid address. A NULL terminated array is not required. |
[in] | iovcnt | Size of the iov data uct_iov_t structures array. If iovcnt is zero, the data is considered empty. iovcnt is limited by uct_iface_attr::cap::am::max_iov. |
[in] | flags | Active message flags, see uct_msg_flags. |
[in] | comp | Completion handle as defined by uct_completion_t. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Functions | |
ucs_status_t | uct_ep_atomic_cswap64 (uct_ep_h ep, uint64_t compare, uint64_t swap, uint64_t remote_addr, uct_rkey_t rkey, uint64_t *result, uct_completion_t *comp) |
ucs_status_t | uct_ep_atomic_cswap32 (uct_ep_h ep, uint32_t compare, uint32_t swap, uint64_t remote_addr, uct_rkey_t rkey, uint32_t *result, uct_completion_t *comp) |
ucs_status_t | uct_ep_atomic32_post (uct_ep_h ep, uct_atomic_op_t opcode, uint32_t value, uint64_t remote_addr, uct_rkey_t rkey) |
ucs_status_t | uct_ep_atomic64_post (uct_ep_h ep, uct_atomic_op_t opcode, uint64_t value, uint64_t remote_addr, uct_rkey_t rkey) |
ucs_status_t | uct_ep_atomic32_fetch (uct_ep_h ep, uct_atomic_op_t opcode, uint32_t value, uint32_t *result, uint64_t remote_addr, uct_rkey_t rkey, uct_completion_t *comp) |
ucs_status_t | uct_ep_atomic64_fetch (uct_ep_h ep, uct_atomic_op_t opcode, uint64_t value, uint64_t *result, uint64_t remote_addr, uct_rkey_t rkey, uct_completion_t *comp) |
Defines atomic operations.
+ucs_status_t uct_ep_atomic_cswap64 | +( | +uct_ep_h | +ep, | +
+ | + | uint64_t | +compare, | +
+ | + | uint64_t | +swap, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey, | +
+ | + | uint64_t * | +result, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
ucs_status_t uct_ep_atomic_cswap32 | +( | +uct_ep_h | +ep, | +
+ | + | uint32_t | +compare, | +
+ | + | uint32_t | +swap, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey, | +
+ | + | uint32_t * | +result, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
ucs_status_t uct_ep_atomic32_post | +( | +uct_ep_h | +ep, | +
+ | + | uct_atomic_op_t | +opcode, | +
+ | + | uint32_t | +value, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey | +
+ | ) | ++ |
ucs_status_t uct_ep_atomic64_post | +( | +uct_ep_h | +ep, | +
+ | + | uct_atomic_op_t | +opcode, | +
+ | + | uint64_t | +value, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey | +
+ | ) | ++ |
ucs_status_t uct_ep_atomic32_fetch | +( | +uct_ep_h | +ep, | +
+ | + | uct_atomic_op_t | +opcode, | +
+ | + | uint32_t | +value, | +
+ | + | uint32_t * | +result, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
ucs_status_t uct_ep_atomic64_fetch | +( | +uct_ep_h | +ep, | +
+ | + | uct_atomic_op_t | +opcode, | +
+ | + | uint64_t | +value, | +
+ | + | uint64_t * | +result, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Modules | |
UCT Communication Resource | |
UCT Communication Context | |
UCT Memory Domain | |
UCT Active messages | |
UCT Remote memory access operations | |
UCT Atomic operations | |
UCT Tag matching operations | |
UCT client-server operations | |
This section describes UCT API.
++ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | uct_ep_connect_params |
Parameters for connecting a UCT endpoint by uct_ep_connect. More... | |
struct | uct_cm_attr |
Connection manager attributes, capabilities and limitations. More... | |
struct | uct_listener_attr |
UCT listener attributes, capabilities and limitations. More... | |
struct | uct_listener_params |
Parameters for creating a listener object uct_listener_h by uct_listener_create. More... | |
struct | uct_cm_ep_priv_data_pack_args |
Arguments to the client-server private data pack callback. More... | |
struct | uct_cm_ep_resolve_args |
Arguments to the client-server resolved callback. More... | |
struct | uct_cm_remote_data |
Data received from the remote peer. More... | |
struct | uct_cm_listener_conn_request_args |
Arguments to the listener's connection request callback. More... | |
struct | uct_cm_ep_client_connect_args |
Arguments to the client's connect callback. More... | |
struct | uct_cm_ep_server_conn_notify_args |
Arguments to the server's notify callback. More... | |
+Typedefs | |
typedef struct +uct_cm_ep_priv_data_pack_args | uct_cm_ep_priv_data_pack_args_t |
Arguments to the client-server private data pack callback. More... | |
typedef struct +uct_cm_ep_resolve_args | uct_cm_ep_resolve_args_t |
Arguments to the client-server resolved callback. More... | |
typedef struct uct_cm_remote_data | uct_cm_remote_data_t |
Data received from the remote peer. More... | |
typedef struct +uct_cm_listener_conn_request_args | uct_cm_listener_conn_request_args_t |
Arguments to the listener's connection request callback. More... | |
typedef struct +uct_cm_ep_client_connect_args | uct_cm_ep_client_connect_args_t |
Arguments to the client's connect callback. More... | |
typedef struct +uct_cm_ep_server_conn_notify_args | uct_cm_ep_server_conn_notify_args_t |
Arguments to the server's notify callback. More... | |
typedef void(* | uct_sockaddr_conn_request_callback_t )(uct_iface_h iface, void *arg, uct_conn_request_h conn_request, const void *conn_priv_data, size_t length) |
Callback to process an incoming connection request on the server side. More... | |
typedef void(* | uct_cm_listener_conn_request_callback_t )(uct_listener_h listener, void *arg, const uct_cm_listener_conn_request_args_t *conn_req_args) |
Callback to process an incoming connection request on the server side listener in a connection manager. More... | |
typedef void(* | uct_cm_ep_server_conn_notify_callback_t )(uct_ep_h ep, void *arg, const uct_cm_ep_server_conn_notify_args_t *connect_args) |
Callback to process an incoming connection establishment acknowledgment on the server side listener, from the client, which indicates that the client side is connected. The callback also notifies the server side of a local error on a not-yet-connected endpoint. More... | |
typedef void(* | uct_cm_ep_client_connect_callback_t )(uct_ep_h ep, void *arg, const uct_cm_ep_client_connect_args_t *connect_args) |
Callback to process an incoming connection response on the client side from the server or handle a local error on a not-yet-connected endpoint. More... | |
typedef void(* | uct_ep_disconnect_cb_t )(uct_ep_h ep, void *arg) |
Callback to handle the disconnection of the remote peer. More... | |
typedef ssize_t(* | uct_cm_ep_priv_data_pack_callback_t )(void *arg, const uct_cm_ep_priv_data_pack_args_t *pack_args, void *priv_data) |
Callback to fill the user's private data in a client-server flow. More... | |
typedef ucs_status_t(* | uct_cm_ep_resolve_callback_t )(void *user_data, const uct_cm_ep_resolve_args_t *resolve_args) |
Callback to notify that the client side endpoint is bound to a local device. More... | |
+Enumerations | |
enum | uct_cm_attr_field { UCT_CM_ATTR_FIELD_MAX_CONN_PRIV = UCS_BIT(0) + } |
UCT connection manager attributes field mask. More... | |
enum | uct_listener_attr_field { UCT_LISTENER_ATTR_FIELD_SOCKADDR = UCS_BIT(0) + } |
UCT listener attributes field mask. More... | |
enum | uct_listener_params_field { UCT_LISTENER_PARAM_FIELD_BACKLOG = UCS_BIT(0), +UCT_LISTENER_PARAM_FIELD_CONN_REQUEST_CB = UCS_BIT(1), +UCT_LISTENER_PARAM_FIELD_USER_DATA = UCS_BIT(2) + } |
UCT listener created by uct_listener_create parameters field mask. More... | |
enum | uct_ep_connect_params_field { UCT_EP_CONNECT_PARAM_FIELD_PRIVATE_DATA = UCS_BIT(0), +UCT_EP_CONNECT_PARAM_FIELD_PRIVATE_DATA_LENGTH = UCS_BIT(1) + } |
UCT endpoint connected by uct_ep_connect parameters field mask. More... | |
enum | uct_cm_ep_priv_data_pack_args_field { UCT_CM_EP_PRIV_DATA_PACK_ARGS_FIELD_DEVICE_NAME = UCS_BIT(0) + } |
Client-Server private data pack callback arguments field mask. More... | |
enum | uct_cm_ep_resolve_args_field { UCT_CM_EP_RESOLVE_ARGS_FIELD_DEV_NAME = UCS_BIT(0), +UCT_CM_EP_RESOLVE_ARGS_FIELD_STATUS = UCS_BIT(1) + } |
Client-Server resolve callback arguments field mask. More... | |
enum | uct_cm_remote_data_field { UCT_CM_REMOTE_DATA_FIELD_DEV_ADDR = UCS_BIT(0), +UCT_CM_REMOTE_DATA_FIELD_DEV_ADDR_LENGTH = UCS_BIT(1), +UCT_CM_REMOTE_DATA_FIELD_CONN_PRIV_DATA = UCS_BIT(2), +UCT_CM_REMOTE_DATA_FIELD_CONN_PRIV_DATA_LENGTH = UCS_BIT(3) + } |
Remote data attributes field mask. More... | |
enum | uct_cm_listener_conn_request_args_field { UCT_CM_LISTENER_CONN_REQUEST_ARGS_FIELD_DEV_NAME = UCS_BIT(0), +UCT_CM_LISTENER_CONN_REQUEST_ARGS_FIELD_CONN_REQUEST = UCS_BIT(1), +UCT_CM_LISTENER_CONN_REQUEST_ARGS_FIELD_REMOTE_DATA = UCS_BIT(2), +UCT_CM_LISTENER_CONN_REQUEST_ARGS_FIELD_CLIENT_ADDR = UCS_BIT(3) + } |
Listener's connection request callback arguments field mask. More... | |
enum | uct_cm_ep_client_connect_args_field { UCT_CM_EP_CLIENT_CONNECT_ARGS_FIELD_REMOTE_DATA = UCS_BIT(0), +UCT_CM_EP_CLIENT_CONNECT_ARGS_FIELD_STATUS = UCS_BIT(1) + } |
Field mask flags for client-side connection established callback. More... | |
enum | uct_cm_ep_server_conn_notify_args_field { UCT_CM_EP_SERVER_CONN_NOTIFY_ARGS_FIELD_STATUS = UCS_BIT(0) + } |
Field mask flags for server-side connection established notification callback. More... | |
+Functions | |
ucs_status_t | uct_iface_accept (uct_iface_h iface, uct_conn_request_h conn_request) |
Accept connection request. More... | |
ucs_status_t | uct_iface_reject (uct_iface_h iface, uct_conn_request_h conn_request) |
Reject connection request. Will invoke an error handler uct_error_handler_t on the remote transport interface, if set. More... | |
ucs_status_t | uct_ep_connect (uct_ep_h ep, const uct_ep_connect_params_t *params) |
Connect a client side endpoint after it is bound to a local network device, i.e. uct_ep_params_t::cm_resolve_cb was invoked. More... | |
ucs_status_t | uct_ep_disconnect (uct_ep_h ep, unsigned flags) |
Initiate a disconnection of an endpoint connected to a sockaddr by a connection manager uct_cm_h. More... | |
ucs_status_t | uct_cm_open (uct_component_h component, uct_worker_h worker, const uct_cm_config_t *config, uct_cm_h *cm_p) |
Open a connection manager. More... | |
void | uct_cm_close (uct_cm_h cm) |
Close a connection manager. More... | |
ucs_status_t | uct_cm_query (uct_cm_h cm, uct_cm_attr_t *cm_attr) |
Get connection manager attributes. More... | |
ucs_status_t | uct_cm_config_read (uct_component_h component, const char *env_prefix, const char *filename, uct_cm_config_t **config_p) |
Read the configuration for a connection manager. More... | |
ucs_status_t | uct_cm_client_ep_conn_notify (uct_ep_h ep) |
Notify the server about client-side connection establishment. More... | |
ucs_status_t | uct_listener_create (uct_cm_h cm, const struct sockaddr *saddr, socklen_t socklen, const uct_listener_params_t *params, uct_listener_h *listener_p) |
Create a new transport listener object. More... | |
void | uct_listener_destroy (uct_listener_h listener) |
Destroy a transport listener. More... | |
ucs_status_t | uct_listener_reject (uct_listener_h listener, uct_conn_request_h conn_request) |
Reject a connection request. More... | |
ucs_status_t | uct_listener_query (uct_listener_h listener, uct_listener_attr_t *listener_attr) |
Get attributes specific to a particular listener. More... | |
Defines client-server operations. The client-server API allows the connection establishment between an active side - a client, and its peer - the passive side - a server. The connection can be established through a UCT transport that supports listening and connecting via IP address and port (listening can also be on INADDR_ANY).
+The following is a general overview of the operations on the server side:
+Connecting: uct_cm_open Open a connection manager. uct_listener_create Create a listener on the CM and start listening on a given IP,port / INADDR_ANY. uct_cm_listener_conn_request_callback_t This callback is invoked by the UCT transport to handle an incoming connection request from a client. Accept or reject the client's connection request. uct_ep_create Connect to the client by creating an endpoint if the request is accepted. The server creates a new endpoint for every connection request that it accepts. uct_cm_ep_server_conn_notify_callback_t This callback is invoked by the UCT transport to handle the connection notification from the client.
+Disconnecting: uct_ep_disconnect Disconnect the server's endpoint from the client. Can be called when initiating a disconnect or when receiving a disconnect notification from the remote side. uct_ep_disconnect_cb_t This callback is invoked by the UCT transport when the client side calls uct_ep_disconnect as well. uct_ep_destroy Destroy the endpoint connected to the remote peer. If this function is called before the endpoint was disconnected, the uct_ep_disconnect_cb_t will not be invoked.
+Destroying the server's resources: uct_listener_destroy Destroy the listener object. uct_cm_close Close the connection manager.
+The following is a general overview of the operations on the client side:
+Connecting: uct_cm_open Open a connection manager. uct_ep_create Create an endpoint for establishing a connection to the server. uct_cm_ep_resolve_callback_t This callback is invoked on the client side of the connection manager, after the remote server address was resolved to the local device to be used for connection establishment. uct_ep_connect This function should be called on the client side, in order to send private data and resume connection establishment, following an address-resolved notification via uct_cm_ep_resolve_callback_t. uct_cm_ep_client_connect_callback_t This callback is invoked by the UCT transport to handle a connection response from the server. After invoking this callback, the UCT transport will finalize the client's connection to the server. uct_cm_client_ep_conn_notify After the client's connection establishment is completed, the client should call this function in which it sends a notification message to the server stating that it (the client) is connected. The notification message that is sent depends on the transport's implementation.
+Disconnecting: uct_ep_disconnect Disconnect the client's endpoint from the server. Can be called when initiating a disconnect or when receiving a disconnect notification from the remote side. uct_ep_disconnect_cb_t This callback is invoked by the UCT transport when the server side calls uct_ep_disconnect as well. uct_ep_destroy Destroy the endpoint connected to the remote peer.
+Destroying the client's resources: uct_cm_close Close the connection manager.
+struct uct_ep_connect_params | +
Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_ep_connect_params_field. Fields not specified by this mask will be ignored. + |
+const void * | ++private_data | +
+ User's private data to be passed from client to server. + |
+size_t | ++private_data_length | +
+ Length of uct_ep_connect_params::private_data, the maximal allowed value is indicated by the uct_cm_attr::max_conn_priv. + |
struct uct_cm_attr | +
Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_cm_attr_field. Fields not specified by this mask will be ignored. + |
+size_t | ++max_conn_priv | +
+ Max size of the connection manager's private data used for connection establishment with sockaddr. + |
struct uct_listener_attr | +
Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_listener_attr_field. Fields not specified by this mask will be ignored. + |
+struct sockaddr_storage | ++sockaddr | +
+ Sockaddr on which this listener is listening. + |
struct uct_listener_params | +
Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_listener_params_field. Fields not specified by this mask will be ignored. + |
+int | ++backlog | +
+ Backlog of incoming connection requests. If specified, must be a positive value. If not specified, each CM component will use its maximal allowed value, based on the system's setting. + |
+uct_cm_listener_conn_request_callback_t | ++conn_request_cb | +
+ Callback function for handling incoming connection requests. + |
+void * | ++user_data | +
+ User data associated with the listener. + |
struct uct_cm_ep_priv_data_pack_args | +
Used with the client-server API on a connection manager.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_cm_ep_priv_data_pack_args_field. Fields not specified by this mask should not be accessed by the callback. + |
+char | ++dev_name[UCT_DEVICE_NAME_MAX] | +
+ Device name. This routine may fill the user's private data according to the given device name. The device name that is passed to this routine, corresponds to uct_tl_resource_desc_t::dev_name as returned from uct_md_query_tl_resources. + |
struct uct_cm_ep_resolve_args | +
Used with the client-server API on a connection manager.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_cm_ep_resolve_args_field. Fields not specified by this mask should not be accessed by the callback. + |
+char | ++dev_name[UCT_DEVICE_NAME_MAX] | +
+ Device name indicates the device that the endpoint was bound to during address and route resolution. The device name that is passed to this callback corresponds to uct_tl_resource_desc_t::dev_name as returned from uct_md_query_tl_resources. + |
+ucs_status_t | ++status | +
+ Indicates address resolution status: UCS_OK - address of the remote server was resolved successfully. UCS_ERR_UNREACHABLE - the remote server is unreachable. Otherwise - indicates an internal connection establishment error on the local (client) side. + |
struct uct_cm_remote_data | +
The remote peer's device address, the data received from it and their lengths. Used with the client-server API on a connection manager.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_cm_remote_data_field. Fields not specified by this mask will be ignored. + |
+const uct_device_addr_t * | ++dev_addr | +
+ Device address of the remote peer. + |
+size_t | ++dev_addr_length | +
+ Length of the remote device address. + |
+const void * | ++conn_priv_data | +
+ Pointer to the received data. This is the private data that was passed to uct_ep_params_t::sockaddr_pack_cb. + |
+size_t | ++conn_priv_data_length | +
+ Length of the received data from the peer. + |
struct uct_cm_listener_conn_request_args | +
The local device name, connection request handle and the data the client sent. Used with the client-server API on a connection manager.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_cm_listener_conn_request_args_field. Fields not specified by this mask should not be accessed by the callback. + |
+char | ++dev_name[UCT_DEVICE_NAME_MAX] | +
+ Local device name which handles the incoming connection request. + |
+uct_conn_request_h | ++conn_request | +
+ Connection request handle. Can be passed to this callback from the transport and will be used by it to accept or reject the connection request from the client. + |
+const uct_cm_remote_data_t * | ++remote_data | +
+ Remote data from the client. + |
+ucs_sock_addr_t | ++client_address | +
+ Client's address. + |
struct uct_cm_ep_client_connect_args | +
Used with the client-server API on a connection manager.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_cm_ep_client_connect_args_field. Fields not specified by this mask should not be accessed by the callback. + |
+const uct_cm_remote_data_t * | ++remote_data | +
+ Remote data from the server. + |
+ucs_status_t | ++status | +
+ Indicates the connection establishment response from the remote server: UCS_OK - the remote server accepted the connection request. UCS_ERR_REJECTED - the remote server rejected the connection request. UCS_ERR_CONNECTION_RESET - the server's connection was reset during the connection establishment to the client. Otherwise - indicates an internal connection establishment error on the local (client) side. + |
struct uct_cm_ep_server_conn_notify_args | +
Used with the client-server API on a connection manager.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_cm_ep_server_conn_notify_args_field. Fields not specified by this mask should not be accessed by the callback. + |
+ucs_status_t | ++status | +
+ Indicates the client's ucs_status_t status: UCS_OK - the client completed its connection establishment and called uct_cm_client_ep_conn_notify UCS_ERR_CONNECTION_RESET - the client's connection was reset during the connection establishment to the server. Otherwise - indicates an internal connection establishment error on the local (server) side. + |
typedef struct uct_cm_ep_priv_data_pack_args uct_cm_ep_priv_data_pack_args_t | +
Used with the client-server API on a connection manager.
+ +typedef struct uct_cm_ep_resolve_args uct_cm_ep_resolve_args_t | +
Used with the client-server API on a connection manager.
+ +typedef struct uct_cm_remote_data uct_cm_remote_data_t | +
The remote peer's device address, the data received from it and their lengths. Used with the client-server API on a connection manager.
+ +typedef struct uct_cm_listener_conn_request_args uct_cm_listener_conn_request_args_t | +
The local device name, connection request handle and the data the client sent. Used with the client-server API on a connection manager.
+ +typedef struct uct_cm_ep_client_connect_args uct_cm_ep_client_connect_args_t | +
Used with the client-server API on a connection manager.
+ +typedef struct uct_cm_ep_server_conn_notify_args uct_cm_ep_server_conn_notify_args_t | +
Used with the client-server API on a connection manager.
+ +typedef void(* uct_sockaddr_conn_request_callback_t)(uct_iface_h iface, void *arg, uct_conn_request_h conn_request, const void *conn_priv_data, size_t length) | +
This callback routine will be invoked on the server side upon receiving an incoming connection request. It should be set by the server side while initializing an interface. Incoming data is placed inside the conn_priv_data buffer. This callback has to be thread safe. Other than communication progress routines, it is allowed to call other UCT communication routines from this callback.
+[in] | iface | Transport interface. |
[in] | arg | User defined argument for this callback. |
[in] | conn_request | Transport level connection request. The user should accept or reject the request by calling uct_iface_accept or uct_iface_reject routines respectively. conn_request should not be used outside the scope of this callback. |
[in] | conn_priv_data | Points to the received data. This is the private data that was passed to the uct_ep_params_t::sockaddr_pack_cb on the client side. |
[in] | length | Length of the received data. |
typedef void(* uct_cm_listener_conn_request_callback_t)(uct_listener_h listener, void *arg, const uct_cm_listener_conn_request_args_t *conn_req_args) | +
This callback routine will be invoked on the server side upon receiving an incoming connection request. It should be set by the server side while initializing a listener in a connection manager. This callback has to be thread safe. Other than communication progress routines, it is allowed to call other UCT communication routines from this callback.
+[in] | listener | Transport listener. |
[in] | arg | User argument for this callback as defined in uct_listener_params_t::user_data |
[in] | conn_req_args | Listener's arguments to handle the connection request from the client. |
typedef void(* uct_cm_ep_server_conn_notify_callback_t)(uct_ep_h ep, void *arg, const uct_cm_ep_server_conn_notify_args_t *connect_args) | +
This callback routine will be invoked on the server side upon receiving an incoming connection establishment acknowledgment from the client, which is sent from it once the client is connected to the server. Used to connect the server side to the client or handle an error from it - depending on the status field. This callback will also be invoked in the event of an internal local error with a failed uct_cm_ep_server_conn_notify_args::status if the endpoint was not connected yet. This callback has to be thread safe. Other than communication progress routines, it is permissible to call other UCT communication routines from this callback.
+[in] | ep | Transport endpoint. |
[in] | arg | User argument for this callback as defined in uct_ep_params_t::user_data |
[in] | connect_args | Server's connect callback arguments. |
typedef void(* uct_cm_ep_client_connect_callback_t)(uct_ep_h ep, void *arg, const uct_cm_ep_client_connect_args_t *connect_args) | +
This callback routine will be invoked on the client side upon receiving an incoming connection response from the server. Used to connect the client side to the server or handle an error from it - depending on the status field. This callback will also be invoked in the event of an internal local error with a failed uct_cm_ep_client_connect_args::status if the endpoint was not connected yet. This callback has to be thread safe. Other than communication progress routines, it is permissible to call other UCT communication routines from this callback.
+[in] | ep | Transport endpoint. |
[in] | arg | User argument for this callback as defined in uct_ep_params_t::user_data. |
[in] | connect_args | Client's connect callback arguments |
typedef void(* uct_ep_disconnect_cb_t)(uct_ep_h ep, void *arg) | +
This callback routine will be invoked on the client and server sides upon a disconnect of the remote peer. It will disconnect the given endpoint from the remote peer. This callback won't be invoked if the endpoint was not connected to the remote peer yet. This callback has to be thread safe. Other than communication progress routines, it is permissible to call other UCT communication routines from this callback.
+[in] | ep | Transport endpoint to disconnect. |
[in] | arg | User argument for this callback as defined in uct_ep_params_t::user_data. |
typedef ssize_t(* uct_cm_ep_priv_data_pack_callback_t)(void *arg, const uct_cm_ep_priv_data_pack_args_t *pack_args, void *priv_data) | +
This callback routine will be invoked on the client side, before sending the transport's connection request to the server, or on the server side before sending a connection response to the client. This callback routine can be set when creating an endpoint. The user's private data should be placed inside the priv_data buffer to be sent to the remote side. The maximal allowed length of the private data is indicated by the field max_conn_priv inside uct_iface_attr or inside uct_cm_attr when using a connection manager. Communication progress routines should not be called from this callback. It is allowed to call other UCT communication routines from this callback.
+[in] | arg | User defined argument for this callback. |
[in] | pack_args | Handle for the the private data packing. |
[out] | priv_data | User's private data to be passed to the remote side. |
typedef ucs_status_t(* uct_cm_ep_resolve_callback_t)(void *user_data, const uct_cm_ep_resolve_args_t *resolve_args) | +
This callback routine will be invoked, when the client side endpoint is bound to a local device. The callback routine can be set when creating an endpoint. Communication progress routines should not be called from this callback. It is allowed to call other UCT communication routines from this callback.
+[in] | user_data | User argument as defined in uct_ep_params_t::user_data. |
[in] | resolve_args | Handle for the extra arguments provided by the transport. |
enum uct_cm_attr_field | +
The enumeration allows specifying which fields in uct_cm_attr_t are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_CM_ATTR_FIELD_MAX_CONN_PRIV |
+ Enables uct_cm_attr::max_conn_priv + |
enum uct_listener_attr_field | +
The enumeration allows specifying which fields in uct_listener_attr_t are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_LISTENER_ATTR_FIELD_SOCKADDR |
+ Enables uct_listener_attr::sockaddr + |
enum uct_listener_params_field | +
The enumeration allows specifying which fields in uct_listener_params_t are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_LISTENER_PARAM_FIELD_BACKLOG |
+ Enables uct_listener_params::backlog + |
UCT_LISTENER_PARAM_FIELD_CONN_REQUEST_CB | + + |
UCT_LISTENER_PARAM_FIELD_USER_DATA |
+ Enables uct_listener_params::user_data + |
enum uct_ep_connect_params_field | +
The enumeration allows specifying which fields in uct_ep_connect_params_t are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_EP_CONNECT_PARAM_FIELD_PRIVATE_DATA | + + |
UCT_EP_CONNECT_PARAM_FIELD_PRIVATE_DATA_LENGTH | + + |
enum uct_cm_ep_priv_data_pack_args_field | +
The enumeration allows specifying which fields in uct_cm_ep_priv_data_pack_args are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_CM_EP_PRIV_DATA_PACK_ARGS_FIELD_DEVICE_NAME |
+ Enables uct_cm_ep_priv_data_pack_args::dev_name Indicates that dev_name field in uct_cm_ep_priv_data_pack_args_t is valid. + |
enum uct_cm_ep_resolve_args_field | +
The enumeration allows specifying which fields in uct_cm_ep_resolve_args are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_CM_EP_RESOLVE_ARGS_FIELD_DEV_NAME |
+ Indicates that uct_cm_ep_resolve_args::dev_name is valid. + |
UCT_CM_EP_RESOLVE_ARGS_FIELD_STATUS |
+ Indicates that uct_cm_ep_resolve_args::status is valid. + |
enum uct_cm_remote_data_field | +
The enumeration allows specifying which fields in uct_cm_remote_data are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_CM_REMOTE_DATA_FIELD_DEV_ADDR |
+ Enables uct_cm_remote_data::dev_addr + |
UCT_CM_REMOTE_DATA_FIELD_DEV_ADDR_LENGTH | + + |
UCT_CM_REMOTE_DATA_FIELD_CONN_PRIV_DATA | + + |
UCT_CM_REMOTE_DATA_FIELD_CONN_PRIV_DATA_LENGTH | + + |
enum uct_cm_listener_conn_request_args_field | +
The enumeration allows specifying which fields in uct_cm_listener_conn_request_args are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_CM_LISTENER_CONN_REQUEST_ARGS_FIELD_DEV_NAME |
+ Enables uct_cm_listener_conn_request_args::dev_name Indicates that dev_name field in uct_cm_listener_conn_request_args_t is valid. + |
UCT_CM_LISTENER_CONN_REQUEST_ARGS_FIELD_CONN_REQUEST |
+ Enables uct_cm_listener_conn_request_args::conn_request Indicates that conn_request field in uct_cm_listener_conn_request_args_t is valid. + |
UCT_CM_LISTENER_CONN_REQUEST_ARGS_FIELD_REMOTE_DATA |
+ Enables uct_cm_listener_conn_request_args::remote_data Indicates that remote_data field in uct_cm_listener_conn_request_args_t is valid. + |
UCT_CM_LISTENER_CONN_REQUEST_ARGS_FIELD_CLIENT_ADDR |
+ Enables uct_cm_listener_conn_request_args::client_address Indicates that client_address field in uct_cm_listener_conn_request_args_t is valid. + |
enum uct_cm_ep_client_connect_args_field | +
The enumeration allows specifying which fields in uct_cm_ep_client_connect_args are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_CM_EP_CLIENT_CONNECT_ARGS_FIELD_REMOTE_DATA | + + |
UCT_CM_EP_CLIENT_CONNECT_ARGS_FIELD_STATUS | + + |
enum uct_cm_ep_server_conn_notify_args_field | +
The enumeration allows specifying which fields in uct_cm_ep_server_conn_notify_args are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_CM_EP_SERVER_CONN_NOTIFY_ARGS_FIELD_STATUS |
+ Enables uct_cm_ep_server_conn_notify_args::status Indicates that status field in uct_cm_ep_server_conn_notify_args_t is valid. + |
ucs_status_t uct_iface_accept | +( | +uct_iface_h | +iface, | +
+ | + | uct_conn_request_h | +conn_request | +
+ | ) | ++ |
[in] | iface | Transport interface which generated connection request conn_request. |
[in] | conn_request | Connection establishment request passed as parameter of uct_sockaddr_conn_request_callback_t. |
ucs_status_t uct_iface_reject | +( | +uct_iface_h | +iface, | +
+ | + | uct_conn_request_h | +conn_request | +
+ | ) | ++ |
[in] | iface | Interface which generated connection establishment request conn_request. |
[in] | conn_request | Connection establishment request passed as parameter of uct_sockaddr_conn_request_callback_t. |
ucs_status_t uct_ep_connect | +( | +uct_ep_h | +ep, | +
+ | + | const uct_ep_connect_params_t * | +params | +
+ | ) | ++ |
This non-blocking routine establishes connection of the client side endpoint and sends private data to the peer.
+[in] | ep | Endpoint to connect. |
[in] | params | Parameters as defined in uct_ep_connect_params_t. |
ucs_status_t uct_ep_disconnect | +( | +uct_ep_h | +ep, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
This non-blocking routine will send a disconnect notification on the endpoint, so that uct_ep_disconnect_cb_t will be called on the remote peer. The remote side should also call this routine when handling the initiator's disconnect. After a call to this function, the given endpoint may not be used for communications anymore. The uct_ep_flush / uct_iface_flush routines will guarantee that the disconnect notification is delivered to the remote peer. uct_ep_destroy should be called on this endpoint after invoking this routine and uct_ep_params::disconnect_cb was called.
+[in] | ep | Endpoint to disconnect. |
[in] | flags | Reserved for future use. |
ucs_status_t uct_cm_open | +( | +uct_component_h | +component, | +
+ | + | uct_worker_h | +worker, | +
+ | + | const uct_cm_config_t * | +config, | +
+ | + | uct_cm_h * | +cm_p | +
+ | ) | ++ |
Open a connection manager. All client server connection establishment operations are performed in the context of a specific connection manager.
+[in] | component | Component on which to open the connection manager, as returned from uct_query_components. |
[in] | worker | Worker on which to open the connection manager. |
[in] | config | CM configuration options. Either obtained from uct_cm_config_read() function, or pointer to CM-specific structure that extends uct_cm_config_t. |
[out] | cm_p | Filled with a handle to the connection manager. |
void uct_cm_close | +( | +uct_cm_h | +cm | ) | ++ |
[in] | cm | Connection manager to close. |
ucs_status_t uct_cm_query | +( | +uct_cm_h | +cm, | +
+ | + | uct_cm_attr_t * | +cm_attr | +
+ | ) | ++ |
This routine queries the cm for its attributes uct_cm_attr_t.
+[in] | cm | Connection manager to query. |
[out] | cm_attr | Filled with connection manager attributes. |
ucs_status_t uct_cm_config_read | +( | +uct_component_h | +component, | +
+ | + | const char * | +env_prefix, | +
+ | + | const char * | +filename, | +
+ | + | uct_cm_config_t ** | +config_p | +
+ | ) | ++ |
[in] | component | Read the configuration of the connection manager on this component. |
[in] | env_prefix | If non-NULL, search for environment variables starting with this UCT_<prefix>_. Otherwise, search for environment variables starting with just UCT_. |
[in] | filename | If non-NULL, read configuration from this file. If the file does not exist, or exists but cannot be opened or read, it will be ignored. |
[out] | config_p | Filled with a pointer to the configuration. |
ucs_status_t uct_cm_client_ep_conn_notify | +( | +uct_ep_h | +ep | ) | ++ |
This routine should be called on the client side after the client completed establishing its connection to the server. The routine will send a notification message to the server indicating that the client is connected.
+[in] | ep | The connected endpoint on the client side. |
ucs_status_t uct_listener_create | +( | +uct_cm_h | +cm, | +
+ | + | const struct sockaddr * | +saddr, | +
+ | + | socklen_t | +socklen, | +
+ | + | const uct_listener_params_t * | +params, | +
+ | + | uct_listener_h * | +listener_p | +
+ | ) | ++ |
This routine creates a new listener on the given CM which will start listening on a given sockaddr.
+[in] | cm | Connection manager on which to open the listener. This cm should not be closed as long as there are open listeners on it. |
[in] | saddr | The socket address to listen on. |
[in] | socklen | The saddr length. |
[in] | params | User defined uct_listener_params_t configurations for the listener_p. |
[out] | listener_p | Filled with handle to the new listener. |
void uct_listener_destroy | +( | +uct_listener_h | +listener | ) | ++ |
[in] | listener | Listener to destroy. |
ucs_status_t uct_listener_reject | +( | +uct_listener_h | +listener, | +
+ | + | uct_conn_request_h | +conn_request | +
+ | ) | ++ |
This routine can be invoked on the server side. It rejects a connection request from the client.
+[in] | listener | Listener which will reject the connection request. |
[in] | conn_request | Connection establishment request passed as parameter of uct_cm_listener_conn_request_callback_t in uct_cm_listener_conn_request_args_t::conn_request. |
ucs_status_t uct_listener_query | +( | +uct_listener_h | +listener, | +
+ | + | uct_listener_attr_t * | +listener_attr | +
+ | ) | ++ |
This routine queries the listener for its attributes uct_listener_attr_t.
+[in] | listener | Listener object to query. |
[out] | listener_attr | Filled with attributes of the listener. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Enumerations | |
enum | uct_alloc_method_t { + UCT_ALLOC_METHOD_THP, +UCT_ALLOC_METHOD_MD, +UCT_ALLOC_METHOD_HEAP, +UCT_ALLOC_METHOD_MMAP, + + UCT_ALLOC_METHOD_HUGE, +UCT_ALLOC_METHOD_LAST, +UCT_ALLOC_METHOD_DEFAULT = UCT_ALLOC_METHOD_LAST + + } |
Memory allocation methods. More... | |
+Functions | |
ucs_status_t | uct_worker_create (ucs_async_context_t *async, ucs_thread_mode_t thread_mode, uct_worker_h *worker_p) |
Create a worker object. More... | |
void | uct_worker_destroy (uct_worker_h worker) |
Destroy a worker object. More... | |
void | uct_worker_progress_register_safe (uct_worker_h worker, ucs_callback_t func, void *arg, unsigned flags, uct_worker_cb_id_t *id_p) |
Add a slow path callback function to a worker progress. More... | |
void | uct_worker_progress_unregister_safe (uct_worker_h worker, uct_worker_cb_id_t *id_p) |
Remove a slow path callback function from worker's progress. More... | |
ucs_status_t | uct_config_get (void *config, const char *name, char *value, size_t max) |
Get value by name from interface configuration (uct_iface_config_t), memory domain configuration (uct_md_config_t) or connection manager configuration (uct_cm_config_t). More... | |
ucs_status_t | uct_config_modify (void *config, const char *name, const char *value) |
Modify interface configuration (uct_iface_config_t), memory domain configuration (uct_md_config_t) or connection manager configuration (uct_cm_config_t). More... | |
unsigned | uct_worker_progress (uct_worker_h worker) |
Explicit progress for UCT worker. More... | |
UCT context abstracts all the resources required for network communication. It is designed to enable either share or isolate resources for multiple programming models used by an application.
+This section provides a detailed description of this concept and routines associated with it.
+enum uct_alloc_method_t | +
Enumerator | |
---|---|
UCT_ALLOC_METHOD_THP |
+ Allocate from OS using libc allocator with Transparent Huge Pages enabled + |
UCT_ALLOC_METHOD_MD |
+ Allocate using memory domain + |
UCT_ALLOC_METHOD_HEAP |
+ Allocate from heap using libc allocator + |
UCT_ALLOC_METHOD_MMAP |
+ Allocate from OS using mmap() syscall + |
UCT_ALLOC_METHOD_HUGE |
+ Allocate huge pages + |
UCT_ALLOC_METHOD_LAST | + |
UCT_ALLOC_METHOD_DEFAULT |
+ Use default method + |
ucs_status_t uct_worker_create | +( | +ucs_async_context_t * | +async, | +
+ | + | ucs_thread_mode_t | +thread_mode, | +
+ | + | uct_worker_h * | +worker_p | +
+ | ) | ++ |
The worker represents a progress engine. Multiple progress engines can be created in an application, for example to be used by multiple threads. Transports can allocate separate communication resources for every worker, so that every worker can be progressed independently of others.
+[in] | async | Context for async event handlers. Must not be NULL. |
[in] | thread_mode | Thread access mode to the worker and all interfaces and endpoints associated with it. |
[out] | worker_p | Filled with a pointer to the worker object. |
void uct_worker_destroy | +( | +uct_worker_h | +worker | ) | ++ |
[in] | worker | Worker object to destroy. |
void uct_worker_progress_register_safe | +( | +uct_worker_h | +worker, | +
+ | + | ucs_callback_t | +func, | +
+ | + | void * | +arg, | +
+ | + | unsigned | +flags, | +
+ | + | uct_worker_cb_id_t * | +id_p | +
+ | ) | ++ |
If *id_p is equal to UCS_CALLBACKQ_ID_NULL, this function will add a callback which will be invoked every time progress is made on the worker. *id_p will be updated with an id which refers to this callback and can be used in uct_worker_progress_unregister_safe to remove it from the progress path.
+[in] | worker | Handle to the worker whose progress should invoke the callback. |
[in] | func | Pointer to the callback function. |
[in] | arg | Argument for the callback function. |
[in] | flags | Callback flags, see ucs_callbackq_flags. |
[in,out] | id_p | Points to a location to store a callback identifier. If *id_p is equal to UCS_CALLBACKQ_ID_NULL, a callback will be added and *id_p will be replaced with a callback identifier which can be subsequently used to remove the callback. Otherwise, no callback will be added and *id_p will be left unchanged. |
void uct_worker_progress_unregister_safe | +( | +uct_worker_h | +worker, | +
+ | + | uct_worker_cb_id_t * | +id_p | +
+ | ) | ++ |
If *id_p is not equal to UCS_CALLBACKQ_ID_NULL, remove a callback which was previously added by uct_worker_progress_register_safe. *id_p will be reset to UCS_CALLBACKQ_ID_NULL.
+[in] | worker | Handle to the worker whose progress should invoke the callback. |
[in,out] | id_p | Points to a callback identifier which indicates the callback to remove. If *id_p is not equal to UCS_CALLBACKQ_ID_NULL, the callback will be removed and *id_p will be reset to UCS_CALLBACKQ_ID_NULL. If *id_p is equal to UCS_CALLBACKQ_ID_NULL, no operation will be performed and *id_p will be left unchanged. |
ucs_status_t uct_config_get | +( | +void * | +config, | +
+ | + | const char * | +name, | +
+ | + | char * | +value, | +
+ | + | size_t | +max | +
+ | ) | ++ |
[in] | config | Configuration to get from. |
[in] | name | Configuration variable name. |
[out] | value | Pointer to get value. Should be allocated/freed by caller. |
[in] | max | Available memory space at value pointer. |
ucs_status_t uct_config_modify | +( | +void * | +config, | +
+ | + | const char * | +name, | +
+ | + | const char * | +value | +
+ | ) | ++ |
[in] | config | Configuration to modify. |
[in] | name | Configuration variable name. |
[in] | value | Value to set. |
unsigned uct_worker_progress | +( | +uct_worker_h | +worker | ) | ++ |
This routine explicitly progresses any outstanding communication operations and active message requests.
+[in] | worker | Handle to worker. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | uct_md_attr |
Memory domain attributes. More... | |
struct | uct_md_attr.cap |
struct | uct_md_mem_attr |
Memory domain attributes. More... | |
struct | uct_allocated_memory |
Describes a memory allocated by UCT. More... | |
struct | uct_rkey_bundle |
Remote key with its type. More... | |
struct | uct_mem_alloc_params_t |
Parameters for allocating memory using uct_mem_alloc. More... | |
struct | uct_mem_alloc_params_t.mds |
+Typedefs | |
typedef enum uct_md_mem_attr_field | uct_md_mem_attr_field_t |
UCT MD memory attributes field mask. More... | |
typedef struct uct_md_mem_attr | uct_md_mem_attr_t |
Memory domain attributes. More... | |
typedef struct uct_allocated_memory | uct_allocated_memory_t |
Describes a memory allocated by UCT. More... | |
typedef struct uct_rkey_bundle | uct_rkey_bundle_t |
Remote key with its type. More... | |
+Enumerations | |
enum | uct_sockaddr_accessibility_t { UCT_SOCKADDR_ACC_LOCAL, +UCT_SOCKADDR_ACC_REMOTE + } |
Socket address accessibility type. More... | |
enum | { + UCT_MD_FLAG_ALLOC = UCS_BIT(0), +UCT_MD_FLAG_REG = UCS_BIT(1), +UCT_MD_FLAG_NEED_MEMH = UCS_BIT(2), +UCT_MD_FLAG_NEED_RKEY = UCS_BIT(3), + + UCT_MD_FLAG_ADVISE = UCS_BIT(4), +UCT_MD_FLAG_FIXED = UCS_BIT(5), +UCT_MD_FLAG_RKEY_PTR = UCS_BIT(6), +UCT_MD_FLAG_SOCKADDR = UCS_BIT(7), + + UCT_MD_FLAG_INVALIDATE = UCS_BIT(8), +UCT_MD_FLAG_EXPORTED_MKEY = UCS_BIT(9), +UCT_MD_FLAG_REG_DMABUF = UCS_BIT(10), +UCT_MD_FLAG_LAST = UCS_BIT(11) + + } |
Memory domain capability flags. More... | |
enum | uct_md_mem_flags { + UCT_MD_MEM_FLAG_NONBLOCK = UCS_BIT(0), +UCT_MD_MEM_FLAG_FIXED = UCS_BIT(1), +UCT_MD_MEM_FLAG_LOCK = UCS_BIT(2), +UCT_MD_MEM_FLAG_HIDE_ERRORS = UCS_BIT(3), + + UCT_MD_MEM_ACCESS_REMOTE_PUT = UCS_BIT(5), +UCT_MD_MEM_ACCESS_REMOTE_GET = UCS_BIT(6), +UCT_MD_MEM_ACCESS_REMOTE_ATOMIC = UCS_BIT(7), +UCT_MD_MEM_ACCESS_LOCAL_READ = UCS_BIT(8), + + UCT_MD_MEM_ACCESS_LOCAL_WRITE = UCS_BIT(9), +UCT_MD_MEM_ACCESS_ALL, +UCT_MD_MEM_ACCESS_RMA + + } |
Memory allocation/registration flags. More... | |
enum | uct_mem_advice_t { UCT_MADV_NORMAL = 0, +UCT_MADV_WILLNEED + } |
list of UCT memory use advice More... | |
enum | uct_md_mem_attr_field { + UCT_MD_MEM_ATTR_FIELD_MEM_TYPE = UCS_BIT(0), +UCT_MD_MEM_ATTR_FIELD_SYS_DEV = UCS_BIT(1), +UCT_MD_MEM_ATTR_FIELD_BASE_ADDRESS = UCS_BIT(2), +UCT_MD_MEM_ATTR_FIELD_ALLOC_LENGTH = UCS_BIT(3), + + UCT_MD_MEM_ATTR_FIELD_DMABUF_FD = UCS_BIT(4), +UCT_MD_MEM_ATTR_FIELD_DMABUF_OFFSET = UCS_BIT(5) + + } |
UCT MD memory attributes field mask. More... | |
enum | uct_mem_alloc_params_field_t { + UCT_MEM_ALLOC_PARAM_FIELD_FLAGS = UCS_BIT(0), +UCT_MEM_ALLOC_PARAM_FIELD_ADDRESS = UCS_BIT(1), +UCT_MEM_ALLOC_PARAM_FIELD_MEM_TYPE = UCS_BIT(2), +UCT_MEM_ALLOC_PARAM_FIELD_MDS = UCS_BIT(3), + + UCT_MEM_ALLOC_PARAM_FIELD_NAME = UCS_BIT(4) + + } |
UCT allocation parameters specification field mask. More... | |
+Functions | |
ucs_status_t | uct_md_mem_query (uct_md_h md, const void *address, size_t length, uct_md_mem_attr_t *mem_attr) |
Query attributes of a given pointer. More... | |
ucs_status_t | uct_md_query (uct_md_h md, uct_md_attr_t *md_attr) |
Query for memory domain attributes. More... | |
ucs_status_t | uct_md_mem_advise (uct_md_h md, uct_mem_h memh, void *addr, size_t length, uct_mem_advice_t advice) |
Give advice about the use of memory. More... | |
ucs_status_t | uct_md_mem_reg (uct_md_h md, void *address, size_t length, unsigned flags, uct_mem_h *memh_p) |
Register memory for zero-copy sends and remote access. More... | |
ucs_status_t | uct_md_mem_dereg (uct_md_h md, uct_mem_h memh) |
Undo the operation of uct_md_mem_reg(). More... | |
ucs_status_t | uct_md_detect_memory_type (uct_md_h md, const void *addr, size_t length, ucs_memory_type_t *mem_type_p) |
Detect memory type. More... | |
ucs_status_t | uct_mem_alloc (size_t length, const uct_alloc_method_t *methods, unsigned num_methods, const uct_mem_alloc_params_t *params, uct_allocated_memory_t *mem) |
Allocate memory for zero-copy communications and remote access. More... | |
ucs_status_t | uct_mem_free (const uct_allocated_memory_t *mem) |
Release allocated memory. More... | |
ucs_status_t | uct_md_config_read (uct_component_h component, const char *env_prefix, const char *filename, uct_md_config_t **config_p) |
Read the configuration for a memory domain. More... | |
int | uct_md_is_sockaddr_accessible (uct_md_h md, const ucs_sock_addr_t *sockaddr, uct_sockaddr_accessibility_t mode) |
Check if remote sock address is accessible from the memory domain. More... | |
ucs_status_t | uct_md_mkey_pack (uct_md_h md, uct_mem_h memh, void *rkey_buffer) |
Pack a remote key. More... | |
ucs_status_t | uct_rkey_unpack (uct_component_h component, const void *rkey_buffer, uct_rkey_bundle_t *rkey_ob) |
Unpack a remote key. More... | |
ucs_status_t | uct_rkey_ptr (uct_component_h component, uct_rkey_bundle_t *rkey_ob, uint64_t remote_addr, void **addr_p) |
Get a local pointer to remote memory. More... | |
ucs_status_t | uct_rkey_release (uct_component_h component, const uct_rkey_bundle_t *rkey_ob) |
Release a remote key. More... | |
The Memory Domain abstracts resources required for network communication, which typically includes memory, transport mechanisms, compute and network resources. It is an isolation mechanism that can be employed by the applications for isolating resources between multiple programming models. The attributes of the Memory Domain are defined by the structure uct_md_attr(). The communication and memory operations are defined in the context of Memory Domain.
+struct uct_md_attr | +
This structure defines the attributes of a Memory Domain which includes maximum memory that can be allocated, credentials required for accessing the memory, CPU mask indicating the proximity of CPUs, and bitmaps indicating the types of memory (CPU/CUDA/ROCM) that can be detected, allocated and accessed.
+Data Fields | ||
---|---|---|
+struct uct_md_attr | ++cap | ++ |
+ucs_linear_func_t | ++reg_cost | +
+ Memory registration cost estimation (time,seconds) as a linear function of the buffer size. + |
+char | ++component_name[UCT_COMPONENT_NAME_MAX] | +
+ Component name + |
+size_t | ++rkey_packed_size | +
+ Size of buffer needed for packed rkey + |
+ucs_cpu_set_t | ++local_cpus | +
+ Mask of CPUs near the resource + |
struct uct_md_attr.cap | +
Data Fields | ||
---|---|---|
+uint64_t | ++max_alloc | +
+ Maximal allocation size + |
+size_t | ++max_reg | +
+ Maximal registration size + |
+uint64_t | ++flags | +
+ UCT_MD_FLAG_xx + |
+uint64_t | ++reg_mem_types | +
+ Bitmap of memory types that Memory Domain can be registered with + |
+uint64_t | ++detect_mem_types | +
+ Bitmap of memory types that Memory Domain can detect if address belongs to it + |
+uint64_t | ++alloc_mem_types | +
+ Bitmap of memory types that Memory Domain can allocate memory on + |
+uint64_t | ++access_mem_types | +
+ Memory types that Memory Domain can access + |
struct uct_md_mem_attr | +
This structure defines the attributes of a memory pointer which may include the memory type of the pointer, and the system device that backs the pointer depending on the bit fields populated in field_mask.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_md_mem_attr_field_t. + |
+ucs_memory_type_t | ++mem_type | +
+ The type of memory. E.g. CPU/GPU memory or some other valid type. If the md does not support sys_dev query, then UCS_MEMORY_TYPE_UNKNOWN is returned. + |
+ucs_sys_device_t | ++sys_dev | +
+ Index of the system device on which the buffer resides. eg: NUMA/GPU If the md does not support sys_dev query, then UCS_SYS_DEVICE_ID_UNKNOWN is returned. + |
+void * | ++base_address | +
+ Base address of the allocation to which the provided buffer belongs to. If the md not support base address query, then the pointer passed to uct_md_mem_query is returned as is. + |
+size_t | ++alloc_length | +
+ Length of the whole allocation to which the provided buffer belongs to. If the md not support querying allocation length, then the length passed to uct_md_mem_query is returned as is. + |
+int | ++dmabuf_fd | +
+ Dmabuf file descriptor to expose memory regions across devices. Refer (https://01.org/linuxgraphics/gfx-docs/drm/driver-api/dma-buf.html). If the md does not support querying the fd object associated with the region, then dmabuf_fd is set to UCT_DMABUF_FD_INVALID by uct_md_mem_query(). It is the responsibility of the user to close the returned fd using close (2) when it's no longer needed. + |
+size_t | ++dmabuf_offset | +
+ Offset of the given address from the start of the memory region (identified by dmabuf_fd) backing the memory region being queried. + |
struct uct_allocated_memory | +
This structure describes the memory block which includes the address, size, and Memory Domain used for allocation. This structure is passed to interface and the memory is allocated by memory allocation functions uct_mem_alloc.
+Data Fields | ||
---|---|---|
+void * | ++address | +
+ Address of allocated memory + |
+size_t | ++length | +
+ Real size of allocated memory + |
+uct_alloc_method_t | ++method | +
+ Method used to allocate the memory + |
+ucs_memory_type_t | ++mem_type | +
+ type of allocated memory + |
+uct_md_h | ++md | +
+ if method==MD: MD used to allocate the memory + |
+uct_mem_h | ++memh | +
+ if method==MD: MD memory handle + |
struct uct_rkey_bundle | +
This structure describes the credentials (typically key) and information required to access the remote memory by the communication interfaces.
+Data Fields | ||
---|---|---|
+uct_rkey_t | ++rkey | +
+ Remote key descriptor, passed to RMA functions + |
+void * | ++handle | +
+ Handle, used internally for releasing the key + |
+void * | ++type | +
+ Remote key type + |
struct uct_mem_alloc_params_t | +
Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_mem_alloc_params_field_t. Fields not specified in this mask will be ignored. + |
+unsigned | ++flags | +
+ Memory allocation flags, see uct_md_mem_flags If UCT_MEM_ALLOC_PARAM_FIELD_FLAGS is not specified in field_mask, then (UCT_MD_MEM_ACCESS_LOCAL_READ | UCT_MD_MEM_ACCESS_LOCAL_WRITE) is used by default. + |
+void * | ++address | +
+ If address is NULL, the underlying allocation routine will choose the address at which to create the mapping. If address is non-NULL and UCT_MD_MEM_FLAG_FIXED is not set, the address will be interpreted as a hint as to where to establish the mapping. If address is non-NULL and UCT_MD_MEM_FLAG_FIXED is set, then the specified address is interpreted as a requirement. In this case, if the mapping to the exact address cannot be made, the allocation request fails. + |
+ucs_memory_type_t | ++mem_type | +
+ Type of memory to be allocated. + |
+struct uct_mem_alloc_params_t | ++mds | ++ |
+const char * | ++name | +
+ Name of the allocated region, used to track memory usage for debugging and profiling. If UCT_MEM_ALLOC_PARAM_FIELD_NAME is not specified in field_mask, then "anonymous-uct_mem_alloc" is used by default. + |
struct uct_mem_alloc_params_t.mds | +
Data Fields | ||
---|---|---|
+const uct_md_h * | ++mds | +
+ Array of memory domains to attempt to allocate the memory with, for MD allocation method. + |
+unsigned | ++count | +
+ Length of 'mds' array. May be empty, in such case 'mds' may be NULL, and MD allocation method will be skipped. + |
typedef enum uct_md_mem_attr_field uct_md_mem_attr_field_t | +
The enumeration allows specifying which fields in uct_md_mem_attr_t are present.
+ +typedef struct uct_md_mem_attr uct_md_mem_attr_t | +
This structure defines the attributes of a memory pointer which may include the memory type of the pointer, and the system device that backs the pointer depending on the bit fields populated in field_mask.
+ +typedef struct uct_allocated_memory uct_allocated_memory_t | +
This structure describes the memory block which includes the address, size, and Memory Domain used for allocation. This structure is passed to interface and the memory is allocated by memory allocation functions uct_mem_alloc.
+ +typedef struct uct_rkey_bundle uct_rkey_bundle_t | +
This structure describes the credentials (typically key) and information required to access the remote memory by the communication interfaces.
+ +enum uct_sockaddr_accessibility_t | +
anonymous enum | +
Enumerator | |
---|---|
UCT_MD_FLAG_ALLOC |
+ MD supports memory allocation + |
UCT_MD_FLAG_REG |
+ MD supports memory registration + |
UCT_MD_FLAG_NEED_MEMH |
+ The transport needs a valid local memory handle for zero-copy operations + |
UCT_MD_FLAG_NEED_RKEY |
+ The transport needs a valid remote memory key for remote memory operations + |
UCT_MD_FLAG_ADVISE |
+ MD supports memory advice + |
UCT_MD_FLAG_FIXED |
+ MD supports memory allocation with fixed address + |
UCT_MD_FLAG_RKEY_PTR |
+ MD supports direct access to remote memory via a pointer that is returned by uct_rkey_ptr. +
|
UCT_MD_FLAG_SOCKADDR |
+ MD support for client-server connection establishment via sockaddr + |
UCT_MD_FLAG_INVALIDATE |
+ MD supports memory invalidation. +
|
UCT_MD_FLAG_EXPORTED_MKEY |
+ MD supports exporting memory keys with another process using the same device or attaching to an exported memory key. + |
UCT_MD_FLAG_REG_DMABUF |
+ MD supports registering a dmabuf file descriptor. + |
UCT_MD_FLAG_LAST |
+ The enum must not be extended. Any additional flags must be defined in API v2 uct_md_flags_v2_t. + |
enum uct_md_mem_flags | +
Enumerator | |
---|---|
UCT_MD_MEM_FLAG_NONBLOCK |
+ Hint to perform non-blocking allocation/registration: page mapping may be deferred until it is accessed by the CPU or a transport. + |
UCT_MD_MEM_FLAG_FIXED |
+ Place the mapping at exactly defined address. + |
UCT_MD_MEM_FLAG_LOCK |
+ Registered memory should be locked. May incur extra cost for registration, but memory access is usually faster. + |
UCT_MD_MEM_FLAG_HIDE_ERRORS |
+ Hide errors on memory registration. In some cases registration failure is not an error (e. g. for merged memory regions). + |
UCT_MD_MEM_ACCESS_REMOTE_PUT |
+ Enable remote put access. + |
UCT_MD_MEM_ACCESS_REMOTE_GET |
+ Enable remote get access. + |
UCT_MD_MEM_ACCESS_REMOTE_ATOMIC |
+ Enable remote atomic access. + |
UCT_MD_MEM_ACCESS_LOCAL_READ |
+ Enable local read access. + |
UCT_MD_MEM_ACCESS_LOCAL_WRITE |
+ Enable local write access. + |
UCT_MD_MEM_ACCESS_ALL |
+ Enable local and remote access for all operations. + |
UCT_MD_MEM_ACCESS_RMA |
+ Enable local and remote access for put and get operations. + |
enum uct_mem_advice_t | +
Enumerator | |
---|---|
UCT_MADV_NORMAL |
+ No special treatment + |
UCT_MADV_WILLNEED |
+ can be used on the memory mapped with UCT_MD_MEM_FLAG_NONBLOCK to speed up memory mapping and to avoid page faults when the memory is accessed for the first time. + |
enum uct_md_mem_attr_field | +
The enumeration allows specifying which fields in uct_md_mem_attr_t are present.
+Enumerator | |
---|---|
UCT_MD_MEM_ATTR_FIELD_MEM_TYPE |
+ Indicate if memory type is populated. E.g. CPU/GPU + |
UCT_MD_MEM_ATTR_FIELD_SYS_DEV |
+ Indicate if details of system device backing the pointer are populated. For example: GPU device, NUMA domain, etc. + |
UCT_MD_MEM_ATTR_FIELD_BASE_ADDRESS |
+ Request base address of the allocation to which the buffer belongs. + |
UCT_MD_MEM_ATTR_FIELD_ALLOC_LENGTH |
+ Request the whole length of the allocation to which the buffer belongs. + |
UCT_MD_MEM_ATTR_FIELD_DMABUF_FD |
+ Request a cross-device dmabuf file descriptor that represents a memory region, and can be used to register the region with another memory domain. + |
UCT_MD_MEM_ATTR_FIELD_DMABUF_OFFSET |
+ Request the offset of the provided virtual address relative to the beginning of its backing dmabuf region. + |
enum uct_mem_alloc_params_field_t | +
The enumeration allows specifying which fields in uct_mem_alloc_params_t are present.
+Enumerator | |
---|---|
UCT_MEM_ALLOC_PARAM_FIELD_FLAGS |
+ Enables uct_mem_alloc_params_t::flags + |
UCT_MEM_ALLOC_PARAM_FIELD_ADDRESS |
+ Enables uct_mem_alloc_params_t::address + |
UCT_MEM_ALLOC_PARAM_FIELD_MEM_TYPE |
+ Enables uct_mem_alloc_params_t::mem_type + |
UCT_MEM_ALLOC_PARAM_FIELD_MDS |
+ Enables uct_mem_alloc_params_t::mds + |
UCT_MEM_ALLOC_PARAM_FIELD_NAME |
+ Enables uct_mem_alloc_params_t::name + |
ucs_status_t uct_md_mem_query | +( | +uct_md_h | +md, | +
+ | + | const void * | +address, | +
+ | + | size_t | +length, | +
+ | + | uct_md_mem_attr_t * | +mem_attr | +
+ | ) | ++ |
Return attributes such as memory type, base address, allocation length, and system device for the given pointer of specific length.
+[in] | md | Memory domain to run the query on. This function returns an error if the md does not recognize the pointer. |
[in] | address | The address of the pointer. Must be non-NULL else UCS_ERR_INVALID_PARAM error is returned. |
[in] | length | Length of the memory region to examine. Must be nonzero else UCS_ERR_INVALID_PARAM error is returned. |
[in,out] | mem_attr | If successful, filled with ptr attributes. |
ucs_status_t uct_md_query | +( | +uct_md_h | +md, | +
+ | + | uct_md_attr_t * | +md_attr | +
+ | ) | ++ |
[in] | md | Memory domain to query. |
[out] | md_attr | Filled with memory domain attributes. |
ucs_status_t uct_md_mem_advise | +( | +uct_md_h | +md, | +
+ | + | uct_mem_h | +memh, | +
+ | + | void * | +addr, | +
+ | + | size_t | +length, | +
+ | + | uct_mem_advice_t | +advice | +
+ | ) | ++ |
This routine advises the UCT about how to handle memory range beginning at address and size of length bytes. This call does not influence the semantics of the application, but may influence its performance. The advice may be ignored.
+[in] | md | Memory domain memory was allocated or registered on. |
[in] | memh | Memory handle, as returned from uct_mem_alloc |
[in] | addr | Memory base address. Memory range must belong to the memh |
[in] | length | Length of memory to advise. Must be >0. |
[in] | advice | Memory use advice as defined in the uct_mem_advice_t list |
ucs_status_t uct_md_mem_reg | +( | +uct_md_h | +md, | +
+ | + | void * | +address, | +
+ | + | size_t | +length, | +
+ | + | unsigned | +flags, | +
+ | + | uct_mem_h * | +memh_p | +
+ | ) | ++ |
Register memory on the memory domain. In order to use this function, MD must support UCT_MD_FLAG_REG flag.
+[in] | md | Memory domain to register memory on. |
[in] | address | Memory to register. |
[in] | length | Size of memory to register. Must be >0. |
[in] | flags | Memory allocation flags, see uct_md_mem_flags. |
[out] | memh_p | Filled with handle for allocated region. |
ucs_status_t uct_md_mem_dereg | +( | +uct_md_h | +md, | +
+ | + | uct_mem_h | +memh | +
+ | ) | ++ |
[in] | md | Memory domain which was used to register the memory. |
[in] | memh | Local access key to memory region. |
ucs_status_t uct_md_detect_memory_type | +( | +uct_md_h | +md, | +
+ | + | const void * | +addr, | +
+ | + | size_t | +length, | +
+ | + | ucs_memory_type_t * | +mem_type_p | +
+ | ) | ++ |
[in] | md | Memory domain to detect memory type |
[in] | addr | Memory address to detect. |
[in] | length | Size of memory |
[out] | mem_type_p | Filled with memory type of the address range if function succeeds |
ucs_status_t uct_mem_alloc | +( | +size_t | +length, | +
+ | + | const uct_alloc_method_t * | +methods, | +
+ | + | unsigned | +num_methods, | +
+ | + | const uct_mem_alloc_params_t * | +params, | +
+ | + | uct_allocated_memory_t * | +mem | +
+ | ) | ++ |
Allocate potentially registered memory.
+[in] | length | The minimal size to allocate. The actual size may be larger, for example because of alignment restrictions. Must be >0. |
[in] | methods | Array of memory allocation methods to attempt. Each of the provided allocation methods will be tried in array order, to perform the allocation, until one succeeds. Whenever the MD method is encountered, each of the provided MDs will be tried in array order, to allocate the memory, until one succeeds, or they are exhausted. In this case the next allocation method from the initial list will be attempted. |
[in] | num_methods | Length of 'methods' array. |
[in] | params | Memory allocation characteristics, see uct_mem_alloc_params_t. |
[out] | mem | In case of success, filled with information about the allocated memory. uct_allocated_memory_t |
ucs_status_t uct_mem_free | +( | +const uct_allocated_memory_t * | +mem | ) | ++ |
Release the memory allocated by uct_mem_alloc.
+[in] | mem | Description of allocated memory, as returned from uct_mem_alloc. |
ucs_status_t uct_md_config_read | +( | +uct_component_h | +component, | +
+ | + | const char * | +env_prefix, | +
+ | + | const char * | +filename, | +
+ | + | uct_md_config_t ** | +config_p | +
+ | ) | ++ |
[in] | component | Read the configuration of this component. |
[in] | env_prefix | If non-NULL, search for environment variables starting with this UCT_<prefix>_. Otherwise, search for environment variables starting with just UCT_. |
[in] | filename | If non-NULL, read configuration from this file. If the file does not exist, it will be ignored. |
[out] | config_p | Filled with a pointer to the configuration. |
int uct_md_is_sockaddr_accessible | +( | +uct_md_h | +md, | +
+ | + | const ucs_sock_addr_t * | +sockaddr, | +
+ | + | uct_sockaddr_accessibility_t | +mode | +
+ | ) | ++ |
This function checks if a remote sock address can be accessed from a local memory domain. Accessibility can be checked in local or remote mode.
+[in] | md | Memory domain to check accessibility from. This memory domain must support the UCT_MD_FLAG_SOCKADDR flag. |
[in] | sockaddr | Socket address to check accessibility to. |
[in] | mode | Mode for checking accessibility, as defined in uct_sockaddr_accessibility_t. Indicates if accessibility is tested on the server side - for binding to the given sockaddr, or on the client side - for connecting to the given remote peer's sockaddr. |
ucs_status_t uct_md_mkey_pack | +( | +uct_md_h | +md, | +
+ | + | uct_mem_h | +memh, | +
+ | + | void * | +rkey_buffer | +
+ | ) | ++ |
[in] | md | Handle to memory domain. |
[in] | memh | Local key, whose remote key should be packed. |
[out] | rkey_buffer | Filled with packed remote key. |
ucs_status_t uct_rkey_unpack | +( | +uct_component_h | +component, | +
+ | + | const void * | +rkey_buffer, | +
+ | + | uct_rkey_bundle_t * | +rkey_ob | +
+ | ) | ++ |
[in] | component | Component on which to unpack the remote key. |
[in] | rkey_buffer | Packed remote key buffer. |
[out] | rkey_ob | Filled with the unpacked remote key and its type. |
ucs_status_t uct_rkey_ptr | +( | +uct_component_h | +component, | +
+ | + | uct_rkey_bundle_t * | +rkey_ob, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | void ** | +addr_p | +
+ | ) | ++ |
This routine returns a local pointer to the remote memory described by the rkey bundle. The component must support UCT_COMPONENT_FLAG_RKEY_PTR flag.
+[in] | component | Component on which to obtain the pointer to the remote key. |
[in] | rkey_ob | A remote key bundle as returned by the uct_rkey_unpack function. |
[in] | remote_addr | A remote address within the memory area described by the rkey_ob. |
[out] | addr_p | A pointer that can be used for direct access to the remote memory. |
ucs_status_t uct_rkey_release | +( | +uct_component_h | +component, | +
+ | + | const uct_rkey_bundle_t * | +rkey_ob | +
+ | ) | ++ |
[in] | component | Component which was used to unpack the remote key. |
[in] | rkey_ob | Remote key to release. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Modules | |
UCT interface operations and capabilities | |
List of capabilities supported by UCX API. | |
UCT interface for asynchronous event capabilities | |
List of capabilities supported by UCT iface event API. | |
+Data Structures | |
struct | uct_md_resource_desc |
Memory domain resource descriptor. More... | |
struct | uct_component_attr |
UCT component attributes. More... | |
struct | uct_tl_resource_desc |
Communication resource descriptor. More... | |
struct | uct_iface_attr |
Interface attributes: capabilities and limitations. More... | |
struct | uct_iface_attr.cap |
struct | uct_iface_attr.cap.put |
struct | uct_iface_attr.cap.get |
struct | uct_iface_attr.cap.am |
struct | uct_iface_attr.cap.tag |
struct | uct_iface_attr.cap.tag.recv |
struct | uct_iface_attr.cap.tag.eager |
struct | uct_iface_attr.cap.tag.rndv |
struct | uct_iface_attr.cap.atomic32 |
struct | uct_iface_attr.cap.atomic64 |
struct | uct_iface_params |
Parameters used for interface creation. More... | |
union | uct_iface_params.mode |
struct | uct_iface_params.mode.device |
struct | uct_iface_params.mode.sockaddr |
struct | uct_ep_params |
Parameters for creating a UCT endpoint by uct_ep_create. More... | |
struct | uct_completion |
Completion handle. More... | |
struct | uct_pending_req |
Pending request. More... | |
struct | uct_iov |
Structure for scatter-gather I/O. More... | |
+Typedefs | |
typedef struct uct_md_resource_desc | uct_md_resource_desc_t |
Memory domain resource descriptor. More... | |
typedef struct uct_component_attr | uct_component_attr_t |
UCT component attributes. More... | |
typedef struct uct_tl_resource_desc | uct_tl_resource_desc_t |
Communication resource descriptor. More... | |
typedef struct uct_component * | uct_component_h |
typedef struct uct_iface * | uct_iface_h |
typedef struct uct_iface_config | uct_iface_config_t |
typedef struct uct_md_config | uct_md_config_t |
typedef struct uct_cm_config | uct_cm_config_t |
typedef struct uct_ep * | uct_ep_h |
typedef void * | uct_mem_h |
typedef uintptr_t | uct_rkey_t |
typedef struct uct_md * | uct_md_h |
Memory domain handler. More... | |
typedef struct uct_md_ops | uct_md_ops_t |
typedef void * | uct_rkey_ctx_h |
typedef struct uct_iface_attr | uct_iface_attr_t |
typedef struct uct_iface_params | uct_iface_params_t |
typedef struct uct_ep_attr | uct_ep_attr_t |
typedef struct uct_md_attr | uct_md_attr_t |
typedef struct uct_completion | uct_completion_t |
typedef struct uct_pending_req | uct_pending_req_t |
typedef struct uct_worker * | uct_worker_h |
typedef struct uct_md | uct_md_t |
typedef enum uct_am_trace_type | uct_am_trace_type_t |
typedef struct uct_device_addr | uct_device_addr_t |
typedef struct uct_iface_addr | uct_iface_addr_t |
typedef struct uct_ep_addr | uct_ep_addr_t |
typedef struct uct_ep_params | uct_ep_params_t |
typedef struct +uct_ep_connect_params | uct_ep_connect_params_t |
typedef struct uct_cm_attr | uct_cm_attr_t |
typedef struct uct_cm | uct_cm_t |
typedef uct_cm_t * | uct_cm_h |
typedef struct uct_listener_attr | uct_listener_attr_t |
typedef struct uct_listener * | uct_listener_h |
typedef struct uct_listener_params | uct_listener_params_t |
typedef struct uct_tag_context | uct_tag_context_t |
typedef uint64_t | uct_tag_t |
typedef int | uct_worker_cb_id_t |
typedef void * | uct_conn_request_h |
typedef struct uct_iov | uct_iov_t |
Structure for scatter-gather I/O. More... | |
typedef void(* | uct_completion_callback_t )(uct_completion_t *self) |
Callback to process send completion. More... | |
typedef ucs_status_t(* | uct_pending_callback_t )(uct_pending_req_t *self) |
Callback to process pending requests. More... | |
typedef ucs_status_t(* | uct_error_handler_t )(void *arg, uct_ep_h ep, ucs_status_t status) |
Callback to process peer failure. More... | |
typedef void(* | uct_pending_purge_callback_t )(uct_pending_req_t *self, void *arg) |
Callback to purge pending requests. More... | |
typedef size_t(* | uct_pack_callback_t )(void *dest, void *arg) |
Callback for producing data. More... | |
typedef void(* | uct_unpack_callback_t )(void *arg, const void *data, size_t length) |
Callback for consuming data. More... | |
typedef void(* | uct_async_event_cb_t )(void *arg, unsigned flags) |
Callback to process asynchronous events. More... | |
+Enumerations | |
enum | uct_component_attr_field { UCT_COMPONENT_ATTR_FIELD_NAME = UCS_BIT(0), +UCT_COMPONENT_ATTR_FIELD_MD_RESOURCE_COUNT = UCS_BIT(1), +UCT_COMPONENT_ATTR_FIELD_MD_RESOURCES = UCS_BIT(2), +UCT_COMPONENT_ATTR_FIELD_FLAGS = UCS_BIT(3) + } |
UCT component attributes field mask. More... | |
enum | { UCT_COMPONENT_FLAG_CM = UCS_BIT(0), +UCT_COMPONENT_FLAG_RKEY_PTR = UCS_BIT(1) + } |
Capability flags of uct_component_h. More... | |
enum | uct_device_type_t { + UCT_DEVICE_TYPE_NET, +UCT_DEVICE_TYPE_SHM, +UCT_DEVICE_TYPE_ACC, +UCT_DEVICE_TYPE_SELF, + + UCT_DEVICE_TYPE_LAST + + } |
List of UCX device types. More... | |
enum | uct_iface_event_types { UCT_EVENT_SEND_COMP = UCS_BIT(0), +UCT_EVENT_RECV = UCS_BIT(1), +UCT_EVENT_RECV_SIG = UCS_BIT(2) + } |
Asynchronous event types. More... | |
enum | uct_flush_flags { UCT_FLUSH_FLAG_LOCAL = 0, +UCT_FLUSH_FLAG_CANCEL = UCS_BIT(0), +UCT_FLUSH_FLAG_REMOTE = UCS_BIT(1) + } |
Flush modifiers. More... | |
enum | uct_progress_types { UCT_PROGRESS_SEND = UCS_BIT(0), +UCT_PROGRESS_RECV = UCS_BIT(1), +UCT_PROGRESS_THREAD_SAFE = UCS_BIT(7) + } |
UCT progress types. More... | |
enum | uct_cb_flags { UCT_CB_FLAG_RESERVED = UCS_BIT(1), +UCT_CB_FLAG_ASYNC = UCS_BIT(2) + } |
Callback flags. More... | |
enum | uct_iface_open_mode { UCT_IFACE_OPEN_MODE_DEVICE = UCS_BIT(0), +UCT_IFACE_OPEN_MODE_SOCKADDR_SERVER = UCS_BIT(1), +UCT_IFACE_OPEN_MODE_SOCKADDR_CLIENT = UCS_BIT(2) + } |
Mode in which to open the interface. More... | |
enum | uct_iface_params_field { + UCT_IFACE_PARAM_FIELD_CPU_MASK = UCS_BIT(0), +UCT_IFACE_PARAM_FIELD_OPEN_MODE = UCS_BIT(1), +UCT_IFACE_PARAM_FIELD_DEVICE = UCS_BIT(2), +UCT_IFACE_PARAM_FIELD_SOCKADDR = UCS_BIT(3), + + UCT_IFACE_PARAM_FIELD_STATS_ROOT = UCS_BIT(4), +UCT_IFACE_PARAM_FIELD_RX_HEADROOM = UCS_BIT(5), +UCT_IFACE_PARAM_FIELD_ERR_HANDLER_ARG = UCS_BIT(6), +UCT_IFACE_PARAM_FIELD_ERR_HANDLER = UCS_BIT(7), + + UCT_IFACE_PARAM_FIELD_ERR_HANDLER_FLAGS = UCS_BIT(8), +UCT_IFACE_PARAM_FIELD_HW_TM_EAGER_ARG = UCS_BIT(9), +UCT_IFACE_PARAM_FIELD_HW_TM_EAGER_CB = UCS_BIT(10), +UCT_IFACE_PARAM_FIELD_HW_TM_RNDV_ARG = UCS_BIT(11), + + UCT_IFACE_PARAM_FIELD_HW_TM_RNDV_CB = UCS_BIT(12), +UCT_IFACE_PARAM_FIELD_ASYNC_EVENT_ARG = UCS_BIT(13), +UCT_IFACE_PARAM_FIELD_ASYNC_EVENT_CB = UCS_BIT(14), +UCT_IFACE_PARAM_FIELD_KEEPALIVE_INTERVAL = UCS_BIT(15), + + UCT_IFACE_PARAM_FIELD_AM_ALIGNMENT = UCS_BIT(16), +UCT_IFACE_PARAM_FIELD_AM_ALIGN_OFFSET = UCS_BIT(17), +UCT_IFACE_PARAM_FIELD_FEATURES = UCS_BIT(18) + + } |
UCT interface created by uct_iface_open parameters field mask. More... | |
enum | uct_ep_params_field { + UCT_EP_PARAM_FIELD_IFACE = UCS_BIT(0), +UCT_EP_PARAM_FIELD_USER_DATA = UCS_BIT(1), +UCT_EP_PARAM_FIELD_DEV_ADDR = UCS_BIT(2), +UCT_EP_PARAM_FIELD_IFACE_ADDR = UCS_BIT(3), + + UCT_EP_PARAM_FIELD_SOCKADDR = UCS_BIT(4), +UCT_EP_PARAM_FIELD_SOCKADDR_CB_FLAGS = UCS_BIT(5), +UCT_EP_PARAM_FIELD_SOCKADDR_PACK_CB = UCS_BIT(6), +UCT_EP_PARAM_FIELD_CM = UCS_BIT(7), + + UCT_EP_PARAM_FIELD_CONN_REQUEST = UCS_BIT(8), +UCT_EP_PARAM_FIELD_SOCKADDR_CONNECT_CB_CLIENT = UCS_BIT(9), +UCT_EP_PARAM_FIELD_SOCKADDR_NOTIFY_CB_SERVER = UCS_BIT(10), +UCT_EP_PARAM_FIELD_SOCKADDR_DISCONNECT_CB = UCS_BIT(11), + + UCT_EP_PARAM_FIELD_PATH_INDEX = UCS_BIT(12), +UCT_EP_PARAM_FIELD_CM_RESOLVE_CB = UCS_BIT(13), +UCT_EP_PARAM_FIELD_PRIV_DATA = UCS_BIT(14), +UCT_EP_PARAM_FIELD_PRIV_DATA_LENGTH = UCS_BIT(15), + + UCT_EP_PARAM_FIELD_LOCAL_SOCKADDR = UCS_BIT(16) + + } |
UCT endpoint created by uct_ep_create parameters field mask. More... | |
enum | uct_iface_feature { + UCT_IFACE_FEATURE_AM = UCS_BIT(0), +UCT_IFACE_FEATURE_PUT = UCS_BIT(1), +UCT_IFACE_FEATURE_GET = UCS_BIT(2), +UCT_IFACE_FEATURE_AMO32 = UCS_BIT(3), + + UCT_IFACE_FEATURE_AMO64 = UCS_BIT(4), +UCT_IFACE_FEATURE_TAG = UCS_BIT(5), +UCT_IFACE_FEATURE_FLUSH_REMOTE = UCS_BIT(6), +UCT_IFACE_FEATURE_LAST = UCS_BIT(7) + + } |
UCT interface configuration features. More... | |
enum | { UCT_TAG_RECV_CB_INLINE_DATA = UCS_BIT(0) + } |
flags of uct_tag_context. More... | |
enum | uct_cb_param_flags { UCT_CB_PARAM_FLAG_DESC = UCS_BIT(0), +UCT_CB_PARAM_FLAG_FIRST = UCS_BIT(1), +UCT_CB_PARAM_FLAG_MORE = UCS_BIT(2) + } |
Flags for active message and tag-matching offload callbacks (callback's parameters). More... | |
+Functions | |
ucs_status_t | uct_query_components (uct_component_h **components_p, unsigned *num_components_p) |
Query for list of components. More... | |
void | uct_release_component_list (uct_component_h *components) |
Release the list of components returned from uct_query_components. More... | |
ucs_status_t | uct_component_query (uct_component_h component, uct_component_attr_t *component_attr) |
Get component attributes. More... | |
ucs_status_t | uct_md_open (uct_component_h component, const char *md_name, const uct_md_config_t *config, uct_md_h *md_p) |
Open a memory domain. More... | |
void | uct_md_close (uct_md_h md) |
Close a memory domain. More... | |
ucs_status_t | uct_md_query_tl_resources (uct_md_h md, uct_tl_resource_desc_t **resources_p, unsigned *num_resources_p) |
Query for transport resources. More... | |
void | uct_release_tl_resource_list (uct_tl_resource_desc_t *resources) |
Release the list of resources returned from uct_md_query_tl_resources. More... | |
ucs_status_t | uct_md_iface_config_read (uct_md_h md, const char *tl_name, const char *env_prefix, const char *filename, uct_iface_config_t **config_p) |
Read transport-specific interface configuration. More... | |
void | uct_config_release (void *config) |
Release configuration memory returned from uct_md_iface_config_read(), uct_md_config_read(), or from uct_cm_config_read(). More... | |
ucs_status_t | uct_iface_open (uct_md_h md, uct_worker_h worker, const uct_iface_params_t *params, const uct_iface_config_t *config, uct_iface_h *iface_p) |
Open a communication interface. More... | |
void | uct_iface_close (uct_iface_h iface) |
Close and destroy an interface. More... | |
ucs_status_t | uct_iface_query (uct_iface_h iface, uct_iface_attr_t *iface_attr) |
Get interface attributes. More... | |
ucs_status_t | uct_iface_get_device_address (uct_iface_h iface, uct_device_addr_t *addr) |
Get address of the device the interface is using. More... | |
ucs_status_t | uct_iface_get_address (uct_iface_h iface, uct_iface_addr_t *addr) |
Get interface address. More... | |
int | uct_iface_is_reachable (const uct_iface_h iface, const uct_device_addr_t *dev_addr, const uct_iface_addr_t *iface_addr) |
Check if remote iface address is reachable. More... | |
ucs_status_t | uct_ep_check (const uct_ep_h ep, unsigned flags, uct_completion_t *comp) |
check if the destination endpoint is alive in respect to UCT library More... | |
ucs_status_t | uct_iface_event_fd_get (uct_iface_h iface, int *fd_p) |
Obtain a notification file descriptor for polling. More... | |
ucs_status_t | uct_iface_event_arm (uct_iface_h iface, unsigned events) |
Turn on event notification for the next event. More... | |
ucs_status_t | uct_iface_mem_alloc (uct_iface_h iface, size_t length, unsigned flags, const char *name, uct_allocated_memory_t *mem) |
Allocate memory which can be used for zero-copy communications. More... | |
void | uct_iface_mem_free (const uct_allocated_memory_t *mem) |
Release memory allocated with uct_iface_mem_alloc(). More... | |
ucs_status_t | uct_ep_create (const uct_ep_params_t *params, uct_ep_h *ep_p) |
Create new endpoint. More... | |
void | uct_ep_destroy (uct_ep_h ep) |
Destroy an endpoint. More... | |
ucs_status_t | uct_ep_get_address (uct_ep_h ep, uct_ep_addr_t *addr) |
Get endpoint address. More... | |
ucs_status_t | uct_ep_connect_to_ep (uct_ep_h ep, const uct_device_addr_t *dev_addr, const uct_ep_addr_t *ep_addr) |
Connect endpoint to a remote endpoint. More... | |
ucs_status_t | uct_iface_flush (uct_iface_h iface, unsigned flags, uct_completion_t *comp) |
Flush outstanding communication operations on an interface. More... | |
ucs_status_t | uct_iface_fence (uct_iface_h iface, unsigned flags) |
Ensures ordering of outstanding communications on the interface. Operations issued on the interface prior to this call are guaranteed to be completed before any subsequent communication operations to the same interface which follow the call to fence. More... | |
ucs_status_t | uct_ep_pending_add (uct_ep_h ep, uct_pending_req_t *req, unsigned flags) |
Add a pending request to an endpoint. More... | |
void | uct_ep_pending_purge (uct_ep_h ep, uct_pending_purge_callback_t cb, void *arg) |
Remove all pending requests from an endpoint. More... | |
ucs_status_t | uct_ep_flush (uct_ep_h ep, unsigned flags, uct_completion_t *comp) |
Flush outstanding communication operations on an endpoint. More... | |
ucs_status_t | uct_ep_fence (uct_ep_h ep, unsigned flags) |
Ensures ordering of outstanding communications on the endpoint. Operations issued on the endpoint prior to this call are guaranteed to be completed before any subsequent communication operations to the same endpoint which follow the call to fence. More... | |
void | uct_iface_progress_enable (uct_iface_h iface, unsigned flags) |
Enable synchronous progress for the interface. More... | |
void | uct_iface_progress_disable (uct_iface_h iface, unsigned flags) |
Disable synchronous progress for the interface. More... | |
unsigned | uct_iface_progress (uct_iface_h iface) |
Perform a progress on an interface. More... | |
static UCS_F_ALWAYS_INLINE void | uct_completion_update_status (uct_completion_t *comp, ucs_status_t status) |
Update status of UCT completion handle. More... | |
This section describes a concept of the Communication Resource and routines associated with the concept.
+struct uct_md_resource_desc | +
struct uct_component_attr | +
This structure defines the attributes for UCT component. It is used for uct_component_query
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_component_attr_field. Fields not specified in this mask will be ignored. Provides ABI compatibility with respect to adding new fields. + |
+char | ++name[UCT_COMPONENT_NAME_MAX] | +
+ Component name + |
+unsigned | ++md_resource_count | +
+ Number of memory-domain resources + |
+uct_md_resource_desc_t * | ++md_resources | +
+ Array of memory domain resources. When used, it should be initialized prior to calling uct_component_query with a pointer to an array, which is large enough to hold all memory domain resource entries. After the call, this array will be filled with information about existing memory domain resources. In order to allocate this array, you can call uct_component_query twice: The first time would only obtain the amount of entries required, by specifying UCT_COMPONENT_ATTR_FIELD_MD_RESOURCE_COUNT in field_mask. Then the array could be allocated with the returned number of entries, and passed to a second call to uct_component_query, this time setting field_mask to UCT_COMPONENT_ATTR_FIELD_MD_RESOURCES. + |
+uint64_t | ++flags | +
+ Flags as defined by UCT_COMPONENT_FLAG_xx. + |
struct uct_tl_resource_desc | +
Resource descriptor is an object representing the network resource. Resource descriptor could represent a stand-alone communication resource such as an HCA port, network interface, or multiple resources such as multiple network interfaces or communication ports. It could also represent virtual communication resources that are defined over a single physical network interface.
+Data Fields | ||
---|---|---|
+char | ++tl_name[UCT_TL_NAME_MAX] | +
+ Transport name + |
+char | ++dev_name[UCT_DEVICE_NAME_MAX] | +
+ Hardware device name + |
+uct_device_type_t | ++dev_type | +
+ The device represented by this resource (e.g. UCT_DEVICE_TYPE_NET for a network interface) + |
+ucs_sys_device_t | ++sys_device | +
+ The identifier associated with the device bus_id as captured in ucs_sys_bus_id_t struct + |
struct uct_iface_attr | +
Data Fields | ||
---|---|---|
+struct uct_iface_attr | ++cap | +
+ Interface capabilities + |
+size_t | ++device_addr_len | +
+ Size of device address + |
+size_t | ++iface_addr_len | +
+ Size of interface address + |
+size_t | ++ep_addr_len | +
+ Size of endpoint address + |
+size_t | ++max_conn_priv | +
+ Max size of the iface's private data. used for connection establishment with sockaddr + |
+struct sockaddr_storage | ++listen_sockaddr | +
+ Sockaddr on which this iface is listening. + |
+double | ++overhead | +
+ Message overhead, seconds + |
+uct_ppn_bandwidth_t | ++bandwidth | +
+ Bandwidth model + |
+ucs_linear_func_t | ++latency | +
+ Latency as function of number of active endpoints + |
+uint8_t | ++priority | +
+ Priority of device + |
+size_t | ++max_num_eps | +
+ Maximum number of endpoints + |
+unsigned | ++dev_num_paths | +
+ How many network paths can be utilized on the device used by this interface for optimal performance. Endpoints that connect to the same remote address but use different paths can potentially achieve higher total bandwidth compared to using only a single endpoint. + |
struct uct_iface_attr.cap | +
Data Fields | ||
---|---|---|
+cap | ++put | +
+ Attributes for PUT operations + |
+cap | ++get | +
+ Attributes for GET operations + |
+cap | ++am | +
+ Attributes for AM operations + |
+cap | ++tag | +
+ Attributes for TAG operations + |
+cap | ++atomic32 | ++ |
+cap | ++atomic64 | +
+ Attributes for atomic operations + |
+uint64_t | ++flags | +
+ Flags from UCT interface operations and capabilities + |
+uint64_t | ++event_flags | +
+ Flags from UCT interface for asynchronous event capabilities + |
struct uct_iface_attr.cap.put | +
Data Fields | ||
---|---|---|
+size_t | ++max_short | +
+ Maximal size for put_short + |
+size_t | ++max_bcopy | +
+ Maximal size for put_bcopy + |
+size_t | ++min_zcopy | +
+ Minimal size for put_zcopy (total of uct_iov_t::length of the iov parameter) + |
+size_t | ++max_zcopy | +
+ Maximal size for put_zcopy (total of uct_iov_t::length of the iov parameter) + |
+size_t | ++opt_zcopy_align | +
+ Optimal alignment for zero-copy buffer address + |
+size_t | ++align_mtu | +
+ MTU used for alignment + |
+size_t | ++max_iov | +
+ Maximal iovcnt parameter in uct_ep_put_zcopy + |
struct uct_iface_attr.cap.get | +
Data Fields | ||
---|---|---|
+size_t | ++max_short | +
+ Maximal size for get_short + |
+size_t | ++max_bcopy | +
+ Maximal size for get_bcopy + |
+size_t | ++min_zcopy | +
+ Minimal size for get_zcopy (total of uct_iov_t::length of the iov parameter) + |
+size_t | ++max_zcopy | +
+ Maximal size for get_zcopy (total of uct_iov_t::length of the iov parameter) + |
+size_t | ++opt_zcopy_align | +
+ Optimal alignment for zero-copy buffer address + |
+size_t | ++align_mtu | +
+ MTU used for alignment + |
+size_t | ++max_iov | +
+ Maximal iovcnt parameter in uct_ep_get_zcopy + |
struct uct_iface_attr.cap.am | +
Data Fields | ||
---|---|---|
+size_t | ++max_short | ++ + |
+size_t | ++max_bcopy | +
+ Total maximum size (incl. the header) + |
+size_t | ++min_zcopy | +
+ Minimal size for am_zcopy (incl. the header and total of uct_iov_t::length of the iov parameter) + |
+size_t | ++max_zcopy | +
+ Total max. size (incl. the header and total of uct_iov_t::length of the iov parameter) + |
+size_t | ++opt_zcopy_align | +
+ Optimal alignment for zero-copy buffer address + |
+size_t | ++align_mtu | +
+ MTU used for alignment + |
+size_t | ++max_hdr | +
+ Max. header size for zcopy + |
+size_t | ++max_iov | +
+ Maximal iovcnt parameter in uct_ep_am_zcopy + |
struct uct_iface_attr.cap.tag | +
struct uct_iface_attr.cap.tag.recv | +
Data Fields | ||
---|---|---|
+size_t | ++min_recv | +
+ Minimal allowed length of posted receive buffer + |
+size_t | ++max_zcopy | +
+ Maximal allowed data length in uct_iface_tag_recv_zcopy + |
+size_t | ++max_iov | +
+ Maximal iovcnt parameter in uct_iface_tag_recv_zcopy + |
+size_t | ++max_outstanding | +
+ Maximal number of simultaneous receive operations + |
struct uct_iface_attr.cap.tag.eager | +
Data Fields | ||
---|---|---|
+size_t | ++max_short | +
+ Maximal allowed data length in uct_ep_tag_eager_short + |
+size_t | ++max_bcopy | +
+ Maximal allowed data length in uct_ep_tag_eager_bcopy + |
+size_t | ++max_zcopy | +
+ Maximal allowed data length in uct_ep_tag_eager_zcopy + |
+size_t | ++max_iov | +
+ Maximal iovcnt parameter in uct_ep_tag_eager_zcopy + |
struct uct_iface_attr.cap.tag.rndv | +
Data Fields | ||
---|---|---|
+size_t | ++max_zcopy | +
+ Maximal allowed data length in uct_ep_tag_rndv_zcopy + |
+size_t | ++max_hdr | +
+ Maximal allowed header length in uct_ep_tag_rndv_zcopy and uct_ep_tag_rndv_request + |
+size_t | ++max_iov | +
+ Maximal iovcnt parameter in uct_ep_tag_rndv_zcopy + |
struct uct_iface_attr.cap.atomic32 | +
struct uct_iface_attr.cap.atomic64 | +
struct uct_iface_params | +
This structure should be allocated by the user and should be passed to uct_iface_open. User has to initialize all fields of this structure.
+Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_iface_params_field. Fields not specified in this mask will be ignored. + |
+ucs_cpu_set_t | ++cpu_mask | +
+ Mask of CPUs to use for resources + |
+uint64_t | ++open_mode | +
+ Interface open mode bitmap. uct_iface_open_mode + |
+union uct_iface_params | ++mode | +
+ Mode-specific parameters + |
+ucs_stats_node_t * | ++stats_root | +
+ Root in the statistics tree. Can be NULL. If non NULL, it will be a root of uct_iface object in the statistics tree. + |
+size_t | ++rx_headroom | +
+ How much bytes to reserve before the receive segment. + |
+void * | ++err_handler_arg | +
+ Custom argument of err_handler. + |
+uct_error_handler_t | ++err_handler | +
+ The callback to handle transport level error. + |
+uint32_t | ++err_handler_flags | +
+ Callback flags to indicate where the err_handler callback can be invoked from. uct_cb_flags + |
+void * | ++eager_arg | +
+ These callbacks are only relevant for HW Tag Matching + |
+uct_tag_unexp_eager_cb_t | ++eager_cb | +
+ Callback for tag matching unexpected eager messages + |
+void * | ++rndv_arg | ++ |
+uct_tag_unexp_rndv_cb_t | ++rndv_cb | +
+ Callback for tag matching unexpected rndv messages + |
+void * | ++async_event_arg | ++ |
+uct_async_event_cb_t | ++async_event_cb | +
+ Callback for asynchronous event handling. The callback will be invoked from UCT transport when there are new events to be read by user if the iface has UCT_IFACE_FLAG_EVENT_ASYNC_CB capability + |
+ucs_time_t | ++keepalive_interval | ++ |
+size_t | ++am_alignment | +
+ Desired alignment for Active Messages on the receiver. Note that only data received in the UCT descriptor can be aligned (i.e. UCT_CB_PARAM_FLAG_DESC flag is provided in the Active Message handler callback). The provided value must be power of 2. The default value is 1. + |
+size_t | ++am_align_offset | +
+ Offset in the Active Message receive buffer, which should be aligned to the am_alignment boundary. Note this parameter has no effect without setting am_alignment parameter. The provided value must be less than the given am_alignment value. The default value is 0. ++-+ pointer to data in uct_am_callback_t | | + alignment boundary | | v v +----------------—+ | align | | | offset | | +----------------—+ + |
+uint64_t | ++features | +
+ UCT features that are used for interface initialization. + |
union uct_iface_params.mode | +
Mode-specific parameters
+Data Fields | ||
---|---|---|
+mode | ++device | +
+ The fields in this structure (tl_name and dev_name) need to be set only when the UCT_IFACE_OPEN_MODE_DEVICE bit is set in uct_iface_params_t::open_mode This will make uct_iface_open open the interface on the specified device. + |
+mode | ++sockaddr | +
+ These callbacks and address are only relevant for client-server connection establishment with sockaddr and are needed on the server side. The callbacks and address need to be set when the UCT_IFACE_OPEN_MODE_SOCKADDR_SERVER bit is set in uct_iface_params_t::open_mode. This will make uct_iface_open open the interface on the specified address as a server. + |
struct uct_iface_params.mode.device | +
The fields in this structure (tl_name and dev_name) need to be set only when the UCT_IFACE_OPEN_MODE_DEVICE bit is set in uct_iface_params_t::open_mode This will make uct_iface_open open the interface on the specified device.
+Data Fields | ||
---|---|---|
+const char * | ++tl_name | +
+ Transport name + |
+const char * | ++dev_name | +
+ Device Name + |
struct uct_iface_params.mode.sockaddr | +
These callbacks and address are only relevant for client-server connection establishment with sockaddr and are needed on the server side. The callbacks and address need to be set when the UCT_IFACE_OPEN_MODE_SOCKADDR_SERVER bit is set in uct_iface_params_t::open_mode. This will make uct_iface_open open the interface on the specified address as a server.
+Data Fields | ||
---|---|---|
+ucs_sock_addr_t | ++listen_sockaddr | ++ |
+void * | ++conn_request_arg | +
+ Argument for connection request callback + |
+uct_sockaddr_conn_request_callback_t | ++conn_request_cb | +
+ Callback for an incoming connection request on the server + |
+uint32_t | ++cb_flags | +
+ Callback flags to indicate where the callback can be invoked from. uct_cb_flags + |
struct uct_ep_params | +
Data Fields | ||
---|---|---|
+uint64_t | ++field_mask | +
+ Mask of valid fields in this structure, using bits from uct_ep_params_field. Fields not specified by this mask will be ignored. + |
+uct_iface_h | ++iface | +
+ Interface to create the endpoint on. Either iface or cm field must be initialized but not both. + |
+void * | ++user_data | +
+ User data associated with the endpoint. + |
+const uct_device_addr_t * | ++dev_addr | +
+ The device address to connect to on the remote peer. This must be defined together with uct_ep_params_t::iface_addr to create an endpoint connected to a remote interface. + |
+const uct_iface_addr_t * | ++iface_addr | +
+ This specifies the remote address to use when creating an endpoint that is connected to a remote interface. +
|
+const ucs_sock_addr_t * | ++sockaddr | +
+ The sockaddr to connect to on the remote peer. If set, uct_ep_create will create an endpoint for a connection to the remote peer, specified by its socket address. +
|
+uint32_t | ++sockaddr_cb_flags | +
+ uct_cb_flags to indicate uct_ep_params_t::sockaddr_pack_cb, uct_ep_params_t::sockaddr_cb_client, uct_ep_params_t::sockaddr_cb_server, uct_ep_params_t::disconnect_cb and uct_ep_params_t::cm_resolve_cb behavior. If none from these are not set, this field will be ignored. + |
+uct_cm_ep_priv_data_pack_callback_t | ++sockaddr_pack_cb | +
+ Callback that will be used for filling the user's private data to be delivered to the remote peer by the callback on the server or client side. This field is only valid if uct_ep_params_t::sockaddr is set. +
|
+uct_cm_h | ++cm | +
+ The connection manager object as created by uct_cm_open. Either cm or iface field must be initialized but not both. + |
+uct_conn_request_h | ++conn_request | +
+ Connection request that was passed to uct_cm_listener_conn_request_args_t::conn_request. +
|
+uct_cm_ep_client_connect_callback_t | ++sockaddr_cb_client | +
+ Callback that will be invoked when the endpoint on the client side is being connected to the server by a connection manager uct_cm_h . + |
+uct_cm_ep_server_conn_notify_callback_t | ++sockaddr_cb_server | +
+ Callback that will be invoked when the endpoint on the server side is being connected to a client by a connection manager uct_cm_h . + |
+uct_ep_disconnect_cb_t | ++disconnect_cb | +
+ Callback that will be invoked when the endpoint is disconnected. + |
+unsigned | ++path_index | +
+ Index of the path which the endpoint should use, must be in the range 0..(uct_iface_attr_t::dev_num_paths - 1). + |
+uct_cm_ep_resolve_callback_t | ++cm_resolve_cb | +
+ This callback is invoked when the remote server address provided in field uct_ep_params_t::sockaddr is resolved to the local device to be used for connection establishment. +
|
+const void * | ++private_data | +
+ Private data to be passed from server to client. Can be used only along with uct_ep_params::conn_request. +
|
+size_t | ++private_data_length | +
+ Length of uct_ep_params::private_data, the maximal allowed value is indicated by the uct_cm_attr::max_conn_priv. + |
+const ucs_sock_addr_t * | ++local_sockaddr | +
+ The sockaddr to bind locally. If set, uct_ep_create will create an endpoint binding to this local sockaddr. +
|
struct uct_completion | +
This structure should be allocated by the user and can be passed to communication primitives. The user must initialize all fields of the structure. If the operation returns UCS_INPROGRESS, this structure will be in use by the transport until the operation completes. When the operation completes, "count" field is decremented by 1, and whenever it reaches 0 - the callback is called.
+Notes:
+Data Fields | ||
---|---|---|
+uct_completion_callback_t | ++func | +
+ User callback function + |
+int | ++count | +
+ Completion counter + |
+ucs_status_t | ++status | +
+ Completion status, this field must be initialized with UCS_OK before first operation is started. + |
struct uct_pending_req | +
This structure should be passed to uct_ep_pending_add() and is used to signal new available resources back to user.
+Data Fields | ||
---|---|---|
+uct_pending_callback_t | ++func | +
+ User callback function + |
+char | ++priv[UCT_PENDING_REQ_PRIV_LEN] | +
+ Used internally by UCT + |
struct uct_iov | +
Specifies a list of buffers which can be used within a single data transfer function call.
+buffer + | + +-----------+-------+-----------+-------+-----------+ + | payload | empty | payload | empty | payload | + +-----------+-------+-----------+-------+-----------+ + |<-length-->| |<-length-->| |<-length-->| + |<---- stride ----->|<---- stride ----->| +
Data Fields | ||
---|---|---|
+void * | ++buffer | +
+ Data buffer + |
+size_t | ++length | +
+ Length of the payload in bytes + |
+uct_mem_h | ++memh | +
+ Local memory key descriptor for the data + |
+size_t | ++stride | +
+ Stride between beginnings of payload elements in the buffer in bytes + |
+unsigned | ++count | +
+ Number of payload elements in the buffer + |
typedef struct uct_md_resource_desc uct_md_resource_desc_t | +
This structure describes a memory domain resource.
+ +typedef struct uct_component_attr uct_component_attr_t | +
This structure defines the attributes for UCT component. It is used for uct_component_query
+ +typedef struct uct_tl_resource_desc uct_tl_resource_desc_t | +
Resource descriptor is an object representing the network resource. Resource descriptor could represent a stand-alone communication resource such as an HCA port, network interface, or multiple resources such as multiple network interfaces or communication ports. It could also represent virtual communication resources that are defined over a single physical network interface.
+ +typedef struct uct_component* uct_component_h | +
typedef struct uct_iface* uct_iface_h | +
typedef struct uct_iface_config uct_iface_config_t | +
typedef struct uct_md_config uct_md_config_t | +
typedef struct uct_cm_config uct_cm_config_t | +
typedef struct uct_ep* uct_ep_h | +
typedef void* uct_mem_h | +
typedef uintptr_t uct_rkey_t | +
typedef struct uct_md* uct_md_h | +
typedef struct uct_md_ops uct_md_ops_t | +
typedef void* uct_rkey_ctx_h | +
typedef struct uct_iface_attr uct_iface_attr_t | +
typedef struct uct_iface_params uct_iface_params_t | +
typedef struct uct_ep_attr uct_ep_attr_t | +
typedef struct uct_md_attr uct_md_attr_t | +
typedef struct uct_completion uct_completion_t | +
typedef struct uct_pending_req uct_pending_req_t | +
typedef struct uct_worker* uct_worker_h | +
typedef struct uct_md uct_md_t | +
typedef enum uct_am_trace_type uct_am_trace_type_t | +
typedef struct uct_device_addr uct_device_addr_t | +
typedef struct uct_iface_addr uct_iface_addr_t | +
typedef struct uct_ep_addr uct_ep_addr_t | +
typedef struct uct_ep_params uct_ep_params_t | +
typedef struct uct_ep_connect_params uct_ep_connect_params_t | +
typedef struct uct_cm_attr uct_cm_attr_t | +
typedef struct uct_cm uct_cm_t | +
typedef struct uct_listener_attr uct_listener_attr_t | +
typedef struct uct_listener* uct_listener_h | +
typedef struct uct_listener_params uct_listener_params_t | +
typedef struct uct_tag_context uct_tag_context_t | +
typedef uint64_t uct_tag_t | +
typedef int uct_worker_cb_id_t | +
typedef void* uct_conn_request_h | +
Specifies a list of buffers which can be used within a single data transfer function call.
+buffer + | + +-----------+-------+-----------+-------+-----------+ + | payload | empty | payload | empty | payload | + +-----------+-------+-----------+-------+-----------+ + |<-length-->| |<-length-->| |<-length-->| + |<---- stride ----->|<---- stride ----->| +
typedef void(* uct_completion_callback_t)(uct_completion_t *self) | +
[in] | self | Pointer to relevant completion structure, which was initially passed to the operation. |
typedef ucs_status_t(* uct_pending_callback_t)(uct_pending_req_t *self) | +
[in] | self | Pointer to relevant pending structure, which was initially passed to the operation. |
typedef ucs_status_t(* uct_error_handler_t)(void *arg, uct_ep_h ep, ucs_status_t status) | +
[in] | arg | User argument to be passed to the callback. |
[in] | ep | Endpoint which has failed. Upon return from the callback, this ep is no longer usable and all subsequent operations on this ep will fail with the error code passed in status. |
[in] | status | Status indicating error. |
typedef void(* uct_pending_purge_callback_t)(uct_pending_req_t *self, void *arg) | +
[in] | self | Pointer to relevant pending structure, which was initially passed to the operation. |
[in] | arg | User argument to be passed to the callback. |
typedef size_t(* uct_pack_callback_t)(void *dest, void *arg) | +
[in] | dest | Memory buffer to pack the data to. |
[in] | arg | Custom user-argument. |
typedef void(* uct_unpack_callback_t)(void *arg, const void *data, size_t length) | +
[in] | arg | Custom user-argument. |
[in] | data | Memory buffer to unpack the data from. |
[in] | length | How much data to consume (size of "data") |
typedef void(* uct_async_event_cb_t)(void *arg, unsigned flags) | +
[in] | arg | User argument to be passed to the callback. |
[in] | flags | Flags to be passed to the callback (reserved for future use). |
enum uct_component_attr_field | +
The enumeration allows specifying which fields in uct_component_attr_t are present. It is used for backward compatibility support.
+Enumerator | |
---|---|
UCT_COMPONENT_ATTR_FIELD_NAME |
+ Component name + |
UCT_COMPONENT_ATTR_FIELD_MD_RESOURCE_COUNT |
+ MD resource count + |
UCT_COMPONENT_ATTR_FIELD_MD_RESOURCES |
+ MD resources array + |
UCT_COMPONENT_ATTR_FIELD_FLAGS |
+ Capability flags + |
anonymous enum | +
The enumeration defines bit mask of uct_component_h capabilities in uct_component_attr_t::flags which is set by uct_component_query.
+Enumerator | |
---|---|
UCT_COMPONENT_FLAG_CM |
+ If set, the component supports uct_cm_h functionality. See uct_cm_open for details. + |
UCT_COMPONENT_FLAG_RKEY_PTR |
+ If set, the component supports direct access to remote memory using a local pointer returned from uct_rkey_ptr function. + |
enum uct_device_type_t | +
enum uct_iface_event_types | +
Enumerator | |
---|---|
UCT_EVENT_SEND_COMP |
+ Send completion event + |
UCT_EVENT_RECV |
+ Tag or active message received + |
UCT_EVENT_RECV_SIG |
+ Signaled tag or active message received + |
enum uct_flush_flags | +
Enumerator | |
---|---|
UCT_FLUSH_FLAG_LOCAL |
+ Guarantees that the data transfer is completed but the target buffer may not be updated yet. + |
UCT_FLUSH_FLAG_CANCEL |
+ The library will make a best effort attempt to cancel all uncompleted operations. However, there is a chance that some operations will not be canceled in which case the user will need to handle their completions through the relevant callbacks. After uct_ep_flush with this flag is completed, the endpoint will be set to error state, and it becomes unusable for send operations and should be destroyed. + |
UCT_FLUSH_FLAG_REMOTE |
+ Guarantees that all previous UCP memory update operations (put, atomics, etc.) are completed, the target memory of these operation was updated, and the updated memory is globally visible for all processing elements in the system. + |
enum uct_progress_types | +
Enumerator | |
---|---|
UCT_PROGRESS_SEND |
+ Progress send operations + |
UCT_PROGRESS_RECV |
+ Progress receive operations + |
UCT_PROGRESS_THREAD_SAFE |
+ Enable/disable progress while another thread may be calling ucp_worker_progress(). + |
enum uct_cb_flags | +
List of flags for a callback.
+Enumerator | |
---|---|
UCT_CB_FLAG_RESERVED |
+ Reserved for future use. + |
UCT_CB_FLAG_ASYNC |
+ Callback is allowed to be called from any thread in the process, and therefore should be thread-safe. For example, it may be called from a transport async progress thread. To guarantee async invocation, the interface must have the UCT_IFACE_FLAG_CB_ASYNC flag set. If async callback is requested on an interface which only supports sync callback (i.e., only the UCT_IFACE_FLAG_CB_SYNC flag is set), the callback will be invoked only from the context that called uct_iface_progress). + |
enum uct_iface_open_mode | +
Enumerator | |
---|---|
UCT_IFACE_OPEN_MODE_DEVICE |
+ Interface is opened on a specific device + |
UCT_IFACE_OPEN_MODE_SOCKADDR_SERVER |
+ Interface is opened on a specific address on the server side. This mode will be deprecated in the near future for a better API. + |
UCT_IFACE_OPEN_MODE_SOCKADDR_CLIENT |
+ Interface is opened on a specific address on the client side This mode will be deprecated in the near future for a better API. + |
enum uct_iface_params_field | +
The enumeration allows specifying which fields in uct_iface_params_t are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_IFACE_PARAM_FIELD_CPU_MASK |
+ Enables uct_iface_params_t::cpu_mask + |
UCT_IFACE_PARAM_FIELD_OPEN_MODE |
+ Enables uct_iface_params_t::open_mode + |
UCT_IFACE_PARAM_FIELD_DEVICE |
+ Enables uct_iface_params_t::mode::device + |
UCT_IFACE_PARAM_FIELD_SOCKADDR | + + |
UCT_IFACE_PARAM_FIELD_STATS_ROOT |
+ Enables uct_iface_params_t::stats_root + |
UCT_IFACE_PARAM_FIELD_RX_HEADROOM |
+ Enables uct_iface_params_t::rx_headroom + |
UCT_IFACE_PARAM_FIELD_ERR_HANDLER_ARG | + + |
UCT_IFACE_PARAM_FIELD_ERR_HANDLER |
+ Enables uct_iface_params_t::err_handler + |
UCT_IFACE_PARAM_FIELD_ERR_HANDLER_FLAGS | + + |
UCT_IFACE_PARAM_FIELD_HW_TM_EAGER_ARG |
+ Enables uct_iface_params_t::eager_arg + |
UCT_IFACE_PARAM_FIELD_HW_TM_EAGER_CB |
+ Enables uct_iface_params_t::eager_cb + |
UCT_IFACE_PARAM_FIELD_HW_TM_RNDV_ARG |
+ Enables uct_iface_params_t::rndv_arg + |
UCT_IFACE_PARAM_FIELD_HW_TM_RNDV_CB |
+ Enables uct_iface_params_t::rndv_cb + |
UCT_IFACE_PARAM_FIELD_ASYNC_EVENT_ARG | + + |
UCT_IFACE_PARAM_FIELD_ASYNC_EVENT_CB | + + |
UCT_IFACE_PARAM_FIELD_KEEPALIVE_INTERVAL | + + |
UCT_IFACE_PARAM_FIELD_AM_ALIGNMENT |
+ Enables uct_iface_params_t::am_alignment + |
UCT_IFACE_PARAM_FIELD_AM_ALIGN_OFFSET | + + |
UCT_IFACE_PARAM_FIELD_FEATURES |
+ Enables uct_iface_params_t::features + |
enum uct_ep_params_field | +
The enumeration allows specifying which fields in uct_ep_params_t are present, for backward compatibility support.
+Enumerator | |
---|---|
UCT_EP_PARAM_FIELD_IFACE |
+ Enables uct_ep_params::iface + |
UCT_EP_PARAM_FIELD_USER_DATA |
+ Enables uct_ep_params::user_data + |
UCT_EP_PARAM_FIELD_DEV_ADDR |
+ Enables uct_ep_params::dev_addr + |
UCT_EP_PARAM_FIELD_IFACE_ADDR |
+ Enables uct_ep_params::iface_addr + |
UCT_EP_PARAM_FIELD_SOCKADDR |
+ Enables uct_ep_params::sockaddr + |
UCT_EP_PARAM_FIELD_SOCKADDR_CB_FLAGS |
+ Enables uct_ep_params::sockaddr_cb_flags + |
UCT_EP_PARAM_FIELD_SOCKADDR_PACK_CB |
+ Enables uct_ep_params::sockaddr_pack_cb + |
UCT_EP_PARAM_FIELD_CM |
+ Enables uct_ep_params::cm + |
UCT_EP_PARAM_FIELD_CONN_REQUEST |
+ Enables uct_ep_params::conn_request + |
UCT_EP_PARAM_FIELD_SOCKADDR_CONNECT_CB_CLIENT | + + |
UCT_EP_PARAM_FIELD_SOCKADDR_NOTIFY_CB_SERVER | + + |
UCT_EP_PARAM_FIELD_SOCKADDR_DISCONNECT_CB |
+ Enables uct_ep_params::disconnect_cb + |
UCT_EP_PARAM_FIELD_PATH_INDEX |
+ Enables uct_ep_params::path_index + |
UCT_EP_PARAM_FIELD_CM_RESOLVE_CB |
+ Enables uct_ep_params::cm_resolve_cb + |
UCT_EP_PARAM_FIELD_PRIV_DATA |
+ Enables uct_ep_params::private_data + |
UCT_EP_PARAM_FIELD_PRIV_DATA_LENGTH | + + |
UCT_EP_PARAM_FIELD_LOCAL_SOCKADDR |
+ Enables uct_ep_params::local_sockaddr + |
enum uct_iface_feature | +
The enumeration list describes the features supported by UCT. An application can request the features using UCT parameters during UCT iface initialization process.
+Enumerator | |
---|---|
UCT_IFACE_FEATURE_AM |
+ Request Active Message support + |
UCT_IFACE_FEATURE_PUT |
+ Request PUT support + |
UCT_IFACE_FEATURE_GET |
+ Request GET support + |
UCT_IFACE_FEATURE_AMO32 |
+ Request 32-bit atomic operations support + |
UCT_IFACE_FEATURE_AMO64 |
+ Request 64-bit atomic operations support + |
UCT_IFACE_FEATURE_TAG |
+ Request tag matching offload support + |
UCT_IFACE_FEATURE_FLUSH_REMOTE |
+ Request remote flush support + |
UCT_IFACE_FEATURE_LAST |
+ Used to determine the number of features + |
enum uct_cb_param_flags | +
If UCT_CB_PARAM_FLAG_DESC flag is enabled, then data is part of a descriptor which includes the user-defined rx_headroom, and the callback may return UCS_INPROGRESS and hold on to that descriptor. Otherwise, the data can't be used outside the callback. If needed, the data must be copied-out.
+descriptor data + | | + +-------------+-------------------------+ + | rx_headroom | payload | + +-------------+-------------------------+ +
UCT_CB_PARAM_FLAG_FIRST and UCT_CB_PARAM_FLAG_MORE flags are relevant for uct_tag_unexp_eager_cb_t callback only. The former value indicates that the data is the first fragment of the message. The latter value means that more fragments of the message yet to be delivered.
+Enumerator | |
---|---|
UCT_CB_PARAM_FLAG_DESC | + |
UCT_CB_PARAM_FLAG_FIRST | + |
UCT_CB_PARAM_FLAG_MORE | + |
ucs_status_t uct_query_components | +( | +uct_component_h ** | +components_p, | +
+ | + | unsigned * | +num_components_p | +
+ | ) | ++ |
Obtain the list of transport components available on the current system.
+[out] | components_p | Filled with a pointer to an array of component handles. |
[out] | num_components_p | Filled with the number of elements in the array. |
void uct_release_component_list | +( | +uct_component_h * | +components | ) | ++ |
This routine releases the memory associated with the list of components allocated by uct_query_components.
+[in] | components | Array of component handles to release. |
ucs_status_t uct_component_query | +( | +uct_component_h | +component, | +
+ | + | uct_component_attr_t * | +component_attr | +
+ | ) | ++ |
Query various attributes of a component.
+[in] | component | Component handle to query attributes for. The handle can be obtained from uct_query_components. |
[in,out] | component_attr | Filled with component attributes. |
ucs_status_t uct_md_open | +( | +uct_component_h | +component, | +
+ | + | const char * | +md_name, | +
+ | + | const uct_md_config_t * | +config, | +
+ | + | uct_md_h * | +md_p | +
+ | ) | ++ |
Open a specific memory domain. All communications and memory operations are performed in the context of a specific memory domain. Therefore it must be created before communication resources.
+[in] | component | Component on which to open the memory domain, as returned from uct_query_components. |
[in] | md_name | Memory domain name, as returned from uct_component_query. |
[in] | config | MD configuration options. Should be obtained from uct_md_config_read() function, or point to MD-specific structure which extends uct_md_config_t. |
[out] | md_p | Filled with a handle to the memory domain. |
void uct_md_close | +( | +uct_md_h | +md | ) | ++ |
[in] | md | Memory domain to close. |
ucs_status_t uct_md_query_tl_resources | +( | +uct_md_h | +md, | +
+ | + | uct_tl_resource_desc_t ** | +resources_p, | +
+ | + | unsigned * | +num_resources_p | +
+ | ) | ++ |
This routine queries the memory domain for communication resources that are available for it.
+[in] | md | Handle to memory domain. |
[out] | resources_p | Filled with a pointer to an array of resource descriptors. |
[out] | num_resources_p | Filled with the number of resources in the array. |
void uct_release_tl_resource_list | +( | +uct_tl_resource_desc_t * | +resources | ) | ++ |
This routine releases the memory associated with the list of resources allocated by uct_md_query_tl_resources.
+[in] | resources | Array of resource descriptors to release. |
ucs_status_t uct_md_iface_config_read | +( | +uct_md_h | +md, | +
+ | + | const char * | +tl_name, | +
+ | + | const char * | +env_prefix, | +
+ | + | const char * | +filename, | +
+ | + | uct_iface_config_t ** | +config_p | +
+ | ) | ++ |
[in] | md | Memory domain on which the transport's interface was registered. |
[in] | tl_name | Transport name. If md supports UCT_MD_FLAG_SOCKADDR, the transport name is allowed to be NULL. In this case, the configuration returned from this routine should be passed to uct_iface_open with UCT_IFACE_OPEN_MODE_SOCKADDR_SERVER or UCT_IFACE_OPEN_MODE_SOCKADDR_CLIENT set in uct_iface_params_t::open_mode. In addition, if tl_name is not NULL, the configuration returned from this routine should be passed to uct_iface_open with UCT_IFACE_OPEN_MODE_DEVICE set in uct_iface_params_t::open_mode. |
[in] | env_prefix | If non-NULL, search for environment variables starting with this UCT_<prefix>_. Otherwise, search for environment variables starting with just UCT_. |
[in] | filename | If non-NULL, read configuration from this file. If the file does not exist, it will be ignored. |
[out] | config_p | Filled with a pointer to configuration. |
void uct_config_release | +( | +void * | +config | ) | ++ |
[in] | config | Configuration to release. |
ucs_status_t uct_iface_open | +( | +uct_md_h | +md, | +
+ | + | uct_worker_h | +worker, | +
+ | + | const uct_iface_params_t * | +params, | +
+ | + | const uct_iface_config_t * | +config, | +
+ | + | uct_iface_h * | +iface_p | +
+ | ) | ++ |
[in] | md | Memory domain to create the interface on. |
[in] | worker | Handle to worker which will be used to progress communications on this interface. |
[in] | params | User defined uct_iface_params_t parameters. |
[in] | config | Interface configuration options. Should be obtained from uct_md_iface_config_read() function, or point to transport-specific structure which extends uct_iface_config_t. |
[out] | iface_p | Filled with a handle to opened communication interface. |
void uct_iface_close | +( | +uct_iface_h | +iface | ) | ++ |
[in] | iface | Interface to close. |
ucs_status_t uct_iface_query | +( | +uct_iface_h | +iface, | +
+ | + | uct_iface_attr_t * | +iface_attr | +
+ | ) | ++ |
[in] | iface | Interface to query. |
[out] | iface_attr | Filled with interface attributes. |
ucs_status_t uct_iface_get_device_address | +( | +uct_iface_h | +iface, | +
+ | + | uct_device_addr_t * | +addr | +
+ | ) | ++ |
Get underlying device address of the interface. All interfaces using the same device would return the same address.
+[in] | iface | Interface to query. |
[out] | addr | Filled with device address. The size of the buffer provided must be at least uct_iface_attr_t::device_addr_len. |
ucs_status_t uct_iface_get_address | +( | +uct_iface_h | +iface, | +
+ | + | uct_iface_addr_t * | +addr | +
+ | ) | ++ |
requires UCT_IFACE_FLAG_CONNECT_TO_IFACE.
+[in] | iface | Interface to query. |
[out] | addr | Filled with interface address. The size of the buffer provided must be at least uct_iface_attr_t::iface_addr_len. |
int uct_iface_is_reachable | +( | +const uct_iface_h | +iface, | +
+ | + | const uct_device_addr_t * | +dev_addr, | +
+ | + | const uct_iface_addr_t * | +iface_addr | +
+ | ) | ++ |
This function checks if a remote address can be reached from a local interface. If the function returns true, it does not necessarily mean a connection and/or data transfer would succeed, since the reachability check is a local operation it does not detect issues such as network mis-configuration or lack of connectivity.
+[in] | iface | Interface to check reachability from. |
[in] | dev_addr | Device address to check reachability to. It is NULL if iface_attr.dev_addr_len == 0, and must be non-NULL otherwise. |
[in] | iface_addr | Interface address to check reachability to. It is NULL if iface_attr.iface_addr_len == 0, and must be non-NULL otherwise. |
ucs_status_t uct_ep_check | +( | +const uct_ep_h | +ep, | +
+ | + | unsigned | +flags, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
This function checks if the destination endpoint is alive with respect to the UCT library. If the status of ep is known, either UCS_OK or an error is returned immediately. Otherwise, UCS_INPROGRESS is returned, indicating that synchronization on the status is needed. In this case, the status will be be propagated by comp callback.
+[in] | ep | Endpoint to check |
[in] | flags | Flags that define level of check (currently unsupported - set to 0). |
[in] | comp | Handler to process status of ep |
ucs_status_t uct_iface_event_fd_get | +( | +uct_iface_h | +iface, | +
+ | + | int * | +fd_p | +
+ | ) | ++ |
Only interfaces that support at least one of the UCT_IFACE_FLAG_EVENT* flags will implement this function.
+[in] | iface | Interface to get the notification descriptor. |
[out] | fd_p | Location to write the notification file descriptor. |
ucs_status_t uct_iface_event_arm | +( | +uct_iface_h | +iface, | +
+ | + | unsigned | +events | +
+ | ) | ++ |
This routine needs to be called before waiting on each notification on this interface, so will typically be called once the processing of the previous event is over.
+[in] | iface | Interface to arm. |
[in] | events | Events to wakeup on. See uct_iface_event_types |
ucs_status_t uct_iface_mem_alloc | +( | +uct_iface_h | +iface, | +
+ | + | size_t | +length, | +
+ | + | unsigned | +flags, | +
+ | + | const char * | +name, | +
+ | + | uct_allocated_memory_t * | +mem | +
+ | ) | ++ |
Allocate a region of memory which can be used for zero-copy data transfer or remote access on a particular transport interface.
+[in] | iface | Interface to allocate memory on. |
[in] | length | Size of memory region to allocate. |
[in] | flags | Memory allocation flags, see uct_md_mem_flags. |
[in] | name | Allocation name, for debug purposes. |
[out] | mem | Descriptor of allocated memory. |
void uct_iface_mem_free | +( | +const uct_allocated_memory_t * | +mem | ) | ++ |
[in] | mem | Descriptor of memory to release. |
ucs_status_t uct_ep_create | +( | +const uct_ep_params_t * | +params, | +
+ | + | uct_ep_h * | +ep_p | +
+ | ) | ++ |
Create a UCT endpoint in one of the available modes:
+[in] | params | User defined uct_ep_params_t configuration for the ep_p. |
[out] | ep_p | Filled with handle to the new endpoint. |
void uct_ep_destroy | +( | +uct_ep_h | +ep | ) | ++ |
[in] | ep | Endpoint to destroy. |
ucs_status_t uct_ep_get_address | +( | +uct_ep_h | +ep, | +
+ | + | uct_ep_addr_t * | +addr | +
+ | ) | ++ |
[in] | ep | Endpoint to query. |
[out] | addr | Filled with endpoint address. The size of the buffer provided must be at least uct_iface_attr_t::ep_addr_len. |
ucs_status_t uct_ep_connect_to_ep | +( | +uct_ep_h | +ep, | +
+ | + | const uct_device_addr_t * | +dev_addr, | +
+ | + | const uct_ep_addr_t * | +ep_addr | +
+ | ) | ++ |
requires UCT_IFACE_FLAG_CONNECT_TO_EP capability.
+[in] | ep | Endpoint to connect. |
[in] | dev_addr | Remote device address. |
[in] | ep_addr | Remote endpoint address. |
ucs_status_t uct_iface_flush | +( | +uct_iface_h | +iface, | +
+ | + | unsigned | +flags, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
Flushes all outstanding communications issued on the interface prior to this call. The operations are completed at the origin or at the target as well. The exact completion semantic depends on flags parameter.
+[in] | iface | Interface to flush communications from. |
[in] | flags | Flags that control completion semantic (currently only UCT_FLUSH_FLAG_LOCAL is supported). |
[in,out] | comp | Completion handle as defined by uct_completion_t. Can be NULL, which means that the call will return the current state of the interface and no completion will be generated in case of outstanding communications. If it is not NULL completion counter is decremented by 1 when the call completes. Completion callback is called when the counter reaches 0. |
ucs_status_t uct_iface_fence | +( | +uct_iface_h | +iface, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
[in] | iface | Interface to issue communications from. |
[in] | flags | Flags that control ordering semantic (currently unsupported - set to 0). |
ucs_status_t uct_ep_pending_add | +( | +uct_ep_h | +ep, | +
+ | + | uct_pending_req_t * | +req, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
Add a pending request to the endpoint pending queue. The request will be dispatched when the endpoint could potentially have additional send resources.
+[in] | ep | Endpoint to add the pending request to. |
[in] | req | Pending request, which would be dispatched when more resources become available. The user is expected to initialize the "func" field. After being passed to the function, the request is owned by UCT, until the callback is called and returns UCS_OK. |
[in] | flags | Flags that control pending request processing (see uct_cb_flags) |
void uct_ep_pending_purge | +( | +uct_ep_h | +ep, | +
+ | + | uct_pending_purge_callback_t | +cb, | +
+ | + | void * | +arg | +
+ | ) | ++ |
Remove pending requests from the given endpoint and pass them to the provided callback function. The callback return value is ignored.
+[in] | ep | Endpoint to remove pending requests from. |
[in] | cb | Callback to pass the removed requests to. |
[in] | arg | Argument to pass to the cb callback. |
ucs_status_t uct_ep_flush | +( | +uct_ep_h | +ep, | +
+ | + | unsigned | +flags, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
Flushes all outstanding communications issued on the endpoint prior to this call. The operations are completed at the origin or at the target as well. The exact completion semantic depends on flags parameter.
+[in] | ep | Endpoint to flush communications from. |
[in] | flags | Flags uct_flush_flags that control completion semantic. |
[in,out] | comp | Completion handle as defined by uct_completion_t. Can be NULL, which means that the call will return the current state of the endpoint and no completion will be generated in case of outstanding communications. If it is not NULL completion counter is decremented by 1 when the call completes. Completion callback is called when the counter reaches 0. |
ucs_status_t uct_ep_fence | +( | +uct_ep_h | +ep, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
[in] | ep | Endpoint to issue communications from. |
[in] | flags | Flags that control ordering semantic (currently unsupported - set to 0). |
void uct_iface_progress_enable | +( | +uct_iface_h | +iface, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
Notify the transport that it should actively progress communications during uct_worker_progress().
+When the interface is created, its progress is initially disabled.
+[in] | iface | The interface to enable progress. |
[in] | flags | The type of progress to enable as defined by uct_progress_types |
void uct_iface_progress_disable | +( | +uct_iface_h | +iface, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
Notify the transport that it should not progress its communications during uct_worker_progress(). Thus the latency of other transports may be improved.
+By default, progress is disabled when the interface is created.
+[in] | iface | The interface to disable progress. |
[in] | flags | The type of progress to disable as defined by uct_progress_types. |
unsigned uct_iface_progress | +( | +uct_iface_h | +iface | ) | ++ |
+
|
+ +static | +
comp | [in] Completion handle to update. |
status | [in] Status to update comp handle. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
List of capabilities supported by UCX API. +More...
++Macros | |
#define | UCT_IFACE_FLAG_AM_SHORT UCS_BIT(0) |
#define | UCT_IFACE_FLAG_AM_BCOPY UCS_BIT(1) |
#define | UCT_IFACE_FLAG_AM_ZCOPY UCS_BIT(2) |
#define | UCT_IFACE_FLAG_PENDING UCS_BIT(3) |
#define | UCT_IFACE_FLAG_PUT_SHORT UCS_BIT(4) |
#define | UCT_IFACE_FLAG_PUT_BCOPY UCS_BIT(5) |
#define | UCT_IFACE_FLAG_PUT_ZCOPY UCS_BIT(6) |
#define | UCT_IFACE_FLAG_GET_SHORT UCS_BIT(8) |
#define | UCT_IFACE_FLAG_GET_BCOPY UCS_BIT(9) |
#define | UCT_IFACE_FLAG_GET_ZCOPY UCS_BIT(10) |
#define | UCT_IFACE_FLAG_ATOMIC_CPU UCS_BIT(30) |
#define | UCT_IFACE_FLAG_ATOMIC_DEVICE UCS_BIT(31) |
#define | UCT_IFACE_FLAG_ERRHANDLE_SHORT_BUF UCS_BIT(32) |
#define | UCT_IFACE_FLAG_ERRHANDLE_BCOPY_BUF UCS_BIT(33) |
#define | UCT_IFACE_FLAG_ERRHANDLE_ZCOPY_BUF UCS_BIT(34) |
#define | UCT_IFACE_FLAG_ERRHANDLE_AM_ID UCS_BIT(35) |
#define | UCT_IFACE_FLAG_ERRHANDLE_REMOTE_MEM UCS_BIT(36) |
#define | UCT_IFACE_FLAG_ERRHANDLE_BCOPY_LEN UCS_BIT(37) |
#define | UCT_IFACE_FLAG_ERRHANDLE_PEER_FAILURE UCS_BIT(38) |
#define | UCT_IFACE_FLAG_EP_CHECK UCS_BIT(39) |
#define | UCT_IFACE_FLAG_CONNECT_TO_IFACE UCS_BIT(40) |
#define | UCT_IFACE_FLAG_CONNECT_TO_EP UCS_BIT(41) |
#define | UCT_IFACE_FLAG_CONNECT_TO_SOCKADDR UCS_BIT(42) |
#define | UCT_IFACE_FLAG_AM_DUP UCS_BIT(43) |
#define | UCT_IFACE_FLAG_CB_SYNC UCS_BIT(44) |
#define | UCT_IFACE_FLAG_CB_ASYNC UCS_BIT(45) |
#define | UCT_IFACE_FLAG_EP_KEEPALIVE UCS_BIT(46) |
#define | UCT_IFACE_FLAG_TAG_EAGER_SHORT UCS_BIT(50) |
#define | UCT_IFACE_FLAG_TAG_EAGER_BCOPY UCS_BIT(51) |
#define | UCT_IFACE_FLAG_TAG_EAGER_ZCOPY UCS_BIT(52) |
#define | UCT_IFACE_FLAG_TAG_RNDV_ZCOPY UCS_BIT(53) |
The definition list presents a full list of operations and capabilities exposed by UCX API.
+#define UCT_IFACE_FLAG_AM_SHORT UCS_BIT(0) | +
Short active message
+#define UCT_IFACE_FLAG_AM_BCOPY UCS_BIT(1) | +
Buffered active message
+#define UCT_IFACE_FLAG_AM_ZCOPY UCS_BIT(2) | +
Zero-copy active message
+#define UCT_IFACE_FLAG_PENDING UCS_BIT(3) | +
Pending operations
+ +#define UCT_IFACE_FLAG_PUT_SHORT UCS_BIT(4) | +
Short put
+ +#define UCT_IFACE_FLAG_PUT_BCOPY UCS_BIT(5) | +
Buffered put
+ +#define UCT_IFACE_FLAG_PUT_ZCOPY UCS_BIT(6) | +
Zero-copy put
+ +#define UCT_IFACE_FLAG_GET_SHORT UCS_BIT(8) | +
Short get
+ +#define UCT_IFACE_FLAG_GET_BCOPY UCS_BIT(9) | +
Buffered get
+ +#define UCT_IFACE_FLAG_GET_ZCOPY UCS_BIT(10) | +
Zero-copy get
+ +#define UCT_IFACE_FLAG_ATOMIC_CPU UCS_BIT(30) | +
Atomic communications are consistent with respect to CPU operations.
+ +#define UCT_IFACE_FLAG_ATOMIC_DEVICE UCS_BIT(31) | +
Atomic communications are consistent only with respect to other atomics on the same device.
+ +#define UCT_IFACE_FLAG_ERRHANDLE_SHORT_BUF UCS_BIT(32) | +
Invalid buffer for short operation
+ +#define UCT_IFACE_FLAG_ERRHANDLE_BCOPY_BUF UCS_BIT(33) | +
Invalid buffer for buffered operation
+ +#define UCT_IFACE_FLAG_ERRHANDLE_ZCOPY_BUF UCS_BIT(34) | +
Invalid buffer for zero copy operation
+ +#define UCT_IFACE_FLAG_ERRHANDLE_AM_ID UCS_BIT(35) | +
Invalid AM id on remote
+ +#define UCT_IFACE_FLAG_ERRHANDLE_REMOTE_MEM UCS_BIT(36) | +
Remote memory access
+ +#define UCT_IFACE_FLAG_ERRHANDLE_BCOPY_LEN UCS_BIT(37) | +
Invalid length for buffered operation
+ +#define UCT_IFACE_FLAG_ERRHANDLE_PEER_FAILURE UCS_BIT(38) | +
Remote peer failures/outage
+ +#define UCT_IFACE_FLAG_EP_CHECK UCS_BIT(39) | +
Endpoint check
+ +#define UCT_IFACE_FLAG_CONNECT_TO_IFACE UCS_BIT(40) | +
Supports connecting to interface
+#define UCT_IFACE_FLAG_CONNECT_TO_EP UCS_BIT(41) | +
Supports connecting to specific endpoint
+#define UCT_IFACE_FLAG_CONNECT_TO_SOCKADDR UCS_BIT(42) | +
Supports connecting to sockaddr
+ +#define UCT_IFACE_FLAG_AM_DUP UCS_BIT(43) | +
Active messages may be received with duplicates This happens if the transport does not keep enough information to detect retransmissions
+ +#define UCT_IFACE_FLAG_CB_SYNC UCS_BIT(44) | +
Interface supports setting a callback which is invoked only from the calling context of uct_worker_progress()
+ +#define UCT_IFACE_FLAG_CB_ASYNC UCS_BIT(45) | +
Interface supports setting a callback which will be invoked within a reasonable amount of time if uct_worker_progress() is not being called. The callback can be invoked from any progress context and it may also be invoked when uct_worker_progress() is called.
+ +#define UCT_IFACE_FLAG_EP_KEEPALIVE UCS_BIT(46) | +
Transport endpoint has built-in keepalive feature, which guarantees the error callback on the transport interface will be called if the communication channel with remote peer is broken, even if there are no outstanding send operations
+ +#define UCT_IFACE_FLAG_TAG_EAGER_SHORT UCS_BIT(50) | +
Hardware tag matching short eager support
+ +#define UCT_IFACE_FLAG_TAG_EAGER_BCOPY UCS_BIT(51) | +
Hardware tag matching bcopy eager support
+ +#define UCT_IFACE_FLAG_TAG_EAGER_ZCOPY UCS_BIT(52) | +
Hardware tag matching zcopy eager support
+ +#define UCT_IFACE_FLAG_TAG_RNDV_ZCOPY UCS_BIT(53) | +
Hardware tag matching rendezvous zcopy support
+ ++ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
List of capabilities supported by UCT iface event API. +More...
++Macros | |
#define | UCT_IFACE_FLAG_EVENT_SEND_COMP UCS_BIT(0) |
#define | UCT_IFACE_FLAG_EVENT_RECV UCS_BIT(1) |
#define | UCT_IFACE_FLAG_EVENT_RECV_SIG UCS_BIT(2) |
#define | UCT_IFACE_FLAG_EVENT_FD UCS_BIT(3) |
#define | UCT_IFACE_FLAG_EVENT_ASYNC_CB UCS_BIT(4) |
The definition list presents a full list of operations and capabilities supported by UCT iface event.
+#define UCT_IFACE_FLAG_EVENT_SEND_COMP UCS_BIT(0) | +
Event notification of send completion is supported
+ +#define UCT_IFACE_FLAG_EVENT_RECV UCS_BIT(1) | +
Event notification of tag and active message receive is supported
+ +#define UCT_IFACE_FLAG_EVENT_RECV_SIG UCS_BIT(2) | +
Event notification of signaled tag and active message is supported
+ +#define UCT_IFACE_FLAG_EVENT_FD UCS_BIT(3) | +
Event notification through File Descriptor is supported
+ +#define UCT_IFACE_FLAG_EVENT_ASYNC_CB UCS_BIT(4) | +
Event notification through asynchronous callback invocation is supported
+ ++ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Functions | |
ucs_status_t | uct_ep_put_short (uct_ep_h ep, const void *buffer, unsigned length, uint64_t remote_addr, uct_rkey_t rkey) |
ssize_t | uct_ep_put_bcopy (uct_ep_h ep, uct_pack_callback_t pack_cb, void *arg, uint64_t remote_addr, uct_rkey_t rkey) |
ucs_status_t | uct_ep_put_zcopy (uct_ep_h ep, const uct_iov_t *iov, size_t iovcnt, uint64_t remote_addr, uct_rkey_t rkey, uct_completion_t *comp) |
Write data to remote memory while avoiding local memory copy. More... | |
ucs_status_t | uct_ep_get_short (uct_ep_h ep, void *buffer, unsigned length, uint64_t remote_addr, uct_rkey_t rkey) |
ucs_status_t | uct_ep_get_bcopy (uct_ep_h ep, uct_unpack_callback_t unpack_cb, void *arg, size_t length, uint64_t remote_addr, uct_rkey_t rkey, uct_completion_t *comp) |
ucs_status_t | uct_ep_get_zcopy (uct_ep_h ep, const uct_iov_t *iov, size_t iovcnt, uint64_t remote_addr, uct_rkey_t rkey, uct_completion_t *comp) |
Read data from remote memory while avoiding local memory copy. More... | |
Defines remote memory access operations.
+ucs_status_t uct_ep_put_short | +( | +uct_ep_h | +ep, | +
+ | + | const void * | +buffer, | +
+ | + | unsigned | +length, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey | +
+ | ) | ++ |
ssize_t uct_ep_put_bcopy | +( | +uct_ep_h | +ep, | +
+ | + | uct_pack_callback_t | +pack_cb, | +
+ | + | void * | +arg, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey | +
+ | ) | ++ |
ucs_status_t uct_ep_put_zcopy | +( | +uct_ep_h | +ep, | +
+ | + | const uct_iov_t * | +iov, | +
+ | + | size_t | +iovcnt, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
The input data in iov array of uct_iov_t structures sent to remote address ("gather output"). Buffers in iov are processed in array order. This means that the function complete iov[0] before proceeding to iov[1], and so on.
+[in] | ep | Destination endpoint handle. |
[in] | iov | Points to an array of uct_iov_t structures. The iov pointer must be a valid address of an array of uct_iov_t structures. A particular structure pointer must be a valid address. A NULL terminated array is not required. |
[in] | iovcnt | Size of the iov data uct_iov_t structures array. If iovcnt is zero, the data is considered empty. iovcnt is limited by uct_iface_attr::cap::put::max_iov. |
[in] | remote_addr | Remote address to place the iov data. |
[in] | rkey | Remote key descriptor provided by uct_rkey_unpack |
[in] | comp | Completion handle as defined by uct_completion_t. |
ucs_status_t uct_ep_get_short | +( | +uct_ep_h | +ep, | +
+ | + | void * | +buffer, | +
+ | + | unsigned | +length, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey | +
+ | ) | ++ |
ucs_status_t uct_ep_get_bcopy | +( | +uct_ep_h | +ep, | +
+ | + | uct_unpack_callback_t | +unpack_cb, | +
+ | + | void * | +arg, | +
+ | + | size_t | +length, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
ucs_status_t uct_ep_get_zcopy | +( | +uct_ep_h | +ep, | +
+ | + | const uct_iov_t * | +iov, | +
+ | + | size_t | +iovcnt, | +
+ | + | uint64_t | +remote_addr, | +
+ | + | uct_rkey_t | +rkey, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
The output data in iov array of uct_iov_t structures received from remote address ("scatter input"). Buffers in iov are processed in array order. This means that the function complete iov[0] before proceeding to iov[1], and so on.
+[in] | ep | Destination endpoint handle. |
[in] | iov | Points to an array of uct_iov_t structures. The iov pointer must be a valid address of an array of uct_iov_t structures. A particular structure pointer must be a valid address. A NULL terminated array is not required. |
[in] | iovcnt | Size of the iov data uct_iov_t structures array. If iovcnt is zero, the data is considered empty. iovcnt is limited by uct_iface_attr::cap::get::max_iov. |
[in] | remote_addr | Remote address of the data placed to the iov. |
[in] | rkey | Remote key descriptor provided by uct_rkey_unpack |
[in] | comp | Completion handle as defined by uct_completion_t. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
+Data Structures | |
struct | uct_tag_context |
Posted tag context. More... | |
+Typedefs | |
typedef ucs_status_t(* | uct_tag_unexp_eager_cb_t )(void *arg, void *data, size_t length, unsigned flags, uct_tag_t stag, uint64_t imm, void **context) |
Callback to process unexpected eager tagged message. More... | |
typedef ucs_status_t(* | uct_tag_unexp_rndv_cb_t )(void *arg, unsigned flags, uint64_t stag, const void *header, unsigned header_length, uint64_t remote_addr, size_t length, const void *rkey_buf) |
Callback to process unexpected rendezvous tagged message. More... | |
+Functions | |
ucs_status_t | uct_ep_tag_eager_short (uct_ep_h ep, uct_tag_t tag, const void *data, size_t length) |
Short eager tagged-send operation. More... | |
ssize_t | uct_ep_tag_eager_bcopy (uct_ep_h ep, uct_tag_t tag, uint64_t imm, uct_pack_callback_t pack_cb, void *arg, unsigned flags) |
Bcopy eager tagged-send operation. More... | |
ucs_status_t | uct_ep_tag_eager_zcopy (uct_ep_h ep, uct_tag_t tag, uint64_t imm, const uct_iov_t *iov, size_t iovcnt, unsigned flags, uct_completion_t *comp) |
Zcopy eager tagged-send operation. More... | |
ucs_status_ptr_t | uct_ep_tag_rndv_zcopy (uct_ep_h ep, uct_tag_t tag, const void *header, unsigned header_length, const uct_iov_t *iov, size_t iovcnt, unsigned flags, uct_completion_t *comp) |
Rendezvous tagged-send operation. More... | |
ucs_status_t | uct_ep_tag_rndv_cancel (uct_ep_h ep, void *op) |
Cancel outstanding rendezvous operation. More... | |
ucs_status_t | uct_ep_tag_rndv_request (uct_ep_h ep, uct_tag_t tag, const void *header, unsigned header_length, unsigned flags) |
Send software rendezvous request. More... | |
ucs_status_t | uct_iface_tag_recv_zcopy (uct_iface_h iface, uct_tag_t tag, uct_tag_t tag_mask, const uct_iov_t *iov, size_t iovcnt, uct_tag_context_t *ctx) |
Post a tag to a transport interface. More... | |
ucs_status_t | uct_iface_tag_recv_cancel (uct_iface_h iface, uct_tag_context_t *ctx, int force) |
Cancel a posted tag. More... | |
Defines tag matching operations.
+typedef ucs_status_t(* uct_tag_unexp_eager_cb_t)(void *arg, void *data, size_t length, unsigned flags, uct_tag_t stag, uint64_t imm, void **context) | +
This callback is invoked when tagged message sent by eager protocol has arrived and no corresponding tag has been posted.
+[in] | arg | User-defined argument |
[in] | data | Points to the received unexpected data. |
[in] | length | Length of data. |
[in] | flags | Mask with uct_cb_param_flags flags. If it contains UCT_CB_PARAM_FLAG_DESC value, this means data is part of a descriptor which must be released later using uct_iface_release_desc by the user if the callback returns UCS_INPROGRESS. |
[in] | stag | Tag from sender. |
[in] | imm | Immediate data from sender. |
[in,out] | context | Storage for a per-message user-defined context. In this context, the message is defined by the sender side as a single call to uct_ep_tag_eager_short/bcopy/zcopy. On the transport level the message can be fragmented and delivered to the target over multiple fragments. The fragments will preserve the original order of the message. Each fragment will result in invocation of the above callback. The user can use UCT_CB_PARAM_FLAG_FIRST to identify the first fragment, allocate the context object and use the context as a token that is set by the user and passed to subsequent callbacks of the same message. The user is responsible for allocation and release of the context. |
UCS_OK | - data descriptor was consumed, and can be released by the caller. |
UCS_INPROGRESS | - data descriptor is owned by the callee, and will be released later. |
typedef ucs_status_t(* uct_tag_unexp_rndv_cb_t)(void *arg, unsigned flags, uint64_t stag, const void *header, unsigned header_length, uint64_t remote_addr, size_t length, const void *rkey_buf) | +
This callback is invoked when rendezvous send notification has arrived and no corresponding tag has been posted.
+[in] | arg | User-defined argument |
[in] | flags | Mask with uct_cb_param_flags |
[in] | stag | Tag from sender. |
[in] | header | User defined header. |
[in] | header_length | User defined header length in bytes. |
[in] | remote_addr | Sender's buffer virtual address. |
[in] | length | Sender's buffer length. |
[in] | rkey_buf | Sender's buffer packed remote key. It can be passed to uct_rkey_unpack() to create uct_rkey_t. |
UCS_OK | - descriptor was consumed, and can be released by the caller. |
UCS_INPROGRESS | - descriptor is owned by the callee, and would be released later. |
ucs_status_t uct_ep_tag_eager_short | +( | +uct_ep_h | +ep, | +
+ | + | uct_tag_t | +tag, | +
+ | + | const void * | +data, | +
+ | + | size_t | +length | +
+ | ) | ++ |
This routine sends a message using short eager protocol. Eager protocol means that the whole data is sent to the peer immediately without any preceding notification. The data is provided as buffer and its length,and must not be larger than the corresponding max_short value in uct_iface_attr. The immediate value delivered to the receiver is implicitly equal to 0. If it's required to pass nonzero imm value, uct_ep_tag_eager_bcopy should be used.
+[in] | ep | Destination endpoint handle. |
[in] | tag | Tag to use for the eager message. |
[in] | data | Data to send. |
[in] | length | Data length. |
ssize_t uct_ep_tag_eager_bcopy | +( | +uct_ep_h | +ep, | +
+ | + | uct_tag_t | +tag, | +
+ | + | uint64_t | +imm, | +
+ | + | uct_pack_callback_t | +pack_cb, | +
+ | + | void * | +arg, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
This routine sends a message using bcopy eager protocol. Eager protocol means that the whole data is sent to the peer immediately without any preceding notification. Custom data callback is used to copy the data to the network buffers.
+[in] | ep | Destination endpoint handle. |
[in] | tag | Tag to use for the eager message. |
[in] | imm | Immediate value which will be available to the receiver. |
[in] | pack_cb | User callback to pack the data. |
[in] | arg | Custom argument to pack_cb. |
[in] | flags | Tag message flags, see uct_msg_flags. |
ucs_status_t uct_ep_tag_eager_zcopy | +( | +uct_ep_h | +ep, | +
+ | + | uct_tag_t | +tag, | +
+ | + | uint64_t | +imm, | +
+ | + | const uct_iov_t * | +iov, | +
+ | + | size_t | +iovcnt, | +
+ | + | unsigned | +flags, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
This routine sends a message using zcopy eager protocol. Eager protocol means that the whole data is sent to the peer immediately without any preceding notification. The input data (which has to be previously registered) in iov array of uct_iov_t structures sent to remote side ("gather output"). Buffers in iov are processed in array order, so the function complete iov[0] before proceeding to iov[1], and so on.
+[in] | ep | Destination endpoint handle. |
[in] | tag | Tag to use for the eager message. |
[in] | imm | Immediate value which will be available to the receiver. |
[in] | iov | Points to an array of uct_iov_t structures. A particular structure pointer must be a valid address. A NULL terminated array is not required. |
[in] | iovcnt | Size of the iov array. If iovcnt is zero, the data is considered empty. Note that iovcnt is limited by the corresponding max_iov value in uct_iface_attr. |
[in] | flags | Tag message flags, see uct_msg_flags. |
[in] | comp | Completion callback which will be called when the data is reliably received by the peer, and the buffer can be reused or invalidated. |
ucs_status_ptr_t uct_ep_tag_rndv_zcopy | +( | +uct_ep_h | +ep, | +
+ | + | uct_tag_t | +tag, | +
+ | + | const void * | +header, | +
+ | + | unsigned | +header_length, | +
+ | + | const uct_iov_t * | +iov, | +
+ | + | size_t | +iovcnt, | +
+ | + | unsigned | +flags, | +
+ | + | uct_completion_t * | +comp | +
+ | ) | ++ |
This routine sends a message using rendezvous protocol. Rendezvous protocol means that only a small notification is sent at first, and the data itself is transferred later (when there is a match) to avoid extra memory copy.
+[in] | ep | Destination endpoint handle. |
[in] | tag | Tag to use for the eager message. |
[in] | header | User defined header. |
[in] | header_length | User defined header length in bytes. Note that it is limited by the corresponding max_hdr value in uct_iface_attr. |
[in] | iov | Points to an array of uct_iov_t structures. A particular structure pointer must be valid address. A NULL terminated array is not required. |
[in] | iovcnt | Size of the iov array. If iovcnt is zero, the data is considered empty. Note that iovcnt is limited by the corresponding max_iov value in uct_iface_attr. |
[in] | flags | Tag message flags, see uct_msg_flags. |
[in] | comp | Completion callback which will be called when the data is reliably received by the peer, and the buffer can be reused or invalidated. |
ucs_status_t uct_ep_tag_rndv_cancel | +( | +uct_ep_h | +ep, | +
+ | + | void * | +op | +
+ | ) | ++ |
This routine signals the underlying transport disregard the outstanding operation without calling completion callback provided in uct_ep_tag_rndv_zcopy.
+[in] | ep | Destination endpoint handle. |
[in] | op | Rendezvous operation handle, as returned from uct_ep_tag_rndv_zcopy. |
ucs_status_t uct_ep_tag_rndv_request | +( | +uct_ep_h | +ep, | +
+ | + | uct_tag_t | +tag, | +
+ | + | const void * | +header, | +
+ | + | unsigned | +header_length, | +
+ | + | unsigned | +flags | +
+ | ) | ++ |
This routine sends a rendezvous request only, which indicates that the data transfer should be completed in software.
+[in] | ep | Destination endpoint handle. |
[in] | tag | Tag to use for matching. |
[in] | header | User defined header |
[in] | header_length | User defined header length in bytes. Note that it is limited by the corresponding max_hdr value in uct_iface_attr. |
[in] | flags | Tag message flags, see uct_msg_flags. |
ucs_status_t uct_iface_tag_recv_zcopy | +( | +uct_iface_h | +iface, | +
+ | + | uct_tag_t | +tag, | +
+ | + | uct_tag_t | +tag_mask, | +
+ | + | const uct_iov_t * | +iov, | +
+ | + | size_t | +iovcnt, | +
+ | + | uct_tag_context_t * | +ctx | +
+ | ) | ++ |
This routine posts a tag to be matched on a transport interface. When a message with the corresponding tag arrives it is stored in the user buffer (described by iov and iovcnt) directly. The operation completion is reported using callbacks on the ctx structure.
+[in] | iface | Interface to post the tag on. |
[in] | tag | Tag to expect. |
[in] | tag_mask | Mask which specifies what bits of the tag to compare. |
[in] | iov | Points to an array of uct_iov_t structures. The iov pointer must be a valid address of an array of uct_iov_t structures. A particular structure pointer must be a valid address. A NULL terminated array is not required. |
[in] | iovcnt | Size of the iov data uct_iov_t structures array. If iovcnt is zero, the data is considered empty. iovcnt is limited by uct_iface_attr::cap::tag::max_iov. |
[in,out] | ctx | Context associated with this particular tag, "priv" field in this structure is used to track the state internally. |
ucs_status_t uct_iface_tag_recv_cancel | +( | +uct_iface_h | +iface, | +
+ | + | uct_tag_context_t * | +ctx, | +
+ | + | int | +force | +
+ | ) | ++ |
This routine cancels a tag, which was previously posted by uct_iface_tag_recv_zcopy. The tag would be either matched or canceled, in a bounded time, regardless of the peer actions. The original completion callback of the tag would be called with the status if force is not set.
+[in] | iface | Interface to cancel the tag on. |
[in] | ctx | Tag context which was used for posting the tag. If force is 0, ctx->completed_cb will be called with either UCS_OK which means the tag was matched and data received despite the cancel request, or UCS_ERR_CANCELED which means the tag was successfully canceled before it was matched. |
[in] | force | Whether to report completions to ctx->completed_cb. If nonzero, the cancel is assumed to be successful, and the callback is not called. |
+ |
+ UCX
+ 1.15
+
+ Unified Communication X
+ |
+
This document describes the UCX programming interface. The programming interface exposes a high performance communication API, which provides basic building blocks for PGAS, Message Passing Interface (MPI), Big-Data, Analytics, File I/O, and storage library developers.
+This manual is intended for programmers who want to develop parallel programming models like OpenSHMEM, MPI, UPC, Chapel, etc. The manual assumes that the reader is familiar with the following:
+This section briefly describes a list of open issues in the UCX specification.
+UCX project follows open source development model and the software is licensed under BSD-3 license.
+t |