From ae194c3a343c42194dfb7b3007995b6e8b5f9d26 Mon Sep 17 00:00:00 2001 From: David Solt Date: Thu, 31 Mar 2022 15:20:29 -0500 Subject: [PATCH] Improvements to publish_lookup chapter Signed-off-by: David Solt - Add PMIX_DATA_TO_PUBLISH directive - Clarify how an implementation distinguished between attributes and data - require that ranges for publish and lookup match - Determine how custom ranges are handled: Publisher need not be a member of a custom range - simplified retrieval rules - Add PMIX_ACCESS_USERIDS and GRPIDS to list of attributes related to publish - move callback presentation to Chap_API_Struct chapter - move lookup structure prior to its use in the chapter --- Chap_API_Publish.tex | 221 +++++++++++++++++++++--------------------- Chap_API_Struct.tex | 34 +++++++ Chap_Introduction.tex | 2 + 3 files changed, 148 insertions(+), 109 deletions(-) diff --git a/Chap_API_Publish.tex b/Chap_API_Publish.tex index 894c57b7..137f2462 100644 --- a/Chap_API_Publish.tex +++ b/Chap_API_Publish.tex @@ -5,20 +5,17 @@ \chapter{Publish/Lookup Operations} \label{chap:pub} Chapter~\ref{chap:api_rsvd_keys} and Chapter~\ref{chap:data_sharing:non_rsvd_keys} -discussed how reserved and non-reserved keys dealt with -information that either was associated with a specific process (i.e., the -retrieving process knew the identifier of the process that posted it) or -required a synchronization operation prior to retrieval (e.g., the case of +present how reserved and non-reserved keys deal with +information that either is associated with a specific process (i.e., the +retrieving process knows the identifier of the process that posted it) or +requires a synchronization operation prior to retrieval (e.g., the case of globally unique non-reserved keys). However, another requirement exists for an asynchronous exchange of data where neither the posting nor the retrieving -process is known in advance. For example, two separate namespaces may need to -rendezvous with each other without knowing in advance the identity of the other -namespace or when that namespace might become active. - +process is known in advance, for example, two namespaces that do not share a child-parent relationship. The \acp{API} defined in this section focus on resolving that specific situation by allowing processes to publish data that can subsequently be retrieved solely by referral to its key. Mechanisms for constraining -availability of the information are also provided as a means for better +the scope of availability of the information are also provided as a means for better targeting of the eventual recipient(s). Note that no presumption is made regarding how the published information is to be stored, nor as to the entity (host environment or \ac{PMIx} implementation) that shall act as the datastore. The descriptions in the remainder of this chapter shall simply refer to that entity as the \emph{datastore}. @@ -60,6 +57,8 @@ \section{\code{PMIx_Publish}} \pasteAttributeItem{PMIX_TIMEOUT} \pasteAttributeItem{PMIX_RANGE} \pasteAttributeItem{PMIX_PERSISTENCE} +\pasteAttributeItem{PMIX_ACCESS_USERIDS} +\pasteAttributeItem{PMIX_ACCESS_GRPIDS} \pasteAttributeItem{PMIX_ACCESS_PERMISSIONS} \optattrend @@ -74,9 +73,34 @@ \section{\code{PMIx_Publish}} The blocking form of this call will block until it has obtained confirmation from the datastore that the data is available for lookup. The \refarg{info} array can be released upon return from the blocking function call. Publishing duplicate keys is permitted provided they are published to different -ranges. Duplicate keys being published on the same data range shall return the +ranges. Custom ranges are consider different if they have different members. +Duplicate keys being published on the same data range shall return the \refconst{PMIX_ERR_DUPLICATE_KEY} error. - +publishing to a \refconst{PMIX_RANGE_CUSTOM} +range which does not include the publisher will prevent +any processes from using \refapi{PMIx_Lookup} to access the published data. + +In some cases, implementations may be incapable of distinguishing which +info keys in the \refarg{info} array are for publishing and which info keys are +directives. To make it clear, it is recommended that the keys to be published +are designated by passing them as a \refstruct{pmix_data_array_t} using the +\refattr{PMIX_DATA_TO_PUBLISH} directive. +If the \refarg{info} array contains a \refattr{PMIX_DATA_TO_PUBLISH} info, +all other elements of the info array will be treated as directives. +If the info array does not include a \refattr{PMIX_DATA_TO_PUBLISH} info, +the implementation should +distinguishing between info array elements that specify keys and directives as follows: +All standardized directives to the publish call, +including optional attributes the implementation does not support, +should be treated as +directives. Non-supported directives +may be ignored as outlined in Section \ref{intro:portability:attributes}, +but should not be treated as data to +publish. The implementation may treat any custom (non-standardized) directives it +supports as directives. All other \refarg{info} array elements +should it be assumed to be data to be published. +Since additional directives may be added to the standard and implementations may add support for additional custom directives, the use of \refattr{PMIX_DATA_TO_PUBLISH} is the only reliable way to ensure that +future implementations will not mis-classify elements of an \refarg{info} array. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -123,6 +147,8 @@ \section{\code{PMIx_Publish_nb}} \pasteAttributeItem{PMIX_TIMEOUT} \pasteAttributeItem{PMIX_RANGE} \pasteAttributeItem{PMIX_PERSISTENCE} +\pasteAttributeItem{PMIX_ACCESS_USERIDS} +\pasteAttributeItem{PMIX_ACCESS_GRPIDS} \pasteAttributeItem{PMIX_ACCESS_PERMISSIONS} \optattrend @@ -155,7 +181,7 @@ \section{Publish-specific attributes} % \declareAttribute{PMIX_RANGE}{"pmix.range"}{pmix_data_range_t}{ -Define constraints on the processes that can access the provided data. Only processes that meet the constraints are allowed to access it. +Define constraints on the processes that can access published data or generated events or define constraints on the provider of data when looking up published data. } % \declareAttribute{PMIX_PERSISTENCE}{"pmix.persist"}{pmix_persistence_t}{ @@ -181,12 +207,13 @@ \section{Publish-Lookup Datatypes} The following data types are defined for use with the \refapi{PMIx_Publish} \acp{API}. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \subsection{Range of Published Data} \declarestruct{pmix_data_range_t} \versionMarker{1.0} -The \refstruct{pmix_data_range_t} structure is a \code{uint8_t} type that defines a range for both data \textit{published} via the \refapi{PMIx_Publish} \ac{API} and generated events. +The \refstruct{pmix_data_range_t} structure is a \code{uint8_t} type that defines a range for data \textit{published} via the \refapi{PMIx_Publish} \ac{API} and events generated via the \refapi{PMIx_Notify_event}. The following constants can be used to set a variable of the type \refstruct{pmix_data_range_t}. \begin{constantdesc} @@ -195,25 +222,33 @@ \subsection{Range of Published Data} Undefined range. % \declareconstitemvalue{PMIX_RANGE_RM}{1} -Data is intended for the host environment, or lookup is restricted to data published by the host environment. +Published data and generated events are restricted to processes executing under the same instance of the host environment as the publisher or event creator. Lookup of data is restricted to data published by processes running under the same instance of the host environment as the requester. % \declareconstitemvalue{PMIX_RANGE_LOCAL}{2} -Data is only available to processes on the local node, or lookup is restricted to data published by processes on the local node of the requester. +Published data and generated events are restricted to processes on the same node as the publisher or event creator. Lookup of data is restricted to data published by processes on the same node as the requester. % \declareconstitemvalue{PMIX_RANGE_NAMESPACE}{3} -Data is only available to processes in the same namespace, or lookup is restricted to data published by processes in the same namespace as the requester. +Published data and generated events are restricted to processes in the same namespace as the publisher or event creator. +Lookup of data is restricted to data published by procesess in the same namespace as the requester. % \declareconstitemvalue{PMIX_RANGE_SESSION}{4} -Data is only available to all processes in the session, or lookup is restricted to data published by other processes in the same session as the requester. +Published data and generated events are restricted to processes in the same session as the publisher or event creator. +Lookup of data is restricted to data published by procesess in the same session as the requester. % \declareconstitemvalue{PMIX_RANGE_GLOBAL}{5} -Data is available to all processes, or lookup is open to data published by anyone. -% -\declareconstitemvalue{PMIX_RANGE_CUSTOM}{6} -Data is available only to processes as specified in the \refstruct{pmix_info_t} associated with this call, or lookup is restricted to data published by processes as specified in the \refstruct{pmix_info_t}. +Published data and generated events are available to all processes within the domain of the host environment. +Lookup of data is unrestricted and open to data published by any processes within the domain of the host enivornment as the requester. This range differs from \refconst{PMIX_RANGE_RM} only on systems which have mechanisms to share events and +publish/lookup data across multiple instances of a host environment. % \declareconstitemvalue{PMIX_RANGE_PROC_LOCAL}{7} -Data is only available to this process, or lookup is restricted to data published by this process. +Published data and generated events are available only to calling process. +Lookup of data is restricted to data published by the calling process. +% +\declareconstitemvalue{PMIX_RANGE_CUSTOM}{6} +Published data and generated events are restricted to processes +described in the \refstruct{pmix_info_t} associated with this call. +Lookup of data is restricted to data published by the processes described in +in the \refstruct{pmix_info_t}. % \declareconstitemvalue{PMIX_RANGE_INVALID}{UINT8_MAX} Invalid value - typically used to indicate that a range has not yet been set. @@ -232,7 +267,7 @@ \subsection{Data Persistence Structure} \begin{constantdesc} % \declareconstitemvalue{PMIX_PERSIST_INDEF}{0} -Retain data until specifically deleted. +Retain data until unpublished. % \declareconstitemvalue{PMIX_PERSIST_FIRST_READ}{1} Retain data until the first access, then the data is deleted. @@ -252,6 +287,33 @@ \subsection{Data Persistence Structure} \end{constantdesc} +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\subsection{Lookup Related Data Structures} + +\declarestruct{pmix_pdata_t} + +The \refstruct{pmix_pdata_t} structure is used both to request the lookup of keys and to describe the value and publishing process of any keys that were successfully retrieved. +A request to lookup published values is described by an array of \refstruct{pmix_pdata_t} structures. +Only the key field is used in the lookup request. +The results of the lookup operation are returned in the same array with the proc and value fields set when the key is successfully found. +The value field's data type is set to \refconst{PMIX_UNDEF} in the associated \refarg{value} struct of any key which was not retrieved. +% +\copySignature{pmix_pdata_t}{1.0}{ +typedef struct pmix_pdata \{ \\ +\hspace*{4\sigspace}pmix_proc_t proc; \\ +\hspace*{4\sigspace}pmix_key_t key; \\ +\hspace*{4\sigspace}pmix_value_t value; \\ +\} pmix_pdata_t; +} + +where: +\begin{itemize} + \item \emph{proc} is the process identifier of the data publisher. + \item \emph{key} is the string key of the published data. + \item \emph{value} is the value associated with the \emph{key}. +\end{itemize} + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{\code{PMIx_Lookup}} @@ -260,7 +322,7 @@ \section{\code{PMIx_Lookup}} %%%% \summary -Lookup information published by this or another process with \refapi{PMIx_Publish} or \refapi{PMIx_Publish_nb}. +Lookup information published by a process or host environment using \refapi{PMIx_Publish} or \refapi{PMIx_Publish_nb}. %%%% \format @@ -307,9 +369,11 @@ \section{\code{PMIx_Lookup}} %%%% \descr -Lookup information published by this or another process. -By default, the search will be constrained to publishers that fall within the \refconst{PMIX_RANGE_SESSION} range in case duplicate keys exist on different ranges. -Changes to the range (e.g., expanding the search to all potential publishers via the \refconst{PMIX_RANGE_GLOBAL} constant), and any additional directives, can be provided in the \refstruct{pmix_info_t} array. Data is returned per the retrieval rules of Section \ref{chap:pub:retrules}. +Lookup information published by a process or host environment using \refapi{PMIx_Publish} or \refapi{PMIx_Publish_nb}. +A lookup operation is always performed on a range which can be specified using the directive \refAttributeItem{PMIX_RANGE} or otherwise defaults to \refconst{PMIX_RANGE_SESSION}. + +The lookup operation will be constrained to data published to the specified range. +Data is returned per the retrieval rules of Section \ref{chap:pub:retrules}. The \argref{data} parameter consists of an array of \refstruct{pmix_pdata_t} structures with the keys specifying the requested information. Data will be returned for each \code{key} field in the associated \code{value} field of this structure as per the above description of return values. The \code{proc} field in each \refstruct{pmix_pdata_t} structure will contain the namespace/rank of the process that published the data. @@ -359,7 +423,7 @@ \section{\code{PMIx_Lookup_nb}} \item \refconst{PMIX_ERR_NOT_FOUND} None of the requested data was available within the requester's range. The \refarg{pdata} array in the callback function shall be \code{NULL} and the \refarg{npdata} parameter set to zero. \item \refconst{PMIX_ERR_PARTIAL_SUCCESS} Some of the requested data was found. -Only found data will be included in the returned \refarg{pdata} array. Note that the specific reason for a particular piece of missing information (e.g., lack of permissions) cannot be communicated back to the requester in this situation. +Only found data will be included in the returned \refarg{pdata} array. Note that the specific reason for a particular piece of missing information (e.g., lack of permissions or the data has not been published) cannot be communicated back to the requester in this situation. \item \refconst{PMIX_ERR_NOT_SUPPORTED} There is no available datastore (either at the host environment or \ac{PMIx} implementation level) on this system that supports this function. @@ -387,29 +451,6 @@ \section{\code{PMIx_Lookup_nb}} Non-blocking form of the \refapi{PMIx_Lookup} function. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\subsection{Lookup Returned Data Structure} -\declarestruct{pmix_pdata_t} - -The \refstruct{pmix_pdata_t} structure is used by \refapi{PMIx_Lookup} to describe the data being accessed. - -\copySignature{pmix_pdata_t}{1.0}{ -typedef struct pmix_pdata \{ \\ -\hspace*{4\sigspace}pmix_proc_t proc; \\ -\hspace*{4\sigspace}pmix_key_t key; \\ -\hspace*{4\sigspace}pmix_value_t value; \\ -\} pmix_pdata_t; -} - -where: -\begin{itemize} - \item \emph{proc} is the process identifier of the data publisher. - \item \emph{key} is the string key of the published data. - \item \emph{value} is the value associated with the \emph{key}. -\end{itemize} - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \subsubsection{Lookup data structure support macros} @@ -542,59 +583,28 @@ \subsubsection{Lookup data structure support macros} Key, process identifier, and data will all be copied into the destination \refstruct{pmix_pdata_t} - thus, the source \refstruct{pmix_pdata_t} may free'd without affecting the copied data once the macro has completed. \adviceuserend - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\subsection{Lookup Callback Function} -\declareapi{pmix_lookup_cbfunc_t} - -%%%% -\summary - -The \refapi{pmix_lookup_cbfunc_t} is used by \refapi{PMIx_Lookup_nb} to return data. - -\copySignature{pmix_lookup_cbfunc_t}{1.0}{ -typedef void (*pmix_lookup_cbfunc_t) \\ -\hspace*{4\sigspace}(pmix_status_t status, \\ -\hspace*{5\sigspace}pmix_pdata_t data[], size_t ndata, \\ -\hspace*{5\sigspace}void *cbdata); -} - -\begin{arglist} -\argin{status}{Status associated with the operation (handle)} -\argin{data}{Array of data returned (\refstruct{pmix_pdata_t})} -\argin{ndata}{Number of elements in the \argref{data} array (\code{size_t})} -\argin{cbdata}{Callback data passed to original API call (memory reference)} -\end{arglist} - - -%%%% -\descr - -A callback function for calls to \refapi{PMIx_Lookup_nb}. -The function will be called upon completion of the \refapi{PMIx_Lookup_nb} \ac{API} with the \refarg{status} indicating the success or failure of the request. -Any retrieved data will be returned in an array of \refstruct{pmix_pdata_t} structs. -The namespace and rank of the process that provided each data element is also returned. - -Note that the \refstruct{pmix_pdata_t} structures will be released upon return from the callback function, so the receiver must copy/protect the data prior to returning if it needs to be retained. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Retrieval rules for published data} \label{chap:pub:retrules} -The retrieval rules for published data primarily revolve around enforcing data access permissions and range constraints. The datastore shall search its stored information for each specified key according to the following precedence logic: +The retrieval rules for published data primarily revolve around enforcing data access permissions and range constraints. +All publish and lookup operations operate on a range. If not specified, the range defaults to \refconst{PMIX_RANGE_SESSION}. +The key being looked up will match with a published key only if all of the following conditions are met: \begin{enumerate} - \item If the requester specified the range, then the search shall be constrained to data where the publishing process falls within the specified range. - - \item If the key of the stored information does not match the specified key, then the search will continue. - - \item If the requester's identifier does not fall within the range specified by the publisher, then the search will continue. - - \item If the publisher specified access permissions, the effective \ac{UID} and \ac{GID} of the requester shall be checked against those permissions, with the datastore rejecting the match if the requester fails to meet the requirements. - - \item If all of the above checks pass, then the value is added to the information that is to be returned. + \item The lookup key matches the published key. + \item The type of range specified by the publisher is the same as the type of range specified by the requester. + \item If a custom range is specified by the publisher and the requester, the members described in both cases must be identical. The publisher is not required to be a member of a custom range. + \item The requestor must be a member of the publisher's range. + +%%option2: +%% +%% \item The publisher's range must describe the same set of processes as the requestor's range. +%% \item The requestor must be a member of the range specified. +%% +%% The difference here is in how custom ranges are handled in which the publisher is not a member of the custom range it describes. The later case allows the publisher to publish to a custom range that includes a set of processes not including itself. + + \item If the publisher specified access permissions, the effective \ac{UID} and \ac{GID} of the requester must meet those requirements. \end{enumerate} The status returned by the datastore shall be set to: @@ -607,18 +617,11 @@ \section{Retrieval rules for published data} \item \refconst{PMIX_ERR_PARTIAL_SUCCESS} Some of the requested data was found. Only found data will be included in the returned information. Note that the specific reason for a particular piece of missing information (e.g., lack of permissions) cannot be communicated back to the requester in this situation. +\item \refconst {PMIX_ERR_NO_PERMISSIONS} All requested data was found and range restrictions were met for each specified key, but none of the matching data could be returned due to lack of access permissions. + \item a non-zero \ac{PMIx} error constant indicating a reason for the request's failure. \end{itemize} -In the case where data was found and range restrictions were met for each specified key, but none of the matching data could be returned due to lack of access permissions, the datastore must return the \refconst{PMIX_ERR_NO_PERMISSIONS} error. - -\adviceuserstart -Note that duplicate keys are allowed to exist on different ranges, and that ranges do overlap each other. Thus, if duplicate keys are published on overlapping ranges, it is possible for the datastore to successfully find multiple responses for a given key should publisher and requester specify sufficiently broad ranges. In this situation, the choice of resolving the duplication is left to the datastore implementation - e.g., it may return the first value found in its search, or the value corresponding to the most limited range of the found values, or it may choose to simply return an error. - -Users are advised to avoid this ambiguity by careful selection of key values and ranges - e.g., by creating range-specific keys where necessary. -\adviceuserend - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{\code{PMIx_Unpublish}} @@ -627,7 +630,7 @@ \section{\code{PMIx_Unpublish}} %%%% \summary -Unpublish data posted by this process using the given keys. +Unpublish a list of keys published by the calling process. %%%% \format @@ -662,7 +665,7 @@ \section{\code{PMIx_Unpublish}} %%%% \descr -Unpublish data posted by this process using the given \refarg{keys}. +Unpublish a list of keys published by the calling process. The function will block until the data has been removed by the server (i.e., it is safe to publish that key again within the specified range). A value of \code{NULL} for the \refarg{keys} parameter instructs the server to remove all data published by this process. diff --git a/Chap_API_Struct.tex b/Chap_API_Struct.tex index e1e5e76d..3298ba89 100644 --- a/Chap_API_Struct.tex +++ b/Chap_API_Struct.tex @@ -2458,6 +2458,40 @@ \subsection{Release Callback Function} Since the data is ``owned'' by the host server, provide a callback function to notify the host server that we are done with the data so it can be released. +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\subsection{Lookup Callback Function} +\declareapi{pmix_lookup_cbfunc_t} + +%%%% +\summary + +The \refapi{pmix_lookup_cbfunc_t} is used by \refapi{PMIx_Lookup_nb} to return data. + +\copySignature{pmix_lookup_cbfunc_t}{1.0}{ +typedef void (*pmix_lookup_cbfunc_t) \\ +\hspace*{4\sigspace}(pmix_status_t status, \\ +\hspace*{5\sigspace}pmix_pdata_t data[], size_t ndata, \\ +\hspace*{5\sigspace}void *cbdata); +} + +\begin{arglist} +\argin{status}{Status associated with the operation (handle)} +\argin{data}{Array of data returned (\refstruct{pmix_pdata_t})} +\argin{ndata}{Number of elements in the \argref{data} array (\code{size_t})} +\argin{cbdata}{Callback data passed to original API call (memory reference)} +\end{arglist} + +%%%% +\descr + +A callback function for calls to \refapi{PMIx_Lookup_nb}. +The function will be called upon completion of the \refapi{PMIx_Lookup_nb} \ac{API} with the \refarg{status} indicating the success or failure of the request. +Any retrieved data will be returned in an array of \refstruct{pmix_pdata_t} structs. +The namespace and rank of the process that provided each data element is also returned. + +Note that the \refstruct{pmix_pdata_t} structures will be released upon return from the callback +function, so the receiver must copy/protect the data prior to returning if it needs to be retained. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \subsection{Op Callback Function} diff --git a/Chap_Introduction.tex b/Chap_Introduction.tex index 4afaf995..4e357378 100644 --- a/Chap_Introduction.tex +++ b/Chap_Introduction.tex @@ -143,6 +143,8 @@ \section{Portability of Functionality} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \subsection{Attributes in PMIx} +\label{intro:portability:attributes} + An area where differences between support on different systems can be challenging is regarding the attributes that provide information to the client process and/or control the behavior of a \ac{PMIx} \ac{API}. Most \ac{PMIx} \ac{API} calls can accept additional information or attributes specified in the form of key/value pairs. These attributes provide information to the \ac{PMIx} implementation that influence the