PMIx v4 Planning

[rhc54]

Coordinated with pmix/pmix-standard#189

Reference implementation extensions:

• Attribute reporting support

• PMIx Groups support

• Add support for new scheduler interfaces
  - [x] Fabric

• Add network coordinate and endpoint computation

• pquery tool for user information requests

• pattrs tool for user requests of attribute support

• Python bindings

• Complete tool support

• Update and release PMIx v4 Standard

[artpol84]

Should we add #1241 to the agenda?

[rhc54]

We can strive for it - I wouldn't necessarily hold up the release but it would indeed be good to get it done. I've also asked for some clarification of the dstore code to ensure others beside Boris and yourself can support it. I'm sure we can identify other areas (e.g., event notification) that would benefit from a second/third pass.

[artpol84]

Regarding dstore - when I talked to @jjhursey he said he thinks that codebase is descriptive enough. @jjhursey, please correct me if I am wrong.
@rhc54, we would be glad to address particular concerns from your side.

[jjhursey]

I was chatting with some folks about doing some "code debrief" sessions targeted for PMIx RI developers. There are sections of the PMIx codebase well understood by a small set of folks which inhibits the community's ability to quickly turn around bug fixes and PR reviews. Some sections of code are needfully complex due to the nature of the behavior they are trying to support, but this makes it difficult to understand the impact of small changes to that functionality. As such I want to organize some (publicly available) recorded webex sessions with the developer(s) of those areas talking through the code design and structure - pointing out along the way subtleties in the code to watch out for.

The hope is that these sessions are fairly lightweight to prepare for, and we have questions to guide the discussion. They are point-in-time developer documentation, but I think the codebase is stable enough in some areas that they would be useful for a while. And this could be an opportunity to highlight code sections that need improvement or clarification.

I think the dstore would be a good candidate for this, as would be the event mechanism, and the cross-version handshake. These are critical code areas in the PRI.

If folks think this is a good idea I'll open up a ticket to poll for topic ideas and try to organize some sessions this summer.

[artpol84]

I don't mind doing that and I would be happy to do a session about the dstore.

Note, however, that #1241 is not about understanding the code - it is about refactoring it to bring it to a production state reducing the probability of errors and testing error code paths.

I'm not having issues understanding any piece of PMIx RI code, though I have issues understanding some high-level ideas behind this code. The near-term example is the variety of connection entities in PTL (in v2.2, v3.1, and master):

• Client
• Tool
• launcher
• Client + Tool
• Client + Tool + Launcher
• Tool + Launcher
• ? Client + Launcher ?

I think what would also be interesting is to have sessions describing those entities. And maybe there are already some pieces of information in PMIx slides page. I have not checked there yet, to be honest.

[rhc54]

Client
Tool
launcher
Client + Tool
Client + Tool + Launcher
Tool + Launcher
? Client + Launcher ?

Those process types are all being defined in the tools chapter, hopefully completed late this summer or early fall.

Some sections of code are needfully complex due to the nature of the behavior they are trying to support, but this makes it difficult to understand the impact of small changes to that functionality.

I think the point here is a good one. The areas identified (dstore, event, and ptl) are all complex and understood by only 1-2 people. Thus, they are all somewhat "fragile" in terms of errors and testing of error paths, and at-risk for long term maintenance. I suspect we'll identify a few more as we do a "maturity" sweep thru the code base, but these will serve as a good starting point.

[jjhursey]

Ref pmix_coord_t was merged: #1349

[rhc54]

REMOVED

• New Cray DRC psec component for obtaining RDMA credentials
  - Delayed due to Cray s/w delays - rescheduled for later release

[rhc54]

REMOVED

• New GDS component for DAOS support, extensions to support setup_application and setup_local_support
  - Deferred to later release

[rhc54]

REMOVED

• Add support for storage directives
  - Deferred to a later release