Updated client for NiFI-2.6.0. Fixed augmentation system to correctly pass env vars. Updated contributing guide to explain common issues and how nifi and registry differ on security. Fixed mustache templates for new NiFi security schemes.Fixed make file passing through env vars to client generator (#391)

Author: Chaffelson

Makefile

@@ -2,7 +2,7 @@
 .DEFAULT_GOAL := help
 
 # Default NiFi/Registry version for docker compose profiles
-NIFI_VERSION ?= 2.5.0
+NIFI_VERSION ?= 2.6.0
 
 # Python command for cross-platform compatibility
 # Defaults to 'python' for conda/venv users, override with PYTHON=python3 for system installs
@@ -245,8 +245,8 @@ wait-ready: ## wait for readiness using profile configuration; requires NIPYAPI_
 # API & Client generation
 fetch-openapi-base: ## refresh base OpenAPI specs for current NIFI_VERSION (always overwrite base)
 	@echo "Refreshing base specs for NIFI_VERSION=$(NIFI_VERSION)"
-	bash resources/client_gen/fetch_nifi_openapi.sh nifi-single || exit 1
-	bash resources/client_gen/fetch_registry_openapi.sh registry-single || exit 1
+	NIFI_VERSION=$(NIFI_VERSION) bash resources/client_gen/fetch_nifi_openapi.sh nifi-single || exit 1
+	NIFI_VERSION=$(NIFI_VERSION) bash resources/client_gen/fetch_registry_openapi.sh registry-single || exit 1
 	@echo "Base specs refreshed."
 
 augment-openapi: ## generate augmented OpenAPI specs from base (always overwrite augmented)
@@ -263,8 +263,8 @@ augment-openapi: ## generate augmented OpenAPI specs from base (always overwrite
 
 fetch-openapi: fetch-openapi-base augment-openapi ## convenience: fetch base then augment
 
-gen-clients: ## generate NiFi and Registry clients from specs (use wv_spec_variant=augmented|base)
-	WV_SPEC_VARIANT=augmented bash resources/client_gen/generate_api_client.sh
+gen-clients: ## generate NiFi and Registry clients from specs (use wv_spec_variant=base|augmented)
+	wv_spec_variant=$${wv_spec_variant:-augmented} bash resources/client_gen/generate_api_client.sh
 
 # Individual testing
 

docs/contributing.rst

@@ -112,6 +112,178 @@ Ready to contribute? Here's how to set up `nipyapi` for local development.
 
 8. Submit a pull request through the GitHub website.
 
+Common Mistakes to Avoid
+------------------------
+
+When contributing to NiPyAPI, watch out for these frequent pitfalls:
+
+**Installation & Environment**
+
+* **Bracket quoting in zsh**: Running ``pip install -e .[dev]`` fails in zsh due to glob expansion. Always use quotes: ``pip install -e ".[dev]"`` or use ``make dev-install``.
+* **Wrong Python version**: Project supports Python 3.9-3.12. Using 3.8 (end of life) or 3.13+ (untested) may cause compatibility issues.
+* **Missing virtual environment**: Always activate a virtual environment before installing dependencies to avoid polluting system Python.
+
+**Testing**
+
+* **Missing NIPYAPI_PROFILE**: Never run ``pytest`` without setting ``NIPYAPI_PROFILE`` environment variable. Always use ``make test NIPYAPI_PROFILE=single-user`` or equivalent.
+* **Services not ready**: Never assume Docker services are immediately ready after ``make up``. Always run ``make wait-ready NIPYAPI_PROFILE=<profile>`` before testing.
+* **Stale certificates**: If you encounter certificate errors, run ``make down`` then ``make certs`` to regenerate fresh certificates. Never run ``make certs`` while containers are running.
+
+**Code Quality**
+
+* **Modifying generated code**: Never edit files in ``nipyapi/nifi/``, ``nipyapi/registry/``, or ``nipyapi/_version.py``. These are auto-generated and your changes will be overwritten.
+* **Skipping lint checks**: Always run ``make lint`` before committing. Both flake8 and pylint must pass.
+* **Incorrect line length**: Project uses 100-character line limit consistently across all tools (flake8, pylint, black, isort).
+
+**NiFi vs Registry Security Differences**
+
+**Important:** NiFi and Registry have different security implementations in their OpenAPI specifications:
+
+* **NiFi 2.6.0+**: Includes native security schemes (``HTTPBearerJWT``, ``CookieSecureAuthorizationBearer``) in base OpenAPI spec
+* **Registry 2.6.0+**: Has NO security schemes in base OpenAPI spec - requires augmentation
+
+**Key implications:**
+
+* Registry augmentation scripts (``resources/client_gen/augmentations/registry_security.py``) remain necessary even though NiFi 2.6.0 has native security
+* Both services use the same authentication flow: username/password → JWT token → Bearer auth
+* The template (``configuration.mustache``) handles both cases via hardcoded ``bearerAuth`` fallback plus native scheme aliases
+* Always use ``augmented`` variant when regenerating clients (default) to support both NiFi and Registry
+* NiFi can work without augmentation using the ``base`` variant (2.6.0+), but Registry cannot
+
+**Docker & Infrastructure**
+
+* **Docker volume caching**: If you experience persistent issues, run ``make clean-docker`` to remove all containers and volumes, then restart the setup process.
+* **Wrong profile for test**: Ensure your ``NIPYAPI_PROFILE`` matches the profile you started with ``make up``. Mixing profiles causes authentication failures.
+
+Make Targets Quick Reference
+-----------------------------
+
+NiPyAPI uses Makefile targets as the primary automation interface. Run ``make help`` to see all available targets organized by category.
+
+**Setup & Installation**
+::
+
+    make dev-install      # Install package with dev dependencies (recommended)
+    make docs-install     # Install documentation dependencies
+    make clean            # Remove build, pyc, and temp artifacts
+    make clean-all        # Nuclear clean: removes ALL including generated code
+
+**Testing Workflow**
+::
+
+    # Basic test workflow
+    make certs                              # Generate certificates (once)
+    make up NIPYAPI_PROFILE=single-user     # Start Docker services
+    make wait-ready NIPYAPI_PROFILE=single-user  # Wait for readiness
+    make test NIPYAPI_PROFILE=single-user   # Run tests
+    make down                               # Stop services
+
+    # Shortcuts for specific profiles
+    make test-su          # single-user profile
+    make test-ldap        # secure-ldap profile
+    make test-mtls        # secure-mtls profile
+
+    # Comprehensive testing
+    make test-all         # Run all automated profiles (single-user, ldap, mtls)
+    make coverage         # Run tests with coverage report
+
+**Code Quality**
+::
+
+    make lint             # Run flake8 + pylint (excludes generated code)
+    make flake8           # Run flake8 only
+    make pylint           # Run pylint only
+    make pre-commit       # Run pre-commit hooks on all files
+
+    # Formatting (manual)
+    black nipyapi/ && isort nipyapi/  # Auto-format code
+
+**Docker Operations**
+::
+
+    make certs            # Generate PKCS12 certificates for secure profiles
+    make up NIPYAPI_PROFILE=<profile>    # Start specific profile
+    make down             # Stop all Docker services
+    make wait-ready NIPYAPI_PROFILE=<profile>  # Wait for services to be ready
+    make clean-docker     # Comprehensive Docker cleanup
+
+    # Available profiles: single-user, secure-ldap, secure-mtls, secure-oidc
+
+**Build & Documentation**
+::
+
+    make dist             # Build wheel and source distribution
+    make check-dist       # Validate distribution files
+    make test-dist        # Test that distribution can be imported
+    make docs             # Generate Sphinx documentation
+
+**Complete Workflows**
+::
+
+    make sandbox NIPYAPI_PROFILE=single-user  # Create sandbox with sample objects
+    make rebuild-all      # Comprehensive rebuild: clean → certs → APIs → clients → test → build → docs
+
+Generated vs Maintained Code
+-----------------------------
+
+Understanding which code is generated vs maintained is crucial for contributing:
+
+**Generated Code (DO NOT MODIFY)**
+
+These files are automatically generated from OpenAPI specifications and should never be edited directly:
+
+* ``nipyapi/nifi/`` - NiFi API client
+* ``nipyapi/registry/`` - Registry API client
+* ``nipyapi/_version.py`` - Git version via setuptools-scm
+
+**Why this matters:**
+
+1. Your changes will be overwritten during the next client generation
+2. These files are excluded from linting (flake8, pylint, black, isort)
+3. Test coverage doesn't include generated code
+4. Pull requests should not modify these paths
+
+**Maintained Code (Where to Contribute)**
+
+Focus your contributions on these core modules:
+
+* ``nipyapi/canvas.py`` - Canvas management functions
+* ``nipyapi/config.py`` - Configuration and endpoints
+* ``nipyapi/parameters.py`` - Parameter context operations
+* ``nipyapi/profiles.py`` - Profile management system
+* ``nipyapi/security.py`` - Authentication and security
+* ``nipyapi/system.py`` - System-level operations
+* ``nipyapi/utils.py`` - Utility functions
+* ``nipyapi/versioning.py`` - Version control operations
+* ``tests/`` - Test suite (always add tests for new features)
+* ``examples/`` - Example scripts and usage patterns
+* ``docs/`` - Documentation (RST files)
+
+**Regenerating Clients**
+
+If you need to update the generated clients (e.g., for a new NiFi version):
+::
+
+    # Set target NiFi version
+    export NIFI_VERSION=2.5.0
+
+    # Fetch and generate
+    make fetch-openapi      # Fetch specs from running NiFi
+    make gen-clients        # Generate Python clients
+
+    # Test with new clients
+    make test-all
+
+**Augmentation System**
+
+The project includes an augmentation system for fixing OpenAPI spec issues:
+
+* Base specs: ``resources/client_gen/api_defs/nifi-<version>.json``
+* Augmentations: ``resources/client_gen/augmentations/*.py``
+* Augmented specs: ``resources/client_gen/api_defs/*-<version>.augmented.json``
+
+If you find spec issues, contribute fixes to the augmentation scripts rather than modifying generated code.
+
 Pull Request Guidelines
 -----------------------
 

nipyapi/nifi/__init__.py

@@ -3,7 +3,7 @@
 
     REST API definition for Apache NiFi web services
 
-    OpenAPI spec version: 2.5.0
+    OpenAPI spec version: 2.6.0
     Contact: [email protected]
     Generated by: https://github.com/swagger-api/swagger-codegen.git
 """

nipyapi/nifi/api_client.py

@@ -3,7 +3,7 @@
 
     REST API definition for Apache NiFi web services
 
-    OpenAPI spec version: 2.5.0
+    OpenAPI spec version: 2.6.0
     Contact: [email protected]
     Generated by: https://github.com/swagger-api/swagger-codegen.git
 """

nipyapi/nifi/apis/access_api.py

@@ -3,7 +3,7 @@
 
     REST API definition for Apache NiFi web services
 
-    OpenAPI spec version: 2.5.0
+    OpenAPI spec version: 2.6.0
     Contact: [email protected]
     Generated by: https://github.com/swagger-api/swagger-codegen.git
 """

nipyapi/nifi/apis/authentication_api.py

@@ -3,7 +3,7 @@
 
     REST API definition for Apache NiFi web services
 
-    OpenAPI spec version: 2.5.0
+    OpenAPI spec version: 2.6.0
     Contact: [email protected]
     Generated by: https://github.com/swagger-api/swagger-codegen.git
 """
@@ -98,7 +98,7 @@ def get_authentication_configuration_with_http_info(self, **kwargs):
             select_header_accept(['application/json'])
 
         # Authentication setting
-        auth_settings = ['bearerAuth']
+        auth_settings = []
 
         return self.api_client.call_api('/authentication/configuration', 'GET',
                                         path_params,

nipyapi/nifi/apis/connections_api.py

@@ -3,7 +3,7 @@
 
     REST API definition for Apache NiFi web services
 
-    OpenAPI spec version: 2.5.0
+    OpenAPI spec version: 2.6.0
     Contact: [email protected]
     Generated by: https://github.com/swagger-api/swagger-codegen.git
 """

nipyapi/nifi/apis/controller_api.py

@@ -3,7 +3,7 @@
 
     REST API definition for Apache NiFi web services
 
-    OpenAPI spec version: 2.5.0
+    OpenAPI spec version: 2.6.0
     Contact: [email protected]
     Generated by: https://github.com/swagger-api/swagger-codegen.git
 """
@@ -149,6 +149,8 @@ def clear_state(self, id, **kwargs):
         Args:
             id (str): 
                 The flow analysis rule id. (required)
+            body (:class:`~nipyapi.nifi.models.ComponentStateEntity`): 
+                Optional component state to perform a selective key removal. If omitted, clears all state.
                 
         Returns:
             :class:`~nipyapi.nifi.models.ComponentStateEntity`: The response data.
@@ -171,12 +173,14 @@ def clear_state_with_http_info(self, id, **kwargs):
         Args:
             id (str): 
                 The flow analysis rule id. (required)
+            body (:class:`~nipyapi.nifi.models.ComponentStateEntity`): 
+                Optional component state to perform a selective key removal. If omitted, clears all state.
                 
         Returns:
             tuple: (:class:`~nipyapi.nifi.models.ComponentStateEntity`, status_code, headers) - Response data with HTTP details.
         """
 
-        all_params = ['id']
+        all_params = ['id', 'body']
         all_params.append('_return_http_data_only')
         all_params.append('_preload_content')
         all_params.append('_request_timeout')
@@ -195,6 +199,7 @@ def clear_state_with_http_info(self, id, **kwargs):
             raise ValueError("Missing the required parameter `id` when calling `clear_state`")
 
 
+ 
         collection_formats = {}
 
         path_params = {}
@@ -209,10 +214,16 @@ def clear_state_with_http_info(self, id, **kwargs):
         local_var_files = {}
 
         body_params = None
+        if 'body' in params:
+            body_params = params['body']
         # HTTP header `Accept`
         header_params['Accept'] = self.api_client.\
             select_header_accept(['application/json'])
 
+        # HTTP header `Content-Type`
+        header_params['Content-Type'] = self.api_client.\
+            select_header_content_type(['*/*', 'application/json'])
+
         # Authentication setting
         auth_settings = ['bearerAuth']
 

nipyapi/nifi/apis/controller_services_api.py

@@ -3,7 +3,7 @@
 
     REST API definition for Apache NiFi web services
 
-    OpenAPI spec version: 2.5.0
+    OpenAPI spec version: 2.6.0
     Contact: [email protected]
     Generated by: https://github.com/swagger-api/swagger-codegen.git
 """
@@ -149,6 +149,8 @@ def clear_state1(self, id, **kwargs):
         Args:
             id (str): 
                 The controller service id. (required)
+            body (:class:`~nipyapi.nifi.models.ComponentStateEntity`): 
+                Optional component state to perform a selective key removal. If omitted, clears all state.
                 
         Returns:
             :class:`~nipyapi.nifi.models.ComponentStateEntity`: The response data.
@@ -171,12 +173,14 @@ def clear_state1_with_http_info(self, id, **kwargs):
         Args:
             id (str): 
                 The controller service id. (required)
+            body (:class:`~nipyapi.nifi.models.ComponentStateEntity`): 
+                Optional component state to perform a selective key removal. If omitted, clears all state.
                 
         Returns:
             tuple: (:class:`~nipyapi.nifi.models.ComponentStateEntity`, status_code, headers) - Response data with HTTP details.
         """
 
-        all_params = ['id']
+        all_params = ['id', 'body']
         all_params.append('_return_http_data_only')
         all_params.append('_preload_content')
         all_params.append('_request_timeout')
@@ -195,6 +199,7 @@ def clear_state1_with_http_info(self, id, **kwargs):
             raise ValueError("Missing the required parameter `id` when calling `clear_state1`")
 
 
+ 
         collection_formats = {}
 
         path_params = {}
@@ -209,10 +214,16 @@ def clear_state1_with_http_info(self, id, **kwargs):
         local_var_files = {}
 
         body_params = None
+        if 'body' in params:
+            body_params = params['body']
         # HTTP header `Accept`
         header_params['Accept'] = self.api_client.\
             select_header_accept(['application/json'])
 
+        # HTTP header `Content-Type`
+        header_params['Content-Type'] = self.api_client.\
+            select_header_content_type(['*/*', 'application/json'])
+
         # Authentication setting
         auth_settings = ['bearerAuth']