"ValueError: Unable to parse claims from response" on get()

[cjusko]

Hello!

Trying to make any .get() call from my GraphServiceClient results in the following error:

ValueError                                
Traceback (most recent call last)

      1 # GET /users?$top=500
      2 #users = await client.users.by_user_id('cjusko').get()
----> 3 users = await client.users.get()
      4 print(users.value)

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/that/lib/python3.11/site-packages/msgraph/generated/users/users_request_builder.py:69, in UsersRequestBuilder.get(self, request_configuration)
     66     raise Exception("Http core is null") 
     67 from ..models.user_collection_response import UserCollectionResponse
---> 69 return await self.request_adapter.send_async(request_info, UserCollectionResponse, error_mapping)

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/that/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:178, in HttpxRequestAdapter.send_async(self, request_info, parsable_factory, error_map)
    175     parent_span.record_exception(REQUEST_IS_NULL)
    176     raise REQUEST_IS_NULL
--> 178 response = await self.get_http_response_message(request_info, parent_span)
    180 response_handler = self.get_response_handler(request_info)
    181 if response_handler:

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/that/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:543, in HttpxRequestAdapter.get_http_response_message(self, request_info, parent_span, claims)
    541     parent_span.set_attribute("http.response_content_type", content_type)
    542 _get_http_resp_span.end()
--> 543 return await self.retry_cae_response_if_required(resp, request_info, claims)

File ~/.local/share/hatch/env/virtual/that/hYHKRVU-/hat/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:560, in HttpxRequestAdapter.retry_cae_response_if_required(self, resp, request_info, claims)
    558 claims_match = re.search('claims="(.+)"', auth_header_value)
    559 if not claims_match:
--> 560     raise ValueError("Unable to parse claims from response")
    561 response_claims = claims_match.group().split('="')[1]
    562 parent_span.add_event(AUTHENTICATE_CHALLENGED_EVENT_KEY)

ValueError: Unable to parse claims from response

Some context:
This is in virtalenv running
python==3.11.6
and
msgraph-sdk==1.2.0 (but I received the same error on version 1.3.0 as well, tried downgrading to see if it resolved, it did not)

the code I'm running is fairly simple:

from azure.identity import AzureAuthorityHosts
from azure.identity.aio import ClientSecretCredential
from msgraph import GraphServiceClient

auth = {
 # DICT holding 'tenant ID',  'client ID', 'client secret key'
}

credential = ClientSecretCredential(
    auth['tenant'],
    auth['clientId'],
    auth['clientSecret'],
    authority=AzureAuthorityHosts.AZURE_GOVERNMENT
)

scopes = ['https://graph.microsoft.us/.default']
client = GraphServiceClient(credentials=credential, scopes=scopes)

users = await client.users.get() # <--- Error on this line

I am running this in a jupyter notebook, but running in IPython leads to same result. Same result when running in a script and calling like this as well:

async def get_user():
    users = await client.users.get() # <--- Error on this line
    # ...

asyncio.run(get_user())

I'm guessing it's likely an issue with my client setup since I can't seem to find anyone else having this issue. But not sure how else to set it up, I can confirm that my tenant and client information is correct. Maybe it's an issue in how I have it set up to hit AZURE_GOVERNMENT?

Thanks in advance!

[shemogumbe]

Hello, to be able to understand the issue,
I see you are trying to get a list of users, there is a possibility that:

• the way the client is instantianted, and the application permissions on your tenant disallow this request.
• The auth lacks a www-authenticate or token, which could still point to how the client is initialised.

My suggestions

• Do you mind trying this on the public cloud.
• Try counter check with your tenant for the correct tenant_id, client_id and client_secret for use in the credential creation.

Note
Kindly counter check the equivalent scope for get users in the government cloud, trying to reproduce your issue got me the error unable to process claims response when using the scope ``

but worked fine with default authority, and default scope in public cloud:

No error here:

redential = ClientSecretCredential(
    auth['tenant_id'],
    client_id=auth['clientId'],
    client_secret=auth['clientSecret'],
    authority=os.getenv('authority'),
   )
)
scopes = ['https://graph.microsoft.com/.default']
client = GraphServiceClient(credentials=credential, scopes=scopes)
print(client.request_adapter)


async def get_users():
    try:
        users = await client.users.get()  # <--- Error on this line
        for user in users.value:
            print(f"User: {user.display_name} {user.id}")
    except APIError as e:
        print(f"Error: {e}")

Having only replaced scopes = ['https://graph.microsoft.us/.default'] with scopes = ['https://graph.microsoft.com/.default']
and authority=AzureAuthorityHosts.AZURE_GOVERNMENT with authority='https://login.microsoftonline.com/common'

[cjusko]

making these changes gives back a different error:
ClientSecretCredential.get_token failed: argument of type 'NoneType' is not iterable
With this code:

credential = ClientSecretCredential(
    tenant_id=auth['tenant'],
    client_id=auth['clientId'],
    client_secret=auth['clientSecret'],
    authority='https://login.microsoftonline.com/common' 
)

scopes = ['https://graph.microsoft.com/.default']
client = GraphServiceClient(credentials=credential, scopes=scopes)

async def get_me():
    try:
        me = await client.me.get()  # <--- Error on this line
        print(me.value)
    except APIError as e:
        print(f"Error: {e}")

Here's the full traceback:

ClientSecretCredential.get_token failed: argument of type 'NoneType' is not iterable
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[26], line 18
     15 client = GraphServiceClient(credentials=credential, scopes=scopes)
     17 
---> 18 me = await client.me.get()
     19 print(me.value)

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/msgraph/generated/users/item/user_item_request_builder.py:156, in UserItemRequestBuilder.get(self, request_configuration)
    153     raise Exception("Http core is null") 
    154 from ...models.user import User
--> 156 return await self.request_adapter.send_async(request_info, User, error_mapping)

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:178, in HttpxRequestAdapter.send_async(self, request_info, parsable_factory, error_map)
    175     parent_span.record_exception(REQUEST_IS_NULL)
    176     raise REQUEST_IS_NULL
--> 178 response = await self.get_http_response_message(request_info, parent_span)
    180 response_handler = self.get_response_handler(request_info)
    181 if response_handler:

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:523, in HttpxRequestAdapter.get_http_response_message(self, request_info, parent_span, claims)
    520 if claims:
    521     additional_authentication_context[self.CLAIMS_KEY] = claims
--> 523 await self._authentication_provider.authenticate_request(
    524     request_info, additional_authentication_context
    525 )
    527 request = self.get_request_from_request_information(
    528     request_info, _get_http_resp_span, parent_span
    529 )
    530 resp = await self._http_client.send(request)

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/kiota_abstractions/authentication/base_bearer_token_authentication_provider.py:50, in BaseBearerTokenAuthenticationProvider.authenticate_request(self, request, additional_authentication_context)
     47     request.headers = HeadersCollection()
     49 if not request.headers.contains(self.AUTHORIZATION_HEADER):
---> 50     token = await self.access_token_provider.get_authorization_token(
     51         request.url, additional_authentication_context
     52     )
     53     if token:
     54         request.headers.add(f'{self.AUTHORIZATION_HEADER}', f'Bearer {token}')

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/kiota_authentication_azure/azure_identity_access_token_provider.py:106, in AzureIdentityAccessTokenProvider.get_authorization_token(self, uri, additional_authentication_context)
    103     result = self._credentials.get_token(*self._scopes, claims=decoded_claim)
    105 if inspect.isawaitable(result):
--> 106     result = await result
    107     await self._credentials.close()  # type: ignore
    109 if result and isinstance(result, AccessToken):

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/aio/_internal/get_token_mixin.py:93, in GetTokenMixin.get_token(self, claims, tenant_id, enable_cae, *scopes, **kwargs)
     91 if not token:
     92     self._last_request_time = int(time.time())
---> 93     token = await self._request_token(
     94         *scopes, claims=claims, tenant_id=tenant_id, enable_cae=enable_cae, **kwargs
     95     )
     96 elif self._should_refresh(token):
     97     try:

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/aio/_credentials/client_secret.py:67, in ClientSecretCredential._request_token(self, *scopes, **kwargs)
     66 async def _request_token(self, *scopes: str, **kwargs: Any) -> AccessToken:
---> 67     return await self._client.obtain_token_by_client_secret(scopes, self._secret, **kwargs)

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/aio/_internal/aad_client.py:48, in AadClient.obtain_token_by_client_secret(self, scopes, secret, **kwargs)
     46 async def obtain_token_by_client_secret(self, scopes: Iterable[str], secret: str, **kwargs) -> AccessToken:
     47     request = self._get_client_secret_request(scopes, secret, **kwargs)
---> 48     return await self._run_pipeline(request, **kwargs)

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/aio/_internal/aad_client.py:85, in AadClient._run_pipeline(self, request, **kwargs)
     83 now = int(time.time())
     84 response = await self._pipeline.run(request, retry_on_methods=self._POST, **kwargs)
---> 85 return self._process_response(response, now, enable_cae=enable_cae, **kwargs)

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/_internal/aad_client_base.py:161, in AadClientBase._process_response(self, response, request_time, **kwargs)
    158             cache.update_rt(cache_entries[0], content["refresh_token"])
    159             del content["refresh_token"]  # prevent caching a redundant entry
--> 161 _raise_for_error(response, content)
    163 if "expires_on" in content:
    164     expires_on = int(content["expires_on"])

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/_internal/aad_client_base.py:390, in _raise_for_error(response, content)
    389 def _raise_for_error(response: PipelineResponse, content: Dict) -> None:
--> 390     if "error" not in content:
    391         return
    393     _scrub_secrets(content)

TypeError: argument of type 'NoneType' is not iterable

[deanevs]

I have the exact same issue, the adapter always get's set to the host: graph.microsoft.com even though the authority is set up with a token for US_GOV??

Setup ->

        self.credential = ClientSecretCredential(
            client_id=config.client_id,
            tenant_id=config.tenant_id,
            client_secret=config.client_secret,
            authority=AzureAuthorityHosts.AZURE_GOVERNMENT
        )

        self.scopes = 'https://graph.microsoft.us/.default'
        allowed_hosts = [NationalClouds.US_GOV]

        auth_provider = AzureIdentityAuthenticationProvider(self.credential,
                                                            scopes=[self.scopes],
                                                            allowed_hosts=allowed_hosts)

        base_middleware = GraphClientFactory.get_default_middleware(options=None)
        base_middleware.append(DebugHandler())

        http_client = GraphClientFactory.create_with_custom_middleware(
            middleware=base_middleware,
            host=NationalClouds.US_GOV,
        )

        adapter = GraphRequestAdapter(auth_provider=auth_provider, client=http_client)
        self.client = GraphServiceClient(request_adapter=adapter)

Debug ->

GET https://graph.microsoft.com/v1.0/users?$orderby=displayName&$select=id,displayName,mail,usageLocation,givenName,surname
host: graph.microsoft.com
accept-encoding: gzip, deflate
connection: keep-alive
consistencylevel: eventual
accept: application/json
authorization: Bearer eyJ0eXAiOiJKV1QiLCJub25jZSI6Ijc4ZRWRkZXph....
user-agent: python-httpx/0.27.0 kiota-python/1.3.1
...
...
Response: 401 Unauthorized
Response headers:
content-type: application/json
content-encoding: gzip
vary: Accept-Encoding
strict-transport-security: max-age=31536000
request-id: 4747f7ff-7ad3-4dbc-b2cf-9e3fd6a08c30
client-request-id: 4747f7ff-7ad3-4dbc-b2cf-9e3fd6a08c30
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"UK South","Slice":"E","Ring":"5","ScaleUnit":"001","RoleInstance":"LN2PEPF00010E3B"}}
www-authenticate: Bearer realm="", authorization_uri="https://login.microsoftonline.com/common/oauth2/authorize", client_id="00000003-0000-0000-c000-000000000000"
date: Wed, 01 May 2024 09:33:06 GMT

Error ->

  File "\kiota_http\httpx_request_adapter.py", line 543, in get_http_response_message
    return await self.retry_cae_response_if_required(resp, request_info, claims)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\kiota_http\httpx_request_adapter.py", line 560, in retry_cae_response_if_required
    raise ValueError("Unable to parse claims from response")
ValueError: Unable to parse claims from response

[deanevs]

Worked it out, you need to set the adapter base_url before passing it to the GraphServiceClient

        adapter = GraphRequestAdapter(auth_provider=auth_provider, client=http_client)
        adapter.base_url = 'https://graph.microsoft.us/v1.0'
        self.client = GraphServiceClient(request_adapter=adapter, scopes=self.scopes)

[shemogumbe]

Moving these two discussions as it may help someone facing a similar issue