[Bug]: Many endpoints are returning 500 Internal Server Error

[tarukumar]

Your current environment

missed to capture

🐛 Describe the bug

During property-based testing of the vLLM API, defined by an OpenAPI 3.1 schema, we observed a recurring issue Server Errors across several endpoints.

1. POST /tokenize:
Issue: Server error (500 Internal Server Error) and undocumented HTTP status code

Details: The server responded with a 500 Internal Server Error, but the OpenAPI schema only documents 200 and 422 as possible responses. This suggests that the server might have an issue handling the request.


2. POST /detokenize:
Issue: Server error (500 Internal Server Error) and undocumented HTTP status code

Details: Similar to the previous failure, the server is returning a 500 Internal Server Error, while the OpenAPI schema only expects 200 or 422.


3. POST /v1/chat/completions:
Issue 1: Server error (500 Internal Server Error)

Details: The server returned a 500 error instead of the documented 200 or 422.



4. POST /v1/completions:

Issue 1: Server error (500 Internal Server Error)

Details: The server returned a 500 Internal Server Error instead of the expected 200 or 422.


5. POST /invocations:
Issue: Server error (500 Internal Server Error) and undocumented HTTP status code

Details: The server returned a 500 Internal Server Error instead of the expected 200. The OpenAPI schema only documents a 200 response for /invocations.

Logs:

test_new.py u,uuuu,,uu,,,,u,,,u.                                                                                                                                                                                                       [100%]

================================================================================================================== FAILURES ==================================================================================================================
_______________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='GET /health') ________________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Missing Content-Type header
E       
E           The following media types are documented in the schema:
E           - `application/json`
E       
E       [200] OK:
E       
E           <EMPTY>
E       
E       Reproduce with: 
E       
E           curl -X GET -H 'Content-Type: application/json' --insecure http://127.0.0.1:8080/health
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

________________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='GET /ping') _________________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Missing Content-Type header
E       
E           The following media types are documented in the schema:
E           - `application/json`
E       
E       [200] OK:
E       
E           <EMPTY>
E       
E       Reproduce with: 
E       
E           curl -X GET -H 'Content-Type: application/json' --insecure http://127.0.0.1:8080/ping
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

________________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /ping') ________________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Missing Content-Type header
E       
E           The following media types are documented in the schema:
E           - `application/json`
E       
E       [200] OK:
E       
E           <EMPTY>
E       
E       Reproduce with: 
E       
E           curl -X POST -H 'Content-Type: application/json' --insecure http://127.0.0.1:8080/ping
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

______________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /tokenize') ______________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'}, body={'messages': []})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Server error
E       
E       2. Undocumented HTTP status code
E       
E           Received: 500
E           Documented: 200, 422
E       
E       [500] Internal Server Error:
E       
E           `Internal Server Error`
E       
E       Reproduce with: 
E       
E           curl -X POST -H 'Content-Type: application/json' -d '{"messages": []}' --insecure http://127.0.0.1:8080/tokenize
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

_____________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /detokenize') _____________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'}, body={'tokens': [-1]})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Server error
E       
E       2. Undocumented HTTP status code
E       
E           Received: 500
E           Documented: 200, 422
E       
E       [500] Internal Server Error:
E       
E           `Internal Server Error`
E       
E       Reproduce with: 
E       
E           curl -X POST -H 'Content-Type: application/json' -d '{"tokens": [-1]}' --insecure http://127.0.0.1:8080/detokenize
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /v1/chat/completions') _________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:
E   schemathesis.exceptions.CheckFailed: Schemathesis found 2 distinct sets of failures.
E   ==================== 
E   
E   1. Server error
E   
E   2. Undocumented HTTP status code
E   
E       Received: 500
E       Documented: 200, 422
E   
E   [500] Internal Server Error:
E   
E       `Internal Server Error`
E   
E   Reproduce with: 
E   
E       curl -X POST -H 'Content-Type: application/json' -d '{"messages": []}' --insecure http://127.0.0.1:8080/v1/chat/completions
E   
E   ==================== 
E   
E   1. Undocumented HTTP status code
E   
E       Received: 400
E       Documented: 200, 422
E   
E   [400] Bad Request:
E   
E       `{"object":"error","message":"list object has no element 0","type":"BadRequestError","param":null,"code":400}`
E   
E   Reproduce with: 
E   
E       curl -X POST -H 'Content-Type: application/json' -d '{"messages": [{"role": "assistant"}]}' --insecure http://127.0.0.1:8080/v1/chat/completions

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

___________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /v1/completions') ___________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:
E   schemathesis.exceptions.CheckFailed: Schemathesis found 2 distinct sets of failures.
E   ==================== 
E   
E   1. Undocumented HTTP status code
E   
E       Received: 400
E       Documented: 200, 422
E   
E   [400] Bad Request:
E   
E       `{"object":"error","message":"please provide at least one prompt","type":"BadRequestError","param":null,"code":400}`
E   
E   Reproduce with: 
E   
E       curl -X POST -H 'Content-Type: application/json' -d '{"prompt": []}' --insecure http://127.0.0.1:8080/v1/completions
E   
E   ==================== 
E   
E   1. Server error
E   
E   2. Undocumented HTTP status code
E   
E       Received: 500
E       Documented: 200, 422
E   
E   [500] Internal Server Error:
E   
E       `Internal Server Error`
E   
E   Reproduce with: 
E   
E       curl -X POST -H 'Content-Type: application/json' -d '{"prompt": [-1]}' --insecure http://127.0.0.1:8080/v1/completions

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

______________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /v1/audio/transcriptions') _______________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'}, body={'file': Binary(data=b'')})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Undocumented HTTP status code
E       
E           Received: 400
E           Documented: 200, 422
E       
E       [400] Bad Request:
E       
E           `{"object":"error","message":"[{'type': 'value_error', 'loc': ('body', 'file'), 'msg': \"Value error, Expected UploadFile, received: <class 'str'>\", 'input': \"b''\", 'ctx': {'error': ValueError(\"Expected UploadFile, received: <class 'str'>\")}}]","type":"BadRequestError","param":null,"code":400}`
E       
E       Reproduce with: 
E       
E           curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' -d file=b%27%27 --insecure http://127.0.0.1:8080/v1/audio/transcriptions
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

____________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /invocations') _____________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Server error
E       
E       2. Undocumented HTTP status code
E       
E           Received: 500
E           Documented: 200
E       
E       [500] Internal Server Error:
E       
E           `Internal Server Error`
E       
E       Reproduce with: 
E       
E           curl -X POST -H 'Content-Type: application/json' --insecure http://127.0.0.1:8080/invocations
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

============================================================================================================== warnings summary ==============================================================================================================
dsd/lib64/python3.12/site-packages/schemathesis/generation/coverage.py:305
  /home/takumar/Downloads/dsd/lib64/python3.12/site-packages/schemathesis/generation/coverage.py:305: DeprecationWarning: jsonschema.exceptions.RefResolutionError is deprecated as of version 4.18.0. If you wish to catch potential reference resolution errors, directly catch referencing.exceptions.Unresolvable.
    ref_error: type[Exception] = jsonschema.RefResolutionError,

dsd/lib64/python3.12/site-packages/schemathesis/internal/deprecation.py:6
  /home/takumar/Downloads/dsd/lib64/python3.12/site-packages/schemathesis/internal/deprecation.py:6: DeprecationWarning: Argument `method` is deprecated and will be removed in Schemathesis 4.0. Use `include` and `exclude` methods instead.
    warnings.warn(

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[tarukumar]

Investigate server-side issues (500 errors) and review error-handling logic to ensure it aligns with the documented schema.

[tunglinwood]

I reproduce with your steps and get the same 500 Internal Server Error using vllm serve. In the backend I got following error,

I will try to fix this later today. @DarkLight1337 can you assign this issue to me?

INFO:     172.18.0.1:45124 - "POST /tokenize HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 409                                                                                                           , in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/uvicorn/middleware/proxy_headers.py", line 60, in _                                                                                                           _call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.12/dist-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/errors.py", line 187, in __cal                                                                                                           l__
    raise exc
  File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/errors.py", line 165, in __cal                                                                                                           l__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/exceptions.py", line 62, in __                                                                                                           call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 53, in wrapp                                                                                                           ed_app
    raise exc
  File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 42, in wrapp                                                                                                           ed_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 714, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 734, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 53, in wrapp                                                                                                           ed_app
    raise exc
  File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 42, in wrapp                                                                                                           ed_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 73, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/fastapi/routing.py", line 212, in run_endpoint_func                                                                                                           tion
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/utils.py", line 58, in wrapper
    return handler_task.result()
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 410, i                                                                                                           n tokenize
    generator = await handler.create_tokenize(request, raw_request)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/serving_tokenization.py", l                                                                                                           ine 72, in create_tokenize
    ) = await self._preprocess_chat(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/serving_engine.py", line 41                                                                                                           5, in _preprocess_chat
    request_prompt = apply_hf_chat_template(
                     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/chat_utils.py", line 1156, in appl                                                                                                           y_hf_chat_template
    return tokenizer.apply_chat_template(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/transformers/tokenization_utils_base.py", line 1640                                                                                                           , in apply_chat_template
    isinstance(conversation[0], (list, tuple)) or hasattr(conversation[0], "messages")
               ~~~~~~~~~~~~^^^
IndexError: list index out of range

[tunglinwood]

Hi, @DarkLight1337 I have question, please kindly check,

When I am modifying vllm/entrypoints/openai/serving_completion.py class OpenAIServingCompletion, I notice there are serveral TODO left here, e.g.

        except ValueError as e:
            # TODO: Use a vllm-specific Validation Error
            return self.create_error_response(str(e))

Could you kindly suggest that if there is a defined class for handling validation Error to be introduced here. If not, I can try to create and submit another PR.

Thank you.

[DarkLight1337]

No, we don't have this yet

[tunglinwood]

@DarkLight1337 Could you kindly check if this issue and #17037 are identical issue?

[tunglinwood]

@DarkLight1337 Could you kindly check if this issue and #17037 are identical issue?

And #17036

[DarkLight1337]

I think it is more related to #17037. You can try running the property-based tests and see if the failures still persist after your fix

[tunglinwood]

I think it is more related to #17037. You can try running the property-based tests and see if the failures still persist after your fix

Okay involving @davidxia and @KubeKyrie since it might be impacting their progress.

[davidxia]

@tunglinwood I drafted #17664 that might handle many of the 500 errors.

@DarkLight1337 are there docs on how to run the property-based tests?