FP8 KV cache doesn't work with prefix caching #3156
Description
FP8 KV cache doesn't work with prefix caching since the Triton kernel does not support KV8 for now.
{{ message }}
FP8 KV cache doesn't work with prefix caching since the Triton kernel does not support KV8 for now.