Character encoding weird effects with letter case sensitivity in ContentType header #150
Description
- Framework version: 1.1
- Implementations: all ?
Let's try to explain the problem with 3 examples :
Raw problem
Problem 1 : "Is this setCharacterEncoding effect valid ?"
// GIVEN a plain request
AwsProxyRequest request = new AwsProxyRequest();
AwsProxyHttpServletRequest httpRequest = new AwsProxyHttpServletRequest(request, null, null);
// WHEN setting the character encoding to UTF-8
httpRequest.setCharacterEncoding("UTF-8");
// THEN I get "; charset=UTF-8", which is not a valid property according to Jersey
String actualCharacterEncoding = httpRequest.getCharacterEncoding();
System.out.println(actualCharacterEncoding); // "UTF-8" -> ok
String contentType = httpRequest.getHeader("Content-Type");
System.out.println(contentType); // /!\ "; charset=UTF-8"
It seems to me that "; charset=UTF-8" is not a valid Content-Type...
Problem 2 : "Double raw media type when forcing the setCharacterEncoding"
// GIVEN a request with "Content-Type" equals to "application/json; charset=utf-8"
AwsProxyRequest request = new AwsProxyRequest();
request.getHeaders().put("Content-Type", "application/json; charset=utf-8");
AwsProxyHttpServletRequest httpRequest = new AwsProxyHttpServletRequest(request, null, null);
// WHEN setting the character encoding to UTF-8
httpRequest.setCharacterEncoding("UTF-8");
// THEN I Get "application/json; application/json; charset=UTF-8", which is not what I expected
String contentType = httpRequest.getHeader("Content-Type");
System.out.println(contentType); // /!\ "application/json; application/json; charset=UTF-8" instead of "application/json; charset=UTF-8"
Problem 3 : "Letter case in content-type header changes the behavior"
// GIVEN a request with "content-type" IN LOWER CASE equals to "application/json; charset=utf-8"
AwsProxyRequest request = new AwsProxyRequest();
String contentTypeHeaderLowerCase = "content-type";
request.getHeaders().put(contentTypeHeaderLowerCase, "application/json");
AwsProxyHttpServletRequest httpRequest = new AwsProxyHttpServletRequest(request, null, null);
// WHEN setting the character encoding to UTF-8
httpRequest.setCharacterEncoding("UTF-8");
// THEN I Get "application/json", which is not what I expected
final String contentTypeFromHeaderLowerCase = httpRequest.getHeader(contentTypeHeaderLowerCase);
System.out.println(contentTypeFromHeaderLowerCase); // /!\ "application/json" instead of "application/json; charset=UTF8"
String actualCharacterEncoding = httpRequest.getCharacterEncoding();
System.out.println(actualCharacterEncoding); // /!\ null instead of "UTF-8"
The same test has a different (and correct) behavior with the header spelled "Content-Type" : "application/json; charset=UTF-8" and actualCharacterEncoding = UTF-8.
Context
One could ask how this problem emerged 😃
I am trying to use a SpringBoot REST application based on Jersey (i.e not a pure Jersey app, and not a Spring MVC app neither). I made a custom ContainerHandler inspired from the SpringBootLambdaContainerHandler to achieve this goal (the latter only deals with the 'dispatcherServlet', and I need to deal with the Jersey one from the Jersey starter).
While testing it locally, I figured out the problem. A Spring filter (OrderedCharacterEncodingFilter) sets the characterEncoding to UTF-8, no matter what the content-type header value is (default behaviour).
if (isForceRequestEncoding() || request.getCharacterEncoding() == null) {
request.setCharacterEncoding(encoding);
}
Later in the request processing, Jersey tries to extract the MediaType and fails (class : InboundMessageContext)
public MediaType getMediaType() {
return singleHeader(HttpHeaders.CONTENT_TYPE, new Function<String, MediaType>() {
@Override
public MediaType apply(String input) {
try {
return MediaType.valueOf(input);
} catch (IllegalArgumentException iae) {
throw new ProcessingException(iae);
}
}
}, false);
}
This leads to the following error when the header value is incorrect :
- Problem 1 : Error parsing media type '; charset=UTF-8' at org.glassfish.jersey.message.internal.MediaTypeProvider.fromString
- Problem 2 : Error parsing media type 'application/json; application/json; charset=UTF-8' at org.glassfish.jersey.message.internal.MediaTypeProvider.fromString
- Problem 3 : Too many "Content-Type" header values: "[application/json; charset=utf-8, application/json; charset=utf-8]" at org.glassfish.jersey.message.internal.InboundMessageContext.singleHeader
I was able to find a case that works locally, by setting the header "Content-Type" with "application/json" (notice the header name case letter and remember it does not work in lower case...).
After deploying the lambda, its execution (via API Gateway) failed with a 400 (problem 3), despite a carefully crafted http request (with "Content-Type"). Ironically, by adding smart logging (i.e System.out(headers) in the handleRequest method), it turned out that from all of the headers sent from my RestClient, only one had been transformed into a lower case form : "content-type" (by the API Gateway??).
Well, no matter if it is appropriate to use the Spring encoding filter and Jersey : the 3 problems detailed above show weird behaviors only by using core aws classes.
What do you think about that ?