Use automatic URI encoding

[ramondeklein]

This PR makes #3343 obsolete.

This PR uses proper URI encoding and enables a hook in the Swagger-based API generation that automatically encodes all parameters that are send in the URL path. The main advantage is that there is no encoding/decoding required in Javascript/Go code, unless paths are manually created.

Pros

• Simpler code and author doesn't need to think about encoding/decoding (unless creating URLs manually).
• Shorter and more readable URLs.

Cons

• Encoded URLs are different.
• Enterprise console also needs to be updated to deal with these new URLs.

I have tested all kind of "difficult" strings (including % or / signs, Chinese characters, ...) for users, policies, groups and it all seems to work fine. Also the object-browser (websocket based) seems to be fine.

IMPORTANT

After merging this PR, the console API effectively changes (due to different encoding). Because the Enterprise console also calls the Enterprise API the Enterprise console needs to be upgraded too.

[ramondeklein]

@harshavardhana I noticed that you mentioned that we want to create a release for console soon. I think this PR is a significant improvement to code-quality and there's one thing less developer's need to worry about (do I need to encode or not?). Because the path/query parameters to the API use a different encoding, this is a breaking change for the Enterprise console.

Although tested pretty extensively, it's pretty big code-change and we may introduce new issues.

[bexsoft]

There is an issue when trying to view files that contain spaces in the name:

Tried with special characters, spaced & slashes in the rest of pages and it looks good

[ramondeklein]

@bexsoft Thanks for reporting. There was a SanitizeEncodedPrefix(...) method in the Go code that isn't necessary anymore. It replaced all spaces with + characters and caused the failure. This has been fixed and it looks like files with spaces now also work.

Another annoying method removed 😄

[ramondeklein]

@dvaldivia @harshavardhana Now we have released the console, I would suggest merge this PR. Then I can work on making the same changes on the Enterprise console and encoding should be a lot less troublesome.

[ramondeklein]

I would like to merge this, but there are no reviewers.

[cesnietor]

@ramondeklein the following api threw me an error.
Do we need to update the UI?

http://localhost:5005/api/v1/buckets/bucket1/objects?prefix=file-s2.txt&with_versions=true

returns

{
    "detailedMessage": "illegal base64 data at input byte 4",
    "message": "an error occurred, please try again"
}

That's while trying to access a bucket with a folder:
UI shows:

[cesnietor]

When clicking on users and clicking on AddGroups the api returns 404 I'm assuming the UI is not passing the username encoded as it would be expected.

[cesnietor]

Similar issue when accessing a Group:

[ramondeklein]

@cesnietor I'll check tomorrow (it's midnight here), but are you sure you also run the backend service on this branch? The utils. DecodeBase64() isn't in the code anymore, so I wouldn't expect a base64 decoding issue.

I checked the code, but since I submitted this PR more code has been added that uses encoding, so I'll also fix that. But that only will affect the /admin/inspect.

[cesnietor]

@cesnietor I'll check tomorrow (it's midnight here), but are you sure you also run the backend service on this branch? The utils. DecodeBase64() isn't in the code anymore, so I wouldn't expect a base64 decoding issue.

I checked the code, but since I submitted this PR more code has been added that uses encoding, so I'll also fix that. But that only will affect the /admin/inspect.

ahh you are right, I was using a different port and messed up my testing.
I used the proper backend and FE and it works as expected.
LGTM

[ramondeklein]

The Enterprise console is now also updated (see https://github.com/miniohq/enterprise-console/pull/879). We need to make sure to merge both of the PRs at once and ensure that clusters will use the updated Minio version.

[ramondeklein]

@cesnietor There was one place where we still had base64 decoding (although it was skipped if it didn't work), so 81949d1 dismissed your review. Could you please re-approve?

@bexsoft I have fixed the bug if files had spaces a while back, so could you please check and approve if you think it's okay?

[ramondeklein]

Can someone merge this (I don't have sufficient permissions). I will merge Enterprise console counterpart afterwards.

PS: We need to upgrade enterprise.min.dev with new Enterprise console and Minio instances.