Determine new API URL structure for warehouse (starting with new JSON API)

[ctheune]

At the PyCon2014 sprint I have started to make bandersnatch easier to cache. This means moving away from XML-RPC in general.

I'm leveraging the existing /pypi//json API which already helps, but I'll need two more endpoints:

• get a list of all packages and their most recent serial
• get the changelog

I implemented the necessary code on a branch for PyPI:
https://bitbucket.org/ctheune/pypi/branch/ctheune-bandersnatch-json

However, I don't wanna force this through but have a decision how the URLs should look like.

Ideally we can implement this in both warehouse and PyPI in a way that bandersnatch can support both of them without breaking when you guys switch the public server (and I might be on vacation. ;) )

[dstufft]

So I have some ideas on both a new API for accessing data about PyPI and also some rough ideas for a new mirroring API in general. I'll take a look at what you have so far.

[dstufft]

So if I read your PR correctly, the new URLs would be https://pypi.python.org/json/changes and https://pypi.python.org/json/packages? If that's the case then I'm not really a big fan for adding those in Warehouse.

Ideally what i'd like to do is get a nice hypermedia based API setup probably rooted at /api/. Using something based on https://jsonapi.org/ is a possibility. There are a few options and need to dive into it to figure out what exactly needs done. Ideally the new API will also replace the existing JSON api and we can deprecate the old JSON api (but leave it in place until (or if!) it's no longer getting traffic.

[r1chardj0n3s]

I echo @dstufft in this. The question I have is whether we go all the way to /api/v0/ to future-proof us a little too. Unless we'd be happy with /api-v1/ or similar later on?

[dstufft]

So there are two ways to deal with that, one way is to version using the content type, so it's always /api/ but it'll select the version based on the content type, github uses this like: Accept: application/vnd.github.beta+json or Accept: application/vnd.github.v3+json. The other way is to do /api/v0/ etc. I lean towards using the content type but we'll need to figure out in general how we want to handle versioning going forward and how the code to handle that looks like.

[steveklabnik]

Just let me know if I can help out at all regarding JSON API stuff.

[brainwane]

As I understand it, this issue (designing and implementing a new Warehouse API) is a prerequisite for integrating twine into pip and thus dealing with pypa/packaging-problems#76 and pypa/packaging-problems#60 , per @dstufft's comment in pypa/twine#127. Is that correct? If so, I'd suggest we add this to one of our upcoming milestones.

[brainwane]

We talked about this issue in today's bug triage meeting and folks explained to me: Even though this may be necessary for some Twine improvements, this is not a ticket we will address before launch. This is a new feature and is best suited for post-launch; Warehouse needs to be done before we can improve twine.