Skip to content

Initialize this repo #1

@fnesveda

Description

@fnesveda

Right now, we have two major Python projects - the Apify SDK and the Apify API client. They're both using some of the same constants and utilities, and right now they're just copied between the two repositories, because it was the easiest way to get the projects out quickly.

In the long term, we're planning to have more Python libraries and use more shared constants and utilities, so it will be beneficial to have those defined here, in one place, and import them from here in other projects (similar to how we do it with https://github.com/apify/apify-shared-js and JavaScript libraries).

To do that, we need to:

  • create a new project on PyPI called apify-shared
  • copy over the project setup from e.g. https://github.com/apify/apify-client-python to here, and change the library name from apify-client to apify-shared
  • copy the constants and the utilities used in the API client and SDK to here (only the relevant ones, some of them make sense only in the context of one project, so they can stay there)
    • they're typically in the consts.py and _utils.py files in the project repositories
  • publish this on PyPI (the CI copied from the API client should work for this)
  • replace the constants in the API client and SDK with those from the shared package (and reexport them for convenience)

There are some things to keep in mind when working on this:

  • documentation
    • this project won't have its own documentation, so we can remove all the docs workflows
    • we want to have the constants (not necessarily the utilities yet, because those are all internal) documented in the SDK and API client docs, so we should somehow set up the documentation building there to merge the documentation from the main project to include them
      • it will be a bit ugly, but I think the easiest would be to git clone the right tag of this repo when building the project docs, build the documentation for this project, and then merge the resulting API reference JSON in the project's API reference JSON.
  • sharing the repo setup across Python projects
    • if we add more and more Python projects, we should somehow share the linter & type checker configs between the repos, so if we e.g. add a linter rule, we don't have to add it to every repo separately
    • this is not necessary for this repo's initialization, just something to start thinking about
  • sharing constants between JS and Python
    • there will be a lot of constants that are shared between JS and Python (probably almost all of them)
    • we should start thinking about how to make sure they're the same
      • either we should single-source them and automatically generate the JS and Python packages
      • but probably it would be enough to just write a CI job which checks that they're the same regularly, there won't be that many changes to make single-sourcing worth the effort
    • again, this is not necessary for this repo's initialization, just something to start thinking about

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request.t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions