[SPARK-53457] Expose fabric8 dependencies as api for downstream users #305
Conversation
|
cc @peter-toth for review - thanks! |
There was a problem hiding this comment.
Could you elaborate on the real example of the cases? AFAIK, the official users of this repositories are supposed to use or provide YAML files only, @jiangzho . Maybe, did you want to say some downstream fork? Exposing something should be very careful.
Our public API surfaces fabric8 types (e.g. PodTemplateSpec), so hiding the dependencies behind implementation may cause comile errors for customers who need to use those types along with our API library.
|
In addition to direct YAML, user may also leverage our Java API library to create and manage Spark workloads. Its a common practice for user to use a microservice to orchestrate and record status at higher level. (example - they invented a set of POJO for Kubeflow operator previously as it's in Golang - but for Apache Spark Operator people may directly ask for Java API). While they build Spark spec in such microservices, they could be interacting with fabric8 models. It would become necessary for us to declare our dependencies in pom. |
|
We don't provide such a promise as a
|
To avoid breaking changes in our current artifact - how about publishing one additional artifact, like |
|
For the record, I landed a new fabric8 dependency which is marked as a breaking change from their side. |
### What changes were proposed in this pull request? This PR exposes fabric8 dependencies as api in gradle build and hence in pom ### Why are the changes needed? Our public API surfaces fabric8 types (e.g. PodTemplateSpec), so hiding the dependencies behind `implementation` may cause comile errors for customers who need to use those types along with our API library. This enhances developer experience and avoid surprises when integrating our API with projects that already depends on fabric8. ### Does this PR introduce _any_ user-facing change? Pom change only ### How was this patch tested? CIs ### Was this patch authored or co-authored using generative AI tooling? No
jiangzho
force-pushed
the
api
branch
3 times, most recently
from
37d4097 to
edf00fa
Compare
|
For the record, I'm currently auditing the dependencies officially. This kind of activity should be done before exposing anything officially in the ASF community. Thank you for raising this kind of use-case issue, @jiangzho .
|
|
BTW, please make the PR title and description up-to-date according to your changes (proposing a new java library module), @jiangzho .
|
| # Use Spark Operator API Library in Your Java Application | ||
|
|
||
| In addition to direct YAML, you may also leverage our Java API library to create and manage Spark | ||
| workloads. |
There was a problem hiding this comment.
Please mention that Apache Spark community doesn't aim to provide a compatibility for this module in terms of Semantic Versioning. Every version can talk to the exactly same version's Spark Operator, @jiangzho .
What changes were proposed in this pull request?
This PR exposes fabric8 dependencies as api in gradle build and hence in pom
Why are the changes needed?
Our public API surfaces fabric8 types (e.g. PodTemplateSpec), so hiding the dependencies behind
implementationmay cause comile errors for customers who need to use those types along with our API library.This enhances developer experience and avoid surprises when integrating our API with projects that already depends on fabric8.
Does this PR introduce any user-facing change?
Pom change only
How was this patch tested?
CIs
Was this patch authored or co-authored using generative AI tooling?
No