Skip to content

extend condition to also filter by field exists or not #1879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 29, 2025

Conversation

dafnapension
Copy link
Collaborator

@dafnapension dafnapension commented Jul 27, 2025

Currently, Filter by condition throws an exception if a field participating in the condition is missing from the instance.
Suggesting here to extend conditions to also handle the very existence of the field in the instance as a measure of this instance being required.
This can be handy for cleaning datasets in some of the cards, which now throw exception for a missing field (breaking the pre-processing pipeline), and also, indirectly, shed light on cards that are totally erroneous, and are better be removed from the catalog:
E.g. for helpful cleaning:
Currently:
image

whereas pushing the new operators in the card, yields:

image

Another example, that makes us consider a removal of a totally erroneous card from the catalog:
image

while with the new filter, we find that the whole subset is missing that field, and we may want to remove the card (for subset 'experts') from the catalog altogether:

image

Just to prove the point that the whole dataset, of subset experts, does not match the preprocessing steps:
image

as compared to a subset that does match:
image

@dafnapension dafnapension requested a review from elronbandel July 27, 2025 08:53
Copy link
Member

@elronbandel elronbandel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dafnapension !

@elronbandel elronbandel merged commit 39a98c1 into main Jul 29, 2025
15 of 16 checks passed
@elronbandel elronbandel deleted the filter_if_missing_field branch July 29, 2025 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants