Adding simulate ingest effective mapping #132833
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
masseyke
added
>enhancement
:Data Management/Ingest Node
elasticsearchmachine
added
the
Team:Data Management
Collaborator
|
Pinging @elastic/es-data-management (Team:Data Management) |
Collaborator
|
Hi @masseyke, I've created a changelog YAML for you. |
…:masseyke/elasticsearch into adding-simulate-ingest-effective-mapping
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds a new effective_mapping field to the simulate ingest API response that shows what the mapping would be for the index where each document would eventually be indexed. This enhancement helps users understand the final mapping structure that would apply to their documents after processing through ingest pipelines.
Key changes:
- Added
effective_mappingfield toSimulateIndexResponseto track and serialize the effective mapping - Updated
TransportSimulateBulkActionto calculate and include the effective mapping during validation - Added comprehensive test coverage for the new functionality including YAML REST tests
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| SimulateIndexResponse.java | Added effective_mapping field with serialization support and transport version handling |
| TransportSimulateBulkAction.java | Modified validation logic to capture and return effective mapping alongside existing validation results |
| TransportVersions.java | Added new transport version constant for effective mapping feature |
| RestSimulateIngestActionTests.java | Updated test cases to include effective mapping in expected responses |
| SimulateIndexResponseTests.java | Added test coverage for effective mapping serialization and XContent output |
| TransportSimulateBulkActionTests.java | Updated test expectations to include effective mapping field |
| 80_ingest_simulate.yml | Added comprehensive YAML REST test for effective mapping functionality with rerouting and data streams |
| 132833.yaml | Added changelog entry for the enhancement |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| if (indicesWithInvalidMappings.contains(imd.getIndex().getName())) { | ||
| if (indicesWithInvalidMappings.contains(imd.getIndex().getName()) | ||
| // We only want to throw exceptions inside TransportSimulateBulkAction: | ||
| && invocation.getArgument(1).getClass().getSimpleName().contains("TransportSimulateBulkAction")) { |
There was a problem hiding this comment.
Using class name string comparison is fragile and could break if the class is renamed or moved. Consider using instanceof or a more robust type checking mechanism instead of relying on the simple class name.
Suggested change
| && invocation.getArgument(1).getClass().getSimpleName().contains("TransportSimulateBulkAction")) { | |
| && invocation.getArgument(1) instanceof TransportSimulateBulkAction) { |
szybia
added a commit
to szybia/elasticsearch
that referenced
this pull request
Aug 15, 2025
* upstream/main: (32 commits) Speed up loading keyword fields with index sorts (elastic#132950) Mute org.elasticsearch.index.mapper.LongFieldMapperTests testSyntheticSourceWithTranslogSnapshot elastic#132964 Simplify EsqlSession (elastic#132848) Implement WriteLoadConstraintDecider#canAllocate (elastic#132041) Mute org.elasticsearch.test.rest.yaml.CcsCommonYamlTestSuiteIT test {p0=search/400_synthetic_source/_doc_count} elastic#132965 Switch to PR-based benchmark pipeline defined in ES repo (elastic#132941) Breakdown undesired allocations by shard routing role (elastic#132235) Implement v_magnitude function (elastic#132765) Introduce execution location marker for better handling of remote/local compatibility (elastic#132205) Mute org.elasticsearch.cluster.ClusterInfoServiceIT testMaxQueueLatenciesInClusterInfo elastic#132957 Unmuting simulate index data stream mapping overrides yaml rest test (elastic#132946) Remove CrossClusterCancellationIT.createLocalIndex() (elastic#132952) Mute org.elasticsearch.index.mapper.LongFieldMapperTests testFetch elastic#132956 Fix failing UT by adding a required capability (elastic#132947) Precompute the BitsetCacheKey hashCode (elastic#132875) Adding simulate ingest effective mapping (elastic#132833) Mute org.elasticsearch.index.mapper.LongFieldMapperTests testFetchMany elastic#132948 Rename skipping logic to remove hard link to skip_unavailable (elastic#132861) Store ignored source in unique stored fields per entry (elastic#132142) Add random tests with match_only_text multi-field (elastic#132380) ...
joshua-adams-1
pushed a commit
to joshua-adams-1/elasticsearch
that referenced
this pull request
Aug 15, 2025
szybia
added a commit
to szybia/elasticsearch
that referenced
this pull request
Aug 15, 2025
…-stats * upstream/main: (36 commits) Fix reproducability of builds against Java EA versions (elastic#132847) Speed up loading keyword fields with index sorts (elastic#132950) Mute org.elasticsearch.index.mapper.LongFieldMapperTests testSyntheticSourceWithTranslogSnapshot elastic#132964 Simplify EsqlSession (elastic#132848) Implement WriteLoadConstraintDecider#canAllocate (elastic#132041) Mute org.elasticsearch.test.rest.yaml.CcsCommonYamlTestSuiteIT test {p0=search/400_synthetic_source/_doc_count} elastic#132965 Switch to PR-based benchmark pipeline defined in ES repo (elastic#132941) Breakdown undesired allocations by shard routing role (elastic#132235) Implement v_magnitude function (elastic#132765) Introduce execution location marker for better handling of remote/local compatibility (elastic#132205) Mute org.elasticsearch.cluster.ClusterInfoServiceIT testMaxQueueLatenciesInClusterInfo elastic#132957 Unmuting simulate index data stream mapping overrides yaml rest test (elastic#132946) Remove CrossClusterCancellationIT.createLocalIndex() (elastic#132952) Mute org.elasticsearch.index.mapper.LongFieldMapperTests testFetch elastic#132956 Fix failing UT by adding a required capability (elastic#132947) Precompute the BitsetCacheKey hashCode (elastic#132875) Adding simulate ingest effective mapping (elastic#132833) Mute org.elasticsearch.index.mapper.LongFieldMapperTests testFetchMany elastic#132948 Rename skipping logic to remove hard link to skip_unavailable (elastic#132861) Store ignored source in unique stored fields per entry (elastic#132142) ...
This was referenced Aug 15, 2025
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a new field to each document response in the response of the simulate ingest API. The field is
effective_mapping, and shows what the mapping would be for the index where the document would be indexed.