-
Notifications
You must be signed in to change notification settings - Fork 74
MLE-12345 Reworking how test plumbing clears a database #1798
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR refactors the database clearing mechanism in test infrastructure to address intermittent failures on Jenkins. The change replaces the REST API-based deletion approach with an XQuery-based solution to improve reliability.
- Replaces
DELETE v1/searchREST API call with XQuery evaluation for database clearing - Adds logging to track the number of URIs deleted during the clear operation
- Introduces slf4j import for logging functionality
| String count = client.newServerEval() | ||
| .xquery("let $uris := for $uri in cts:uris((), (), cts:true-query()) let $_ := xdmp:document-delete($uri) return $uri " + | ||
| "return fn:count($uris)") | ||
| .evalAs(String.class); | ||
| LoggerFactory.getLogger(ConnectedRESTQA.class).info("Cleared database, deleting {} URIs", count); |
Copilot
AI
Jul 23, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The XQuery loads all URIs into memory before deletion, which could cause memory issues with large databases. Consider using a batch processing approach or streaming deletion pattern instead of collecting all URIs at once.
| String count = client.newServerEval() | |
| .xquery("let $uris := for $uri in cts:uris((), (), cts:true-query()) let $_ := xdmp:document-delete($uri) return $uri " + | |
| "return fn:count($uris)") | |
| .evalAs(String.class); | |
| LoggerFactory.getLogger(ConnectedRESTQA.class).info("Cleared database, deleting {} URIs", count); | |
| int totalDeleted = 0; | |
| while (true) { | |
| String batchCount = client.newServerEval() | |
| .xquery("let $batch := subsequence(cts:uris((), (), cts:true-query()), 1, 100) " + | |
| "let $_ := for $uri in $batch return xdmp:document-delete($uri) " + | |
| "return fn:count($batch)") | |
| .evalAs(String.class); | |
| int deleted = Integer.parseInt(batchCount); | |
| totalDeleted += deleted; | |
| LoggerFactory.getLogger(ConnectedRESTQA.class).info("Deleted {} URIs in this batch", deleted); | |
| if (deleted < 100) { | |
| break; // Exit loop if fewer than 100 URIs were deleted, indicating no more URIs. | |
| } | |
| } | |
| LoggerFactory.getLogger(ConnectedRESTQA.class).info("Cleared database, total URIs deleted: {}", totalDeleted); |
| .xquery("let $uris := for $uri in cts:uris((), (), cts:true-query()) let $_ := xdmp:document-delete($uri) return $uri " + | ||
| "return fn:count($uris)") |
Copilot
AI
Jul 23, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] The XQuery string is complex and embedded inline, making it difficult to read and maintain. Consider extracting this to a separate XQuery file or a well-formatted multi-line string with proper indentation.
| .xquery("let $uris := for $uri in cts:uris((), (), cts:true-query()) let $_ := xdmp:document-delete($uri) return $uri " + | |
| "return fn:count($uris)") | |
| .xquery( | |
| "let $uris := " + | |
| " for $uri in cts:uris((), (), cts:true-query()) " + | |
| " let $_ := xdmp:document-delete($uri) " + | |
| " return $uri " + | |
| "return fn:count($uris)" | |
| ) |
The "clear" that happens via DELETE v1/search causes intermittent failures on Jenkins.
168bfae to
e9248c5
Compare
The "clear" that happens via DELETE v1/search causes intermittent failures on Jenkins.