Make the DOI Resolution Fetcher return nothing when the DOI leads to a host for which a tailored fetcher exists

[Toromtomtom]

Fixes #6922

• Change in CHANGELOG.md described (not applicable)
• Tests created for changes (if applicable)
• Manually tested changed features in running JabRef (always required)
• Screenshots added in PR description (for UI changes)
• Checked documentation: Is the information available and up to date? If not created an issue at https://github.com/JabRef/user-documentation/issues or, even better, submitted a pull request to the documentation repository.

[tobiasdiez]

As you noticed, the underlying problem is actually that the DOI fetcher has a higher trust value as the publishers. I think it would be a good idea to change it to "publishers > identifier-based resolution (doi, arXiv) > general search (google)". @JabRef/developers @Toromtomtom do you see any problem with this solution?

[Toromtomtom]

I think it would be a good idea to change it to "publishers > identifier-based resolution (doi, arXiv) > general search (google)".

I also think that this would be a better solution.

[koppor]

+1 from my side, too

[Toromtomtom]

I reverted my previous commits and decreased the trust level of the DOI resolution fetcher. This works for me, but maybe someone more involved in the project wants to weigh in on the ranking of the full text fetchers.

[koppor]

Food for thought:

• A DOI uniquely identifies a paper. Per defition, a DOI leads to the right paper. Everything else is good guessing.
• One title of a paper may lead to different publications of it. One the confernce version, the other the journal version. --> the PDF could be chosen randomly
• What about the consequences for other fetchers? Do we overlook something?
• Can't we contact Springer to fix their DOI 2 PDF mapping?

Proposal: Can we add a special handling for Springer? If a DOI directs to Springer, we use the Springer Fetcher. In all other cases, the functionality is untouched. In this way, we accept that this is a hack.

To really judge, there would be a test needed retrieving 1000 papers and check whether the retrieval rate is higher or lower with this check. - Alternatively, can we add telemetry for that?

[koppor]

This decreases the result quality of the DOI fetcher (always leading to the "right" paper) to the quality of Google Scholar. (From the highest to the lowest)

Can the solution of the title? 😇

Make the DOI Resolution Fetcher return nothing when the DOI leads to a host for which a tailored fetcher exists

[Siedlerchr]

The problem is that a DOI often does not lead to the fulltext version directly, but to the site where to find the fulltext. And our DOIResolution Fetcher does some magic guessing by looking at the first PDF-link the sourcecode of the website.
Every publisher's/journal page looks different.

[tobiasdiez]

The springer fetcher also only looks at the DOI, but uses the springer API to find the correct URL for the download.

jabref/src/main/java/org/jabref/logic/importer/fetcher/SpringerLink.java

Lines 39 to 49 in deb2f20

	Optional<DOI> doi = entry.getField(StandardField.DOI).flatMap(DOI::parse);
	if (!doi.isPresent()) {
	return Optional.empty();
	}
	// Available in catalog?
	try {
	HttpResponse<JsonNode> jsonResponse = Unirest.get(API_URL)
	.queryString("api_key", API_KEY)
	.queryString("q", String.format("doi:%s", doi.get().getDOI()))
	.asJson();

[koppor]

Devcall decision: Use first solution. -- @koppor will do git magic

[Toromtomtom]

All right, thanks for taking care of this!

[tobiasdiez]

@koppor In addition, the SpringerLink should have a higher trust score as the DoiResolution fetcher, since it's also DOI-based but custom-tailored to Springer. I would also merge this class with the other springer fetcher.

[Toromtomtom]

Is there anything I can do to move this forward? Reset the branch or something?

[koppor]

Steps:

1. I do as promised at Make the DOI Resolution Fetcher return nothing when the DOI leads to a host for which a tailored fetcher exists #6937 (comment)
2. I search all issues and PRs to document the idea of the concept (priorities, information sources, ...)
3. I think how to transform the comment Make the DOI Resolution Fetcher return nothing when the DOI leads to a host for which a tailored fetcher exists #6937 (comment) to a follow-up issue. Which implications should be thought of.

In parallel, I discuss with @stefan-kolb, because he invented the whole thing. My mistake was not to enforce that design decisions are documented (either as ADR or as other text files)

[stefan-kolb]

Initial PR #3882
More info and discussion: #3881

[koppor]

I think, I collected all documentation and put it at the appropriate place at #6990.

So, nothing to do for @Toromtomtom in this PR.

[koppor]

The first two commits are in master now. See ce9f714.