Ancestral state reconstruction and gaps

[ElizabethRobbins]

Hello,

Why can't ancestral state reconstruction insert a gap in a node sequence even when both children have gaps at that position?

Is this a possible feature to add? As it would make ASRs more realistic.

Many thanks,
Elizabeth

[roblanf]

Inside ML inference, gaps are treated identically to N's. This means that gaps 'exist' in exactly the same way as other bases as far as the actual inference is concerned.

Typically what we do for feature requests is wait to see how many people are interested, and judge that against the difficulty of the request. So, while we wait for @bqminh @thomaskf and/or other developers to weigh in, it would help if you could encourage other potential users to support this feature request. (Of course, at some point feature requests need funding to implement too).

Even with this request, which seems simple enough on the surface, there are a number of details that need careful thought, and this would be a good thread for this discussion.

A quick list of things that spring to mind:

1. What if not all children have a gap, but if e.g. 99 of 100 children have it (you mention 'both' but I don't think it would be sensible to apply this only to cherries on the tree. It would need to be a general model that applies to any tree)?
2. How do we know where the root is - i.e. how do we know which tips are actually children?

Even this short list suggests some substantial challenges to the simplest implementation, and I suspect is why it doesn't already exist. The central challenge is that we don't use an indel model in ML phylogenetics, which makes reconstructing ancestral indel states very tricky. And thinking about the questions above makes me wonder if the very simple implementation of 'believing' and indel in the restrictive case of every child posessing it is all that useful (not to mention the challenge of knowing what a child is in an unrooted tree).

This is not to say it's not possible, just that in order to move ahead it's important to sort the exact details out first.

If you can encourage other ASR users to contribute to this thread, that will certainly help!

[tufeiliu]

This is definitely a needed feature.

[jb16159]

Gap reconstruction would useful for me too, but I appreciate it would be a lot of work. The best solution I've come up with so far is to use a different tool to reconstruct gaps (e.g. ARPIP), and then to combine that with the residues estimated by IQ-TREE with it's better models of amino acid substitution.

[ElizabethRobbins]

We have come up a hack that only uses IQ-TREE (because we were interested in reducing dependencies) of first converting the MSA to binary sequences (0/1 being gapped/non-gapped positions, respectively). Then using the phylogenetic tree (constructed from the biological sequences) as a constraint tree to reconstruct the ancestral binary sequences using the GTR2 model. Finally, we added the inferred gap positions onto the ASR output for the biological sequences. I am sure this method is not perfect by any means, but seems to provide a sensible output...