Dear Philippe,

here the answers and comments to you comments on the triple-T chapter.

> I have also read through the 3T chapter now, and again I have to say you've
> done an excellent job and I am happy we waited for this one! Your primer on
> topology testing will be an important asset for the book and again I have
> only a few comments (usually to make things more explicit again).

Thanks a lot for the flowers.

> For a novice, however, I think
> it might be useful to have some sort of visual representation of the test
> structure, especially given the dimensionality of a set of trees to test and
> the bootstrapping procedure. I was thinking something along the line of the
> the attached representations for KH, SH and he confidence sets based on
> likelihood weights. If you agree that this might be useful, I could modify
> these according to your recommendations and send you the files, or you are
> of course welcome to make something yourself.

I prepared figures myself. I find them more intuative than yours, but
this might be a matter of taste.

> I have a particular question about section 12.4. When you describe the
> non-parametric and parametric bootstrap to get null distributions of a
> likelihood ratio statistic, you already do so in relationship to tree
> testing. I think this might confuse the novice who is not aware of what
> follows. If one would simply obtain the likelihood ratios for pairs of trees
> for the replicates and use that as null distribution, It wouldn't make a
> proper test. The essential part, that you then describe in the KH, is to
> recenter the likelihoods before getting the log likelihood difference, which
> makes it a proper null distribution. I think there are two solutions to
> this; either you could describe the bootstrapping procedures more generally
> without referring to tree testing, or you could acknowledge that simply
> obtaining the likelihood ratios for pairs of trees for the replicates is not
> a proper null distribution yet and we will have to do something specific to
> make it a null distribution for tree testing. The same rationale is true for
> SH, but we know we are interested in the differences that can be expected
> between a particular tree and the ML tree, so we have to identify those
> differences for each bootstrap replicate.
>
> Since the recentering is in fact essential, I think it would be useful to
> emphasize that the first time you mention it in 12.5.1. Essentially, what we
> are trying to get is a distribution of differences between trees that one
> can expect by chance, right? The bootstrapping procedure allows one to get
> the variability of likelihood support, expected by chance where chance is
> essentially sampling variability, for a particular tree (but not between
> trees). By recentering according to mean likelihoods we are saying that the
> trees have equal support, and the differences we are now obtaining are those
> one can expect because of sampling variability. I think it would be useful
> to spend some words along this line to this.

I did quite some rephrasing to clarify this.

> Some minor things:
> We have the glossary again, could you format the terms in the list in
> bold+italics the first time they are mentioned?

done

> And if possible, program names in new courier format.

see my question in the ML answers... but this is quickly done.

> When you refer to Siegel and Castellan, the reference ends with a '7'?

it's page 7 - changed to "p. 7", or do you prefer "page 7"

> On page 3, you use ML tree, I don't think the abbreviation was defined
> before.

changed

> In the caption of fig 12.2, LTR is used instead of LRT.

corrected

> When you discuss the general structure of tree tests, step (iv) is about
> computing the log-likelihood differences for tree a and tree b. It is not
> clear what are tree a and tree b here, is this just two trees from the
> collection of trees defined earlier, or is either a or b the ML tree? I
> guess this depends on the H0, but it might be useful to explain this more
> explicitly.

reprased to clarify

> The first two sentences of 12.6 read somewhat better if rephrased:
> Strimmer and Rambaut (2002) approach the problem of comparing trees
> from a different perspective. Instead of significance testing, they devised
> a method that generates a confidence set of trees based

OK

> Does the sentence after equation 12.5 need to start with 'with' all
> likelihoods...

done

> The reference for LRT on the first line on page 15, could also be expanded
> to chapter 11

added

> In the same sentence, does 'test' needs to be plural?

corrected

> The second sentence would read better if:
> more programs are available that only implement the KH and SH test.

done

> The second sentence for 12.9.1, seems to have a verb redundancy: "recommend
> refer"

corrected

> In the next sentence: "for the later use" is probably better "for later use"

OK

> On page 17, the sentence: Then we will run TREE-PUZZLE with the following
> settings:
> Might be expanded for the readers, which havent gone through the practice
> section of the likelihood chapter, to: Then we will run TREE-PUZZLE with the
> following settings (changing option k, x, w, and c ):

done

> On page 17, "proceeds to" is probably better "proceeds with"

corrected

> Page 19, do the standard deviations relate to the log likelihoods or the log
> likelihood differences?

differences (the term directly before "their", should hence be clear)

> Maybe mention that CONSEL can be run on PC, unix/linux and MacosX?

done

> When I run CONSEL, the P-values can be slightly different for the au (not
> for the KH and SH). This is probably because of the stochastic effects of
> the multiscale bootstrapping? It might be useful to point this out for
> readers repeating the exercise.

Comments added at the relevant places.

> Page 21: what are the weighted versions of SH and KH?

This was not easy to determine. Hence, I tried to dodge around that in the 
first version. The text in the relevant papers did not realy explain it well
and Shimodaira used a notation which could easily lead to mistakes with
covariance... and the most clear paper seems to be in japanese :(
In the end, I asked Shimodaira directly and he clarified the weighted 
versions. Hence I added a short section explaining the weighted versions
wsh and ksh.

> In Table 12.1, you state that 'stars' mean... But they are '+'

corrected

> Second to last sentence: do mean topological features when you say
> structural features?

I meant the structural features of the tree topology - changed to
"topological features".