{ "id": 672054, "title": "TM-score and pairwise distance matrix", "url": "/competitions/stanford-rna-3d-folding-2/discussion/672054", "author_name": "Bridgeport", "post_date": "2026-02-05T19:33:56.029000", "votes": 1, "comment_count": 9, "views": 0, "content": "

I am trying to understand how TM-score works. I came across the following example.

\n

For target ID = 1ANR, I found an alternative set of coordinates, such that if one computes pairwise distance matrices for original and alternative coordinates, then they are very close (2.31e-11), but TM-score is only 0.21225.

\n

Is not this contradictory?

\n

Here is my notebook with computations:\nhttps://www.kaggle.com/code/bridgeport/tm-score-and-pairwise-distance-matrix

\n

P.S. provided I did not make any mistakes in my calculations.

\n

P.P.S I guess I found an answer. USalign does not optimize over reflections. This means that nearly perfect answers missing reflection will be penalized heavily. Should not we use pairwise distance matrix as the main evaluation metric?

", "messages": [ { "id": 3402370, "postDate": "2026-02-05T21:48:43.083Z", "content": "

It's an interesting idea. But although a mirror-image solution would have the same the pairwise distance matrix, the direction of rotation of the helix (for example) would be reversed, wouldn't it?

", "rawMarkdown": "It's an interesting idea. But although a mirror-image solution would have the same the pairwise distance matrix, the direction of rotation of the helix (for example) would be reversed, wouldn't it?", "votes": 1, "replies": [ { "id": 3402750, "postDate": "2026-02-06T18:01:52.773Z", "content": "

I do not think it is possible to reverse helix and have the same pairwise distance matrix.

", "rawMarkdown": "I do not think it is possible to reverse helix and have the same pairwise distance matrix. ", "replies": [ { "id": 3402776, "postDate": "2026-02-06T19:33:42.060Z", "content": "

When you say \"reflections\" in your P.P.S, I read that as a parity transformation, i.e. taking x --> -x for one of the cartesian components or for all three. (So my apologies if I've misunderstood your meaning.) The pairwise distance matrix is manifestly invariant under this sort of transformation because its elements are all of the form sqrt((x1-x2)^2+(y1-y2)^2+(z1-z2)^2); the squares make the minus signs drop out.

\n

However, a helix is odd under parity -- if you start with a right-handed corkscrew and apply a parity transformation, you'll wind up with a left-handed corkscrew. To see this: a helix around the x-axis might be written as y=cos(x) and z=sin(x). Under the parity transformation x-->-x, this becomes y=cos(x) and z=-sin(x), i.e. the orbit progresses in the opposite sense as one goes toward positive x.

\n

This is the point I was trying to make above in the form of a question. I'm still doing some basic EDA and haven't yet looked at the TM score in detail, but I'd assume it's built in such a way that a reversal of the helix direction should not show up as a good match. The low TM score you observe for a mirror-image aligns with this expectation.

", "rawMarkdown": "When you say \"reflections\" in your P.P.S, I read that as a parity transformation, i.e. taking x --> -x for one of the cartesian components or for all three. (So my apologies if I've misunderstood your meaning.) The pairwise distance matrix is manifestly invariant under this sort of transformation because its elements are all of the form sqrt((x1-x2)^2+(y1-y2)^2+(z1-z2)^2); the squares make the minus signs drop out.\n\nHowever, a helix is odd under parity -- if you start with a right-handed corkscrew and apply a parity transformation, you'll wind up with a left-handed corkscrew. To see this: a helix around the x-axis might be written as y=cos(x) and z=sin(x). Under the parity transformation x-->-x, this becomes y=cos(x) and z=-sin(x), i.e. the orbit progresses in the opposite sense as one goes toward positive x.\n\nThis is the point I was trying to make above in the form of a question. I'm still doing some basic EDA and haven't yet looked at the TM score in detail, but I'd assume it's built in such a way that a reversal of the helix direction should not show up as a good match. The low TM score you observe for a mirror-image aligns with this expectation.\n", "votes": 1 }, { "id": 3402789, "postDate": "2026-02-06T20:25:32.033Z", "content": "

Good point, but we are in discrete situation, so only pairwise distance matrix matters.

", "rawMarkdown": "Good point, but we are in discrete situation, so only pairwise distance matrix matters." }, { "id": 3402792, "postDate": "2026-02-06T20:38:09.907Z", "content": "

I'm not sure I understand -- what do you mean by \"discrete situation\"?

", "rawMarkdown": "I'm not sure I understand -- what do you mean by \"discrete situation\"?" }, { "id": 3402801, "postDate": "2026-02-06T21:31:47.733Z", "content": "

I think only relative (to each other) position of molecules matters, which is fully captured by pairwise distance matrix.

", "rawMarkdown": "I think only relative (to each other) position of molecules matters, which is fully captured by pairwise distance matrix." }, { "id": 3402807, "postDate": "2026-02-06T21:46:27.090Z", "content": "

I have the impression that RNA is usually a right-handed helix, so that conformations that are left-handed are less common in nature. Given that, it makes sense to me that the organizers would choose a metric that assigns a higher score when the handedness of the helix matches the observations. But in any case, it's their call, not mine. šŸ˜

", "rawMarkdown": "I have the impression that RNA is usually a right-handed helix, so that conformations that are left-handed are less common in nature. Given that, it makes sense to me that the organizers would choose a metric that assigns a higher score when the handedness of the helix matches the observations. But in any case, it's their call, not mine. šŸ˜", "votes": 1 }, { "id": 3403950, "postDate": "2026-02-09T16:05:27.380Z", "content": "

Hi, indeed USalign does not search over reflections when aligning structures.

\n

Its heuristic looks over transformations in SO(3) not O(3).

\n

Biology is chiral so mirror image molecules would interact with existing biology differently.

\n

So @particlebbq is right - reflected structures are not considered accurate across most software or evaluations, including in this competition.

\n

Thanks for the question as Iā€™m sure this was a point of confusion for many Kagglers!

", "rawMarkdown": "Hi, indeed USalign does not search over reflections when aligning structures.\n\n Its heuristic looks over transformations in SO(3) not O(3).\n\nBiology is chiral so mirror image molecules would interact with existing biology differently. \n\nSo @particlebbq is right - reflected structures are not considered accurate across most software or evaluations, including in this competition. \n\nThanks for the question as Iā€™m sure this was a point of confusion for many Kagglers!", "votes": 3 }, { "id": 3404013, "postDate": "2026-02-09T18:15:14.697Z", "rawMarkdown": "", "isDeleted": true } ] } ] }, { "id": 3402339, "postDate": "2026-02-05T19:33:56.030Z", "content": "

I am trying to understand how TM-score works. I came across the following example.

\n

For target ID = 1ANR, I found an alternative set of coordinates, such that if one computes pairwise distance matrices for original and alternative coordinates, then they are very close (2.31e-11), but TM-score is only 0.21225.

\n

Is not this contradictory?

\n

Here is my notebook with computations:\nhttps://www.kaggle.com/code/bridgeport/tm-score-and-pairwise-distance-matrix

\n

P.S. provided I did not make any mistakes in my calculations.

\n

P.P.S I guess I found an answer. USalign does not optimize over reflections. This means that nearly perfect answers missing reflection will be penalized heavily. Should not we use pairwise distance matrix as the main evaluation metric?

", "rawMarkdown": "I am trying to understand how TM-score works. I came across the following example.\n\nFor target ID = 1ANR, I found an alternative set of coordinates, such that if one computes pairwise distance matrices for original and alternative coordinates, then they are very close (2.31e-11), but TM-score is only 0.21225. \n\nIs not this contradictory? \n\nHere is my notebook with computations:\nhttps://www.kaggle.com/code/bridgeport/tm-score-and-pairwise-distance-matrix\n\nP.S. provided I did not make any mistakes in my calculations.\n\nP.P.S I guess I found an answer. USalign does not optimize over reflections. This means that nearly perfect answers missing reflection will be penalized heavily. Should not we use pairwise distance matrix as the main evaluation metric?", "votes": 1 } ], "comments": [ { "id": 3402370, "author_name": "particlebbq", "author_url": "", "post_date": "2026-02-05T21:48:43.083000", "content": "

It's an interesting idea. But although a mirror-image solution would have the same the pairwise distance matrix, the direction of rotation of the helix (for example) would be reversed, wouldn't it?

", "votes": 1, "replies": [ { "id": 3402750, "author_name": "Bridgeport", "author_url": "", "post_date": "2026-02-06T18:01:52.773000", "content": "

I do not think it is possible to reverse helix and have the same pairwise distance matrix.

", "votes": 0, "replies": [ { "id": 3402776, "author_name": "particlebbq", "author_url": "", "post_date": "2026-02-06T19:33:42.060000", "content": "

When you say \"reflections\" in your P.P.S, I read that as a parity transformation, i.e. taking x --> -x for one of the cartesian components or for all three. (So my apologies if I've misunderstood your meaning.) The pairwise distance matrix is manifestly invariant under this sort of transformation because its elements are all of the form sqrt((x1-x2)^2+(y1-y2)^2+(z1-z2)^2); the squares make the minus signs drop out.

\n

However, a helix is odd under parity -- if you start with a right-handed corkscrew and apply a parity transformation, you'll wind up with a left-handed corkscrew. To see this: a helix around the x-axis might be written as y=cos(x) and z=sin(x). Under the parity transformation x-->-x, this becomes y=cos(x) and z=-sin(x), i.e. the orbit progresses in the opposite sense as one goes toward positive x.

\n

This is the point I was trying to make above in the form of a question. I'm still doing some basic EDA and haven't yet looked at the TM score in detail, but I'd assume it's built in such a way that a reversal of the helix direction should not show up as a good match. The low TM score you observe for a mirror-image aligns with this expectation.

", "votes": 1, "replies": [] }, { "id": 3402789, "author_name": "Bridgeport", "author_url": "", "post_date": "2026-02-06T20:25:32.033000", "content": "

Good point, but we are in discrete situation, so only pairwise distance matrix matters.

", "votes": 0, "replies": [] }, { "id": 3402792, "author_name": "particlebbq", "author_url": "", "post_date": "2026-02-06T20:38:09.907000", "content": "

I'm not sure I understand -- what do you mean by \"discrete situation\"?

", "votes": 0, "replies": [] }, { "id": 3402801, "author_name": "Bridgeport", "author_url": "", "post_date": "2026-02-06T21:31:47.733000", "content": "

I think only relative (to each other) position of molecules matters, which is fully captured by pairwise distance matrix.

", "votes": 0, "replies": [] }, { "id": 3402807, "author_name": "particlebbq", "author_url": "", "post_date": "2026-02-06T21:46:27.090000", "content": "

I have the impression that RNA is usually a right-handed helix, so that conformations that are left-handed are less common in nature. Given that, it makes sense to me that the organizers would choose a metric that assigns a higher score when the handedness of the helix matches the observations. But in any case, it's their call, not mine. šŸ˜

", "votes": 1, "replies": [] }, { "id": 3403950, "author_name": "Rhiju Das", "author_url": "", "post_date": "2026-02-09T16:05:27.380000", "content": "

Hi, indeed USalign does not search over reflections when aligning structures.

\n

Its heuristic looks over transformations in SO(3) not O(3).

\n

Biology is chiral so mirror image molecules would interact with existing biology differently.

\n

So @particlebbq is right - reflected structures are not considered accurate across most software or evaluations, including in this competition.

\n

Thanks for the question as Iā€™m sure this was a point of confusion for many Kagglers!

", "votes": 3, "replies": [] }, { "id": 3404013, "author_name": "", "author_url": "", "post_date": "2026-02-09T18:15:14.697000", "content": "", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3402370": "It's an interesting idea. But although a mirror-image solution would have the same the pairwise distance matrix, the direction of rotation of the helix (for example) would be reversed, wouldn't it?", "3402339": "I am trying to understand how TM-score works. I came across the following example.\n\nFor target ID = 1ANR, I found an alternative set of coordinates, such that if one computes pairwise distance matrices for original and alternative coordinates, then they are very close (2.31e-11), but TM-score is only 0.21225. \n\nIs not this contradictory? \n\nHere is my notebook with computations:\nhttps://www.kaggle.com/code/bridgeport/tm-score-and-pairwise-distance-matrix\n\nP.S. provided I did not make any mistakes in my calculations.\n\nP.P.S I guess I found an answer. USalign does not optimize over reflections. This means that nearly perfect answers missing reflection will be penalized heavily. Should not we use pairwise distance matrix as the main evaluation metric?" } }