{ "topic": { "id": 77250, "title": "Our solution for 2nd place", "authorName": "Vladimir Boza", "commentCount": 59, "votes": 138, "postDate": "2019-01-10T23:59:34.569000" }, "comments": [ { "id": 453904, "authorName": "William Cook", "votes": 32, "postDate": "2019-01-11T00:13:41.227000", "content": "

Many thanks for the great competition. Your team pushed us everyday to find new methods to improve the tours. The penalty schedule is a beautiful idea. Wish we had thought of that!

" }, { "id": 453905, "authorName": "Vladimir Boza", "votes": 12, "postDate": "2019-01-11T00:14:19.020000", "content": "

We would love to hear about your methods!

" }, { "id": 453913, "authorName": "William Cook", "votes": 17, "postDate": "2019-01-11T00:24:43.833000", "content": "

Thanks. Your summary is great! Keld and I will put something together in the next day or two. Our Kernel score this morning was 1514320.92. We tried another run today using more time in the final optimization, but the random number generator was not on our side and we ended up with a solution 5 points longer.

" }, { "id": 453919, "authorName": "Kostya Atarik", "votes": 9, "postDate": "2019-01-11T00:32:44.137000", "content": "

Can you make it public for educational purposes?

" }, { "id": 454127, "authorName": "Farmár", "votes": 2, "postDate": "2019-01-11T06:55:18.147000", "content": "

We also found random number generation to be doing quite some deal. Our kernel has an engineered LKH seeds at the beginning as it was making a difference of up to 40 raw points on the base score. Resulting optimization had lots of variance as well, it is actually quite hard to reproduce our best run ...

" }, { "id": 454376, "authorName": "William Cook", "votes": 28, "postDate": "2019-01-11T14:52:37.543000", "content": "

Our Kernel with Keld Helsgaun is now public https://www.kaggle.com/bicotsp/pmtest1

\n\n

Not too much to see -- we are newbies with the Kernel format. We worked entirely with the old-school (we our old!) tools of C and bash scripts. It will be interesting to see how much other local-improvement and merging operations can improve the score of our overall best 1513747.36 tour. It is available in TSPLIB format http://www.math.uwaterloo.ca/tsp/pm/santa.1513747.tour

" }, { "id": 454884, "authorName": "YaGana Sheriff-Hussaini", "votes": 1, "postDate": "2019-01-12T12:33:03.793000", "content": "

Thanks @bicotsp. I posted your comment as a thread in the discussion section so that everybody can see it. I hope that is okay.

" }, { "id": 455731, "authorName": "averagemn", "votes": 1, "postDate": "2019-01-14T12:59:34.067000", "content": "

Thanks for sharing! Would you consider writing up a summary for your 1st place similar as in this post (2nd place)?

" }, { "id": 453927, "authorName": "Vindar", "votes": 22, "postDate": "2019-01-11T00:47:33.583000", "content": "

Congratulations to all.

\n\n

And Vlado, I feel no shame :-)

\n\n

I made a first submission to check I that got the file format right and then my final one... I just think its look good like that having just two submissions. Anyway it was a fun challenge !

\n\n

if anyone is interested about my approach. Here are some details:

\n\n

In the end, I ended up re-coding from scratch a TSP solver in C++ using LK heuristic and taking into account the prime penalty (in fact all mod 10 penalty to make computation faster) and I also combined it with IPT. Everything optimized to run on multi-core processors. But unfortunately, I also did not think about the penalty schedule and I started that approach too late to improve it further than my final score...

\n\n

Edit Post (forgot to mention): I also started from a LKH solution (not a very good one, about 200 more than the best one found). I also used k-kicks to escape local minimum. SA with periodic re-heating to improve after local search and a little bit of GA (but this was not really effective because I did not have a diverse enough population).

\n\n

Initially, I had a different idea which, I believe could have been efficient if the penalty had been stronger. The idea was NOT to optimize penalty at 9 mod 10 but instead to take any portions [a,b] of tour and try to minimize instead the minimum penalty for this portion for any i mod 10. This way, the 0 reference points is removed and we can give a score to a portion of tour without having to know where it is located. This creates independence and it is possible to use a multiscale approach: optimize [a,b][c,d] -> [a,d]. This idea is similar to some ideas in spin glasses in statistical physics.

\n\n

Well, anyway I could not make it work (it gave around 4900) so I fall back to the classical approach (which was a bit disappointing). At least, I learned a lot about TSP from this challenge (thanks to the winner for there nice publications !)

\n\n

And once again, congratulation !

" }, { "id": 454177, "authorName": "Farmár", "votes": 4, "postDate": "2019-01-11T07:57:20.763000", "content": "

And here is our kernel which scored 1514637.06 : https://www.kaggle.com/ppershing/lkh-bootstrap-rust-fine-tuning

" }, { "id": 453922, "authorName": "NighTurs", "votes": 4, "postDate": "2019-01-11T00:35:29.060000", "content": "

Congrats! Sorry for being annoying with hiding, at least I made it clear that I have something :)

\n\n

Interesting to see that penalty scheduling worked for you. I tried it and it didn't seem to get me anything. Maybe because my approach was to apply best opt I was able to find during iteration. Otherwise we seem to be pretty close: non-sequantial Kopt, kicks, recombinations.

" }, { "id": 457649, "authorName": "Shafay", "votes": 1, "postDate": "2019-01-17T21:27:19.023000", "content": "

Awesome.

" }, { "id": 454352, "authorName": "corleypc", "votes": 1, "postDate": "2019-01-11T13:53:30.980000", "content": "

A quick and dirty way to score a few hundred points over the LKH base tour is plugging a custom score function onto LKH to calculate penalized costs after each recombination. This is a sort of a Frankenstein solution which optimizes and recombines ignoring penalty, and then scores the full recombined penalized tour. This is only 20 lines of code added to LKH and I did it with no expectations, and still got an improvement of around 400 using GPX2 (local opt + GA). I suppose it would have worked similarly with IPT.

\n\n

Now I am curious how this would have turned out with slowly increasing penalties in the recombined score function.

" }, { "id": 454010, "authorName": "João Araújo", "votes": 1, "postDate": "2019-01-11T02:50:48.520000", "content": "

Congratulations to you and to the winners!! I will try to change the penalty schedule as you did just to know a bit more about how it affects the final score. Also thank you for sharing the merging tours method will take a look at Helsgaun's papers. This was my first kaggle competition and I learned a lot from everyone in the discussions / kernels!

" }, { "id": 453911, "authorName": "Ben Nye", "votes": 1, "postDate": "2019-01-11T00:21:04.697000", "content": "

One of your posts a while ago gave me the idea that led to my own O(k) move eval, which made my code so very much faster :)\nAnd that penalty schedule thing I never tried, or even thought of. That's pretty cool. The rest of it is essentially the same as what I've been doing, except for the 100x or more hardware advantage(I have a fairly old althlon2 with 3 cores.\nI did try doing multiple moves at once, I generated all possible k-opt moves that score no worse than a certain score and then applied a random non-overlapping subset of them. Then ran my usual k-opt over the resulting messed up tour to fix it and possibly improve. It worked, but since sometimes I got moves that were non-reversable and made the result worse(due to how I calculate neighborhoods) I switched to picking a random move and the closest non-overlapping other move from my list. That didn't do significantly better than just the random k-opt moves by themselves, so I went with that for my last 200 points or so. Meaning that I generated all k=3-5 opt moves that hurt my score by no more than some number of points, then sorted them by how little harm they did, then apply one, fix it(possibly improving) then apply another till I used them all up.

" }, { "id": 453908, "authorName": "JLouëdec", "votes": 1, "postDate": "2019-01-11T00:19:00.907000", "content": "

Congrats ! Really smart methods ;)

" }, { "id": 454807, "authorName": "thocevar", "votes": 2, "postDate": "2019-01-12T08:57:52.860000", "content": "

There's another cheap trick that I haven't seen mentioned yet. Build several tours from scratch, see which edges they agree on and fix them. I built 6 tours and surprisingly they agreed on 80% of edges. Dealing with just 20% of edges speeds things up significantly and opens up new possibilities. This might be more appropriate for a later stage so that you don't lock yourself in a local minimum but it worked very well for me.

" }, { "id": 454829, "authorName": "Vladimir Boza", "votes": 0, "postDate": "2019-01-12T10:14:58.713000", "content": "

Yep, this is pretty standard (although we did not use it). Typically you use edges of those tours as candidate edges and you try to find a best tour from that.

" }, { "id": 453975, "authorName": "blacksix", "votes": 2, "postDate": "2019-01-11T01:59:23.370000", "content": "

> Reduction to ATSP. Again, LKH did not want to talk to us after the reduction.

\n\n

I got it sort of working after a heavy session of LKH hacking but hardly managed to get any noteable improvements out of it in the end.

\n\n

The only good use for ATSP formulation I found was for feeding it into the original GPX2 code [1], it was very straightforward, just set n_cities to 20*N, write your distance function and main() and link with the rest of GPX2 code as is. Just kind of slow, 2 min/merge, but about 20-30% of the time it found something quite a bit better than IPT.

" }, { "id": 454119, "authorName": "Vladimir Boza", "votes": 1, "postDate": "2019-01-11T06:36:03.900000", "content": "

How much better was the GPX2?

" }, { "id": 454195, "authorName": "Farmár", "votes": 0, "postDate": "2019-01-11T08:22:55.410000", "content": "

Oh, that is clever, now I am going to bang my head that I did not try it out!. I just assumed that if ATSP doesn't work there is no point in pushing it further. Similarly, I was thinking about GPX2 for a lot and and I did not think it would yield better solutions because it can shift paths. But using GPX2 on top of ATSP might be just the trick that would avoid penalty shift problems ...

" }, { "id": 454353, "authorName": "blacksix", "votes": 1, "postDate": "2019-01-11T13:53:44.487000", "content": "

>How much better was the GPX2?

\n\n

In aggregate on average from what I see in my logs (I ran both GPX2 and my much faster IPT) about 20% better. Always at least as good as IPT and sometimes found improvements where IPT didn't

" }, { "id": 457121, "authorName": "Uche", "votes": -1, "postDate": "2019-01-17T01:19:06.603000", "content": "

Congratulations on your winnings. \nCan you please share your code with me in R language?\nI really need it. \nThanks

" }, { "id": 460792, "authorName": "Che", "votes": 0, "postDate": "2019-01-24T12:31:20.293000", "content": "

Thank. I tried trick with Penalty schedule, but it did not work for me.

" }, { "id": 458905, "authorName": "Kohul Raj", "votes": 0, "postDate": "2019-01-20T19:27:47.177000", "content": "

nice

" }, { "id": 457430, "authorName": "Anna Prus", "votes": 0, "postDate": "2019-01-17T12:02:05.257000", "content": "

Congratulations!

" }, { "id": 456873, "authorName": "Diego Bernardo", "votes": 0, "postDate": "2019-01-16T17:21:29.937000", "content": "

Congratulations!

" }, { "id": 456553, "authorName": "Single", "votes": 0, "postDate": "2019-01-16T03:11:17.437000", "content": "

Great

" }, { "id": 456396, "authorName": "Vishal Shivapujimath", "votes": 0, "postDate": "2019-01-15T18:22:39.237000", "content": "

Congratulations to all!

" }, { "id": 456291, "authorName": "Prakshaal", "votes": 0, "postDate": "2019-01-15T14:03:20.293000", "content": "

congratulations guys

" }, { "id": 456041, "authorName": "urmish", "votes": 0, "postDate": "2019-01-15T02:49:37.797000", "content": "

Thanks for beautiful write up, and sharing your wealth of knowledge!

" }, { "id": 455735, "authorName": "Mohammad Anas", "votes": 0, "postDate": "2019-01-14T13:10:38.800000", "content": "

well done

" }, { "id": 454919, "authorName": "Renze Struiksma", "votes": 0, "postDate": "2019-01-12T14:00:28.510000", "content": "

Congrats with 2nd place!

" }, { "id": 454798, "authorName": "Pai Buabthong", "votes": 0, "postDate": "2019-01-12T08:34:51.687000", "content": "

Congratulations! Nice work.

" }, { "id": 454356, "authorName": "Oleh Orlovskyi", "votes": 0, "postDate": "2019-01-11T14:03:40.267000", "content": "

Congratulation!\nGreat work.

" }, { "id": 454250, "authorName": "Ole Kröger", "votes": 0, "postDate": "2019-01-11T10:05:22.777000", "content": "

Would you mind to give us your final submission.csv?\nI'm interested whether our local approach finds something.

" }, { "id": 454262, "authorName": "Vladimir Boza", "votes": 4, "postDate": "2019-01-11T10:24:31.517000", "content": "

https://github.com/usamec/kaggle2018/blob/master/outputs/best.csv

" }, { "id": 454332, "authorName": "Ole Kröger", "votes": 0, "postDate": "2019-01-11T13:09:14.823000", "content": "

Okay as expected no improvement with our MIP code

" }, { "id": 454235, "authorName": "tombdx", "votes": 0, "postDate": "2019-01-11T09:28:07.293000", "content": "

Well done !\nReally nice work !

" }, { "id": 454203, "authorName": "Niall Maher", "votes": 0, "postDate": "2019-01-11T08:32:39.893000", "content": "

Great result, well done!

" }, { "id": 453965, "authorName": "fsguzi", "votes": 0, "postDate": "2019-01-11T01:48:26.860000", "content": "

Many thanks for sharing the solution!!\nI think many teams would agree that they had at least some common ideas with you but weren't able to push them as far as you guys did. In the end, it takes not only insights and skills to succeed, but also persistence and patience to keep on going as your team have demonstrated. Congratulations!

" }, { "id": 453931, "authorName": "", "votes": 0, "postDate": "2019-01-11T00:50:32", "content": "" }, { "id": 454124, "authorName": "", "votes": 1, "postDate": "2019-01-11T06:48:59.940000", "content": "" }, { "id": 454192, "authorName": "", "votes": 0, "postDate": "2019-01-11T08:18:47.220000", "content": "" }, { "id": 454193, "authorName": "", "votes": 3, "postDate": "2019-01-11T08:20:17.137000", "content": "" }, { "id": 454200, "authorName": "", "votes": 0, "postDate": "2019-01-11T08:28:07.317000", "content": "" }, { "id": 453916, "authorName": "", "votes": 0, "postDate": "2019-01-11T00:30:55.277000", "content": "" }, { "id": 453910, "authorName": "", "votes": 0, "postDate": "2019-01-11T00:20:06.423000", "content": "" }, { "id": 453900, "authorName": "", "votes": 0, "postDate": "2019-01-11T00:08:24.060000", "content": "" }, { "id": 453899, "authorName": "", "votes": 0, "postDate": "2019-01-11T00:06:53.903000", "content": "" }, { "id": 456635, "authorName": "", "votes": 0, "postDate": "2019-01-16T08:21:53.237000", "content": "" }, { "id": 456306, "authorName": "", "votes": 0, "postDate": "2019-01-15T14:29:03.777000", "content": "" }, { "id": 455746, "authorName": "", "votes": 0, "postDate": "2019-01-14T13:30:34.433000", "content": "" }, { "id": 455049, "authorName": "", "votes": 0, "postDate": "2019-01-12T20:22:47.873000", "content": "" }, { "id": 456610, "authorName": "", "votes": 0, "postDate": "2019-01-16T07:21:53.757000", "content": "" }, { "id": 456519, "authorName": "", "votes": 0, "postDate": "2019-01-16T00:38:26.910000", "content": "" }, { "id": 455630, "authorName": "", "votes": 0, "postDate": "2019-01-14T09:08:39.227000", "content": "" }, { "id": 455614, "authorName": "", "votes": 0, "postDate": "2019-01-14T08:49:44.217000", "content": "" }, { "id": 455467, "authorName": "", "votes": 0, "postDate": "2019-01-14T00:59:28.673000", "content": "" } ], "index": { "id": "77250", "title": "Our solution for 2nd place", "authorName": "Vladimir Boza", "commentCount": "59", "votes": "138", "postDate": "2019-01-11 00:16:54.037000" } }