I don't know why I didn't ask this before,
\nbut why doesn't the provided metadata match what we have as a train and valid set ? It would be interesting and most likely useful to have temporal and spacial context for each image.
I don't know if this is a known issue or if it's not an issue at all and i'm the only one who had to make my own metadata but I'd rather ask than not know.
", "messages": [ { "id": 2307649, "postDate": "2023-06-18T11:00:56.683Z", "content": "I don't know why I didn't ask this before,
\nbut why doesn't the provided metadata match what we have as a train and valid set ? It would be interesting and most likely useful to have temporal and spacial context for each image.
I don't know if this is a known issue or if it's not an issue at all and i'm the only one who had to make my own metadata but I'd rather ask than not know.
", "rawMarkdown": "**I don't know why I didn't ask this before,**\nbut why doesn't the provided metadata match what we have as a train and valid set ? It would be interesting and most likely useful to have temporal and spacial context for each image.\n\n*I don't know if this is a known issue or if it's not an issue at all and i'm the only one who had to make my own metadata but I'd rather ask than not know.*", "votes": 6 }, { "id": 2311796, "postDate": "2023-06-21T13:05:35.883Z", "content": "The record_id field matches the directories as far as I can tell. The first record_id in train_metadata.json, which is \"3283699311323360356\", is also a directory inside the train directory with the appropriate .npy files .. and so on.
", "rawMarkdown": "The record_id field matches the directories as far as I can tell. The first record_id in train_metadata.json, which is \"3283699311323360356\", is also a directory inside the train directory with the appropriate .npy files .. and so on.", "votes": 1, "replies": [ { "id": 2311818, "postDate": "2023-06-21T13:15:46.917Z", "content": "yes, but if you read it using Pandas record_id is broken - you have to declare data type in Pandas for this column. I published below how to read json file properly.
", "rawMarkdown": "yes, but if you read it using Pandas record_id is broken - you have to declare data type in Pandas for this column. I published below how to read json file properly. ", "votes": 3 } ] }, { "id": 2310538, "postDate": "2023-06-20T12:45:50.883Z", "content": "Can you elaborate on what doesn't match?
", "rawMarkdown": "Can you elaborate on what doesn't match?", "replies": [ { "id": 2310745, "postDate": "2023-06-20T15:25:47.323Z", "content": "Hello @aaronsarna,
\nThe record_ids from the metadatas are nowhere to be found in the given folders. I tried many things to see if it was a string/int comparaison problem but it is not. Such notebooks are made in response but I don't think that's really optimal for this comp: https://www.kaggle.com/code/takaito/gr-icrgw-make-table-data
I am also interested in this. I just started so maybe I misunderstand, but shouldn't record_ids from train_metadata.json match the subfolder names of train/ ?
", "rawMarkdown": "I am also interested in this. I just started so maybe I misunderstand, but shouldn't record_ids from train_metadata.json match the subfolder names of train/ ?", "votes": 1 }, { "id": 2311814, "postDate": "2023-06-21T13:13:13.877Z", "content": "Everything is ok. You have to declare data type for long numbers (record_id) in Pandas reading json file.
\n = {: str} \n\n = pd.read_json(, dtype=data_types)\n = pd.read_json(, dtype=data_types)\nty, that helps
", "rawMarkdown": "ty, that helps", "votes": 1 }, { "id": 2312423, "postDate": "2023-06-21T21:55:09.230Z", "content": "thx ! that will for sure help
", "rawMarkdown": "thx ! that will for sure help", "votes": 1 }, { "id": 2343221, "postDate": "2023-07-13T13:43:26.560Z", "content": "I was having the same problem.
\nThanks for sharing.
The record_id field matches the directories as far as I can tell. The first record_id in train_metadata.json, which is \"3283699311323360356\", is also a directory inside the train directory with the appropriate .npy files .. and so on.
", "votes": 1, "replies": [ { "id": 2311818, "author_name": "Remek Kinas", "author_url": "", "post_date": "2023-06-21T13:15:46.917000", "content": "yes, but if you read it using Pandas record_id is broken - you have to declare data type in Pandas for this column. I published below how to read json file properly.
", "votes": 3, "replies": [] } ] }, { "id": 2310538, "author_name": "Aaron Sarna", "author_url": "", "post_date": "2023-06-20T12:45:50.883000", "content": "Can you elaborate on what doesn't match?
", "votes": 0, "replies": [ { "id": 2310745, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-06-20T15:25:47.323000", "content": "Hello @aaronsarna,
\nThe record_ids from the metadatas are nowhere to be found in the given folders. I tried many things to see if it was a string/int comparaison problem but it is not. Such notebooks are made in response but I don't think that's really optimal for this comp: https://www.kaggle.com/code/takaito/gr-icrgw-make-table-data
I am also interested in this. I just started so maybe I misunderstand, but shouldn't record_ids from train_metadata.json match the subfolder names of train/ ?
", "votes": 1, "replies": [] }, { "id": 2311814, "author_name": "Remek Kinas", "author_url": "", "post_date": "2023-06-21T13:13:13.877000", "content": "Everything is ok. You have to declare data type for long numbers (record_id) in Pandas reading json file.
\n = {: str} \n\n = pd.read_json(, dtype=data_types)\n = pd.read_json(, dtype=data_types)\nty, that helps
", "votes": 1, "replies": [] }, { "id": 2312423, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-06-21T21:55:09.230000", "content": "thx ! that will for sure help
", "votes": 1, "replies": [] }, { "id": 2343221, "author_name": "moaiinthesky", "author_url": "", "post_date": "2023-07-13T13:43:26.560000", "content": "I was having the same problem.
\nThanks for sharing.