Hello everyone,
\nI'm trying to build an model which takes the input of shape (None,None,None,1) and this has been passed to another conv2d layer which results in the shape of (None,None,None,3). it's then passed into a efficientnet-b5 which results in shape of (None,NoneNone,2048). After this the inputs are needed to be passed to Conv3d. Conv3d accepts the 5d tensors and i have tried of reshaping the tensor into 5d but everything resulted me an error. So, is there any method you can help me to continue the work.
\nThanks in advance,
\nConquer_19.
If the Conv3D accepts 5d tensor, I assume it is [batch, y, x, z, channel] or some such thing. What is your [None,None,None,2048]? Sounds like maybe the embeddings (the output with the top removed from EN) of 2048, so it is something like [batch, slice, ?, 2048]? Please clarify. Without knowing what the dimensions are, it is hard to recommend a solution.
\nAlso, if those are the embeddings, I am not sure you want to do a convolution along the embedding dimension.
", "rawMarkdown": "If the Conv3D accepts 5d tensor, I assume it is [batch, y, x, z, channel] or some such thing. What is your [None,None,None,2048]? Sounds like maybe the embeddings (the output with the top removed from EN) of 2048, so it is something like [batch, slice, ?, 2048]? Please clarify. Without knowing what the dimensions are, it is hard to recommend a solution. \n\nAlso, if those are the embeddings, I am not sure you want to do a convolution along the embedding dimension.\n", "replies": [ { "id": 1918567, "postDate": "2022-08-29T17:10:37.390Z", "content": "@solverworld we can think those as like embeddings which are got after passing through the EN and the included _top = False which means the classification layers are freezed in the pre-trained network.
\nSo after that (batch_size, h,w,channels) are to be the initial inputs given to network but batch_size, h,w,c are not mentioned in inputs so they became (None, None, None, 2048) after passing through EN. So this shape is needed to be passed through the conv3d for another network input which is a 3d model. So it accepts 5d array.
\nHope the explanation is clear, but still some parts maybe missing
\nCan you tell me a solution for this to move forward with my work.
Thanks,
\nConquer _19
Is the output of the EN something like [batch, 10, 10, 2048]? Normally you would do a something like a GlobalPoolingAverage to get the 2048 embeddings. If you wanted to pass this to a 3D convolution as is you need to add a channel. Something like
\n# convert to 300x300x1\nx = tf.reshape(inp, ((-1, 10, 10, 2048,1))) \n#repeat to get 3 channels\ntf.repeat(inp2, repeats=[3] , axis=4, name='repeat') \nwould work in TensorFlow.
", "rawMarkdown": "Is the output of the EN something like [batch, 10, 10, 2048]? Normally you would do a something like a GlobalPoolingAverage to get the 2048 embeddings. If you wanted to pass this to a 3D convolution as is you need to add a channel. Something like\n```\n# convert to 300x300x1\nx = tf.reshape(inp, ((-1, 10, 10, 2048,1))) \n#repeat to get 3 channels\ntf.repeat(inp2, repeats=[3] , axis=4, name='repeat') \n```\nwould work in TensorFlow." }, { "id": 1934107, "postDate": "2022-09-11T06:42:38.463Z", "content": "What would be the inp2 in 4th line.
", "rawMarkdown": "What would be the inp2 in 4th line. " }, { "id": 1936005, "postDate": "2022-09-12T13:09:39.163Z", "content": "Sorry, cut-and-paste error from code. It should be the output of the reshape, so:
\n# convert to 10x10x2048x1\nx = tf.reshape(inp, ((-1, 10, 10, 2048,1))) \n#repeat to get 3 channels 10x10x2048x3\nx = tf.repeat(x, repeats=[3] , axis=4, name='repeat') \nNote that the first dimension is the batch size
", "rawMarkdown": "Sorry, cut-and-paste error from code. It should be the output of the reshape, so:\n```\n# convert to 10x10x2048x1\nx = tf.reshape(inp, ((-1, 10, 10, 2048,1))) \n#repeat to get 3 channels 10x10x2048x3\nx = tf.repeat(x, repeats=[3] , axis=4, name='repeat') \n```\nNote that the first dimension is the batch size" }, { "id": 1936133, "postDate": "2022-09-12T14:26:40.210Z", "content": "@solverworld I got that!
\nBut, I have another question regarding the ensembling of the models.
\nI have an Inception-Resnet3d model and this output is needed to be fed into a 2d efficientnet model. As we know that our images are of shape 512,512,3 shape and I have searched over the internet but didn't find the correct resource. can you help me with doing this?
Thanking you in Advance,
\nConuer_19.
It is not clear what you are trying to do.
\nA 2d (ed) EffNet would want an input of 512x512x3 as you say.
\nWhat is the output of your Resnet3d model? presumably it is some embedding layer, like 10x10x10x2048 maybe because it is 3D, or is it Globally Pooled into something like 2048 features, which you would mix into some number of predictions (8 maybe?).
What Inception-Resnet3d model are you using?
", "rawMarkdown": "It is not clear what you are trying to do.\nA ~~3d ~~2d (ed) EffNet would want an input of 512x512x3 as you say.\nWhat is the output of your Resnet3d model? presumably it is some embedding layer, like 10x10x10x2048 maybe because it is 3D, or is it Globally Pooled into something like 2048 features, which you would mix into some number of predictions (8 maybe?).\n\nWhat Inception-Resnet3d model are you using?\n" }, { "id": 1936708, "postDate": "2022-09-13T02:07:27.097Z", "content": "Hello @solverworld
\nWe know that commonly a 3d network learn in 3 dimensions and those feature are sent to 2d network and finally flattened for classification. This is what I know about 3d and 2d network.
So as you said that a 3d model output is 4 dimensional like an embedding. So this need to be reshaped in a way that it can be used to feed into a 2d pre-trained model and then finally a classification layer is used for classification.
\nWhy 3d to 2d?
\n Features which are learnt by 3d network is more when compared to a 2d model so those features are sent to a 2d model for extraction of more significant features in the 3d model output.
3d model which I'm using is efficientnet B5 and 2d model is inception ResNet-50 model. So can you explain me how can this can be achieved in implementing this ?
\nThanking you in advance,
\nConquer_19.
Sorry, I am not exactly understanding what you are trying to do. Perhaps someone else can help. I am curious about it, though.
\nDo you have a reference for an example of a 3D network output being sent to a 2D network?
I don't have any examples to do so I'm experimenting on my own, that's why I sought help from you!
", "rawMarkdown": "I don't have any examples to do so I'm experimenting on my own, that's why I sought help from you!", "votes": 1 } ] }, { "id": 1918100, "postDate": "2022-08-29T10:19:44.250Z", "content": "Hello everyone,
\nI'm trying to build an model which takes the input of shape (None,None,None,1) and this has been passed to another conv2d layer which results in the shape of (None,None,None,3). it's then passed into a efficientnet-b5 which results in shape of (None,NoneNone,2048). After this the inputs are needed to be passed to Conv3d. Conv3d accepts the 5d tensors and i have tried of reshaping the tensor into 5d but everything resulted me an error. So, is there any method you can help me to continue the work.
\nThanks in advance,
\nConquer_19.
If the Conv3D accepts 5d tensor, I assume it is [batch, y, x, z, channel] or some such thing. What is your [None,None,None,2048]? Sounds like maybe the embeddings (the output with the top removed from EN) of 2048, so it is something like [batch, slice, ?, 2048]? Please clarify. Without knowing what the dimensions are, it is hard to recommend a solution.
\nAlso, if those are the embeddings, I am not sure you want to do a convolution along the embedding dimension.
", "votes": 0, "replies": [ { "id": 1918567, "author_name": "Collide_Conquer_19", "author_url": "", "post_date": "2022-08-29T17:10:37.390000", "content": "@solverworld we can think those as like embeddings which are got after passing through the EN and the included _top = False which means the classification layers are freezed in the pre-trained network.
\nSo after that (batch_size, h,w,channels) are to be the initial inputs given to network but batch_size, h,w,c are not mentioned in inputs so they became (None, None, None, 2048) after passing through EN. So this shape is needed to be passed through the conv3d for another network input which is a 3d model. So it accepts 5d array.
\nHope the explanation is clear, but still some parts maybe missing
\nCan you tell me a solution for this to move forward with my work.
Thanks,
\nConquer _19
Is the output of the EN something like [batch, 10, 10, 2048]? Normally you would do a something like a GlobalPoolingAverage to get the 2048 embeddings. If you wanted to pass this to a 3D convolution as is you need to add a channel. Something like
\n# convert to 300x300x1\nx = tf.reshape(inp, ((-1, 10, 10, 2048,1))) \n#repeat to get 3 channels\ntf.repeat(inp2, repeats=[3] , axis=4, name='repeat') \nwould work in TensorFlow.
", "votes": 0, "replies": [] }, { "id": 1934107, "author_name": "Collide_Conquer_19", "author_url": "", "post_date": "2022-09-11T06:42:38.463000", "content": "What would be the inp2 in 4th line.
", "votes": 0, "replies": [] }, { "id": 1936005, "author_name": "SolverWorld", "author_url": "", "post_date": "2022-09-12T13:09:39.163000", "content": "Sorry, cut-and-paste error from code. It should be the output of the reshape, so:
\n# convert to 10x10x2048x1\nx = tf.reshape(inp, ((-1, 10, 10, 2048,1))) \n#repeat to get 3 channels 10x10x2048x3\nx = tf.repeat(x, repeats=[3] , axis=4, name='repeat') \nNote that the first dimension is the batch size
", "votes": 0, "replies": [] }, { "id": 1936133, "author_name": "Collide_Conquer_19", "author_url": "", "post_date": "2022-09-12T14:26:40.210000", "content": "@solverworld I got that!
\nBut, I have another question regarding the ensembling of the models.
\nI have an Inception-Resnet3d model and this output is needed to be fed into a 2d efficientnet model. As we know that our images are of shape 512,512,3 shape and I have searched over the internet but didn't find the correct resource. can you help me with doing this?
Thanking you in Advance,
\nConuer_19.
It is not clear what you are trying to do.
\nA 2d (ed) EffNet would want an input of 512x512x3 as you say.
\nWhat is the output of your Resnet3d model? presumably it is some embedding layer, like 10x10x10x2048 maybe because it is 3D, or is it Globally Pooled into something like 2048 features, which you would mix into some number of predictions (8 maybe?).
What Inception-Resnet3d model are you using?
", "votes": 0, "replies": [] }, { "id": 1936708, "author_name": "Collide_Conquer_19", "author_url": "", "post_date": "2022-09-13T02:07:27.097000", "content": "Hello @solverworld
\nWe know that commonly a 3d network learn in 3 dimensions and those feature are sent to 2d network and finally flattened for classification. This is what I know about 3d and 2d network.
So as you said that a 3d model output is 4 dimensional like an embedding. So this need to be reshaped in a way that it can be used to feed into a 2d pre-trained model and then finally a classification layer is used for classification.
\nWhy 3d to 2d?
\n Features which are learnt by 3d network is more when compared to a 2d model so those features are sent to a 2d model for extraction of more significant features in the 3d model output.
3d model which I'm using is efficientnet B5 and 2d model is inception ResNet-50 model. So can you explain me how can this can be achieved in implementing this ?
\nThanking you in advance,
\nConquer_19.
Sorry, I am not exactly understanding what you are trying to do. Perhaps someone else can help. I am curious about it, though.
\nDo you have a reference for an example of a 3D network output being sent to a 2D network?
I don't have any examples to do so I'm experimenting on my own, that's why I sought help from you!
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1918505": "If the Conv3D accepts 5d tensor, I assume it is [batch, y, x, z, channel] or some such thing. What is your [None,None,None,2048]? Sounds like maybe the embeddings (the output with the top removed from EN) of 2048, so it is something like [batch, slice, ?, 2048]? Please clarify. Without knowing what the dimensions are, it is hard to recommend a solution. \n\nAlso, if those are the embeddings, I am not sure you want to do a convolution along the embedding dimension.\n", "1918100": "Hello everyone,\n\nI'm trying to build an model which takes the input of shape (None,None,None,1) and this has been passed to another conv2d layer which results in the shape of (None,None,None,3). it's then passed into a efficientnet-b5 which results in shape of (None,NoneNone,2048). After this the inputs are needed to be passed to Conv3d. Conv3d accepts the 5d tensors and i have tried of reshaping the tensor into 5d but everything resulted me an error. So, is there any method you can help me to continue the work.\n\nThanks in advance,\nConquer_19." } }