[R] Instance segmentation of paper fiber networks

Submitted by user11532 t3_yk591e in MachineLearning

I've been trying to perform instance segmentation of 3D images (tomograms) of paper fiber networks for my master thesis, but with no good results. Here is an example of a 2D slice of such an image:

https://preview.redd.it/v8ijpzxdhjx91.png?width=971&format=png&auto=webp&s=79e30e1bf80384630792693c7eefe171f264826d

I've been trying with the model presented here: https://arxiv.org/pdf/1901.01034.pdf. The authors have used it to segment glass fiber networks, which are less complex than paper fiber networks.

Any tips or comments are appreciated. For example, how well could one even expect to solve this problem?

I believe that the biggest difficulty lies in acquiring good enough training data. Using the method mentioned above, one performs instance segmentation on smaller patches of an image, and then merge the results to get the segmentation of the entire image. Here is an 80x80 patch of the above image:

https://preview.redd.it/07l3i3yghjx91.png?width=373&format=png&auto=webp&s=6daa3276bbfc5c6a38169f870ba08add9b294368

I have been manually annotating images like this (of dimensions 5x80x80), but even as a human it is very difficult to see what goes on in most of these images, making the annotations very inaccurate. Any tips/ideas on how to approach this issue?

Thank you!

19

Comments

You must log in or register to comment.

Syno7 t1_iurhbm9 wrote

I work on a very similar task in a large research program (on part is to segment carbon fibers in carbon fiber reinforced concrete). However, my task is a little easier than yours, as I have less fibers to segment than you.

Your initiall approach is correct: manual labelling. I know, this is a pain. I did 4 volumes using 512x512x20 voxels and then applied heavy offline augmentation for volumes using this project . In my case, it worked very well (visually). You can pm me, if you want to know more and see some results.

I used dragonfly by ORS to create the masked volume. They provide a non commercial license and the segmentation tool works pretty well (see their youtube playlist). You can also use their ai feature but I cannot tell you how well that works, cause I used the previous mentioned project.

9

clementiasparrow t1_iutfviu wrote

I know nothing about this field but here is an idea a bit out of the ordinary. I couldn’t help it :)

If you can get instance segmentation to work on fibers in a voxel of some small size, you could dice it into subvoxels and create a graph that links fiber segments to other segments of the same fiber in other neighboring voxels. Now it becomes a link-prediction problem on a graph. If you can associate some mask with the fiber segments through instace segmentation, you could feed a. the local voxel-coordinates, b. the local image data and c. the mask as node-data for each fiber-sement-node. Now you could try some fancy GNN link predictors on the local area of voxels and see if it will connect fiber segments correctly.

Hope you at least got a good laugh and good luck!

3

user11532 OP t1_iv06tvl wrote

Any ideas are appreciated. :)

If I understand you correctly, this would be a way to merge the results for smaller parts of an image? Unfortunately, I haven't really managed to achieve any good results on such smaller image parts either. For that reason I haven't tested the merging algorithm presented in the paper, so I don't know how well that one works.

1

clementiasparrow t1_iv12zuu wrote

Ok. If you cant annotate, it gets difficult … Maybe stack three depth layers (if there is more than one) and interpret as rgb channels. Then it may be easier to visually track fibers that go in and out.

1

blablanonymous t1_iuvr8jt wrote

I’m curious why this is a useful problem to solve. What is an application for this? I also, after looking for a minute, I cannot find more than a few instances of a fiber that goes under another fiber AND comes back up again. I think I would try non DL/classic segmentation techniques

2

user11532 OP t1_iv08x64 wrote

One application is to be able to follow the deformation of a fiber during e.g. tensile tests in future research, to learn more about the behavior of the microstructure of the material. I honestly don't know much about the details.

Apparently, classic segmentation techniques have been tested before, but haven't worked well enough. My thesis project is basically about trying a machine learning approach, and seeing how well that can work.

1

EnjoyableGamer t1_ivaz4qk wrote

It looks like you could generate fake yet convincing training data with blender or sthg, and some post processing. I would go that route

1