[P] We released a new open-source library for metric learning!

Submitted by Zestyclose-Check-751 t3_xxf8i8 in MachineLearning

Hi everyone! My comrades and I have released the PyTorch-based library for representation learning named OpenMetricLearning. I kindly ask you to support us by putting a star on GitHub: https://github.com/OML-Team/open-metric-learning!

OML is an excellent choice when you want to extract features from the images for further clustering, searching, and so on. OML provides a ready-to-use zoo of the models pre-trained in a Self-Supervised way (CLIP, DINO, MoCo) and the pipeline for training / finetuning models on your data.

The training part is built around different variations of triplet loss with various strategies of sampling batches, mining triplets, and using memory banks. The validation part measures retrieval-like metrics when you use a subset of the data as the searching queries and the rest as a searching index (gallery).

You can use OML dealing with PyThon objects or as a non-code solution when you only prepare data in the needed format and write the config file.

We support multi-GPU setup, provide a set of examples on public datasets, well-made documentation, and FAQ.

194

Comments

You must log in or register to comment.

fernandocamargoti t1_irc04s7 wrote

Very interesting. How would you compare your project to https://github.com/KevinMusgrave/pytorch-metric-learning?

11

Zestyclose-Check-751 OP t1_irc75zv wrote

Initially, in our project, we decided to start using PML, but we found it inconvenient and we even were not able to complete our pipeline because we struggled with the validation setup. The following is my IMHO:

  1. The design & documentation of PML is not clear and intuitive
  2. PML provides some Trainer, but even the author does not use it in his examples (which is also a sign of a not perfect design) and he writes train and test functions by himself. As a side effect, it means that I see no easy way to use it with DDP without rewriting the examples' code, which is a big issue.

I think it may be related to the fact that PML was not designed as an open-source project from its beginning and there was no strict plan for the whole library. In contrast, when we started working on OML we already realised the whole structure, which helped us a lot. Anyway, I believe PML's author made a big contribution to the metric learning sphere, especially with his paper "A Metric Learning Reality Check".

UPD. I also forget to mention that we have a ZOO of pre-trained models with automatic checkpoints downloading (as in torchvision), so it's also a big advantage and allows us to start experimenting immediately. I am talking about models like MoCo, CLIP, DINO

UPD. We added the comparison with PML to FAQ and also added examples of the usage our library with losses from PML.

14

I-am_Sleepy t1_irddanm wrote

Is it compatible with PML, or various deep metric learning method need to be re-implement along with OML?

2

Zestyclose-Check-751 OP t1_iri3ymj wrote

I guess you are mostly talking about different losses in PML, right?
The easiest way to work with these losses in our library is to take one of our examples and just replace the criterion object. I think we will add a few examples of this in the future.

2

CommunismDoesntWork t1_irdbsf0 wrote

Does it support SupCon? That's the only network I've had any success with in production

2

Zestyclose-Check-751 OP t1_ire4qd5 wrote

We don't have the implementation of this particular paper (SupCon). But I would say our current pipeline is somehow similar because you may consider triplet loss and its variations also as a Supervised Contrastive method (because we need supervised labels to form the triplets, and the triplet itself has a contrastive nature since it works with positive and negative pairs).
So, you can try our models and see, if they work better or not :)

1

visarga t1_irdctls wrote

Is the library specific to CV? How does it compare to https://sbert.net/docs/package_reference/losses.html

2

Zestyclose-Check-751 OP t1_iri3e1m wrote

From the very beginning, we built the library in a way it can work with both -- CV and NLP. But right now there are only CV-related examples since it's our main focus. Generally, you only need to implement your own dataset returning encoding texts and a model knowing how to deal with that. We gonna add such an example in the future (seems like it will be similar to the content under your link).

2

Consistent-Archer-99 t1_irrerda wrote

Nice to see documentation and not just readme

Have you tried using this in some recent Kaggle competitions?

2

Zestyclose-Check-751 OP t1_irstd6e wrote

>Consistent-Archer-99

Thank you!

As for Kaggle, not yet. But from time to time they host competitions suitable for us like Google Landmark Detection Challange and others.

2