Comments

You must log in or register to comment.

ok531441 t1_jbglnjg wrote

What is your ML model supposed to do?

1

jobeta t1_jbgnfsq wrote

> a model that can generate more accurate embeddings in general

What do you mean by this? Accuracy is an evaluation metric that doesn't really mean anything "in general" but only with respect to a specific prediction being made. I think this is a slightly ill-posed question

1

bubudumbdumb t1_jbgxlt0 wrote

In my experience NLP models are released as public science when trained on datasets scraped from the web.

Things like "models that solve this problem in finance" or "datasets of annotated football matches" or "medical records of millions of people" are not likely to follow the publication patterns of open science.

If you have a model like the one you asked for you likely have a way to profit from it and you are unlikely to publish it.

10

Toilet_Assassin t1_jbh2499 wrote

There are teams of people working on and refining this in the mortgage and quantitative finance industries, it is highly unlikely that they would be willing to open source it for competitors to compare notes and take market share.

6

jobeta t1_jbhg8xy wrote

Right. But to be able to assess this you need to define a task and evaluate your model’s performance to perform that task. Embedding accuracy cannot be discussed completely in the ether. Even the most general comments you will read about a model beating another, will refer to that new model performing better for specific tasks on benchmark datasets.

It would be a lot easier to help you if you explained what you are trying to accomplish that requires “higher accuracy”

1

eyeofthephysics t1_jbhu9d4 wrote

First I would say there exist versions of FinBERT which aren't just tuned for sentiment analysis. There are two groups who developed models they called FinBERT https://arxiv.org/abs/1908.10063 and https://arxiv.org/abs/2006.08097. The first paper's model can be fond here and is tuned for seniment analysis but the second model, found here, was pre-trained using masked language modelling on general financial text. So that one can be fine-tuned for other tasks.

Since you're interested in text embeddings, you may also be interested in this paper https://arxiv.org/pdf/2111.00526.pdf. The focus of that paper is sentiment analysis, but the general idea of using a sentence-BERT model to get better textual embeddings (as opposed to using vanilla BERT) should hold more generally.

2

hrishikamath t1_jbii1e4 wrote

Unrelated question: what dataset are you using for the task?

1