I also tried pre-trained tools like Spacy, but they only have a few fixed entity types they detect.

GPT-J is supposed to be quite good. Do you have a list of the types of entities you'd like to detect?

At this point I would like to imitate the example with position and company. It was taken from gpt-j btw. I thought neox is 3 times bigger so tried that first. Will run gptj and compare the results now.

Thank you

You can try extractive QA if you don't want to fine-tune it. Basically, create a QA pipeline and ask the same questions for different text

I'm doing something similar to your task. My plan is to use GPT-3's Text-divinci-003 as it can do this in Instruct mode without modification and then once I have a hundreds to thousands of examples then fine-tune GPT-J on Forefront.ai using what GPT-3 generated to hopefully cut costs by about 75%.

I am using GPT-3 for this kind of stuff, and fine-tuning small models on the data.

Do you mean free text questions? Like zero shot learning? Are there any examples of this?

How many samples do you use for fine turning?

The way you prompt assume there is a single entity for "name" so you catch "balmer" but not "bill gates".

Why not BIO tagging each token for each of the entity types?

Yeah. Here’s a HF tutorial on how to do this QA tutorial

Cool, will have a look! Do they list any models as being good at question answering?

I am not sure, if there is huge differences from one model to another. This is heavily depending on the training data that you can get.

I would suggest using some existing NER nodels and possibly fine tune them on your own data. Have a look at GENRE https://github.com/facebookresearch/GENRE

My task uses sentence pairs, and I have an efficient prompt that makes many pairs in one go. So in 5 hours I managed to generate 230K pairs. Cost $10. I plan to generate millions to "exfiltrate" more domain knowledge for the small and efficient models I am training downstream.