Viewing a single comment thread. View all comments

External_Juice_8140 t1_ja8e485 wrote

Nobody can just answer this question.

Predict a set of data using the pre trained model. Then, predict the same set of data, manually by a human.

Compare the results. How accurate was the model compared to a human?

7

bluebolt789 t1_ja9mbyc wrote

Yeah I am not looking for a definitive answer, because as you said the only way to know for sure is to try and evaluate the performance.

I’m just trying to gauge whether it’s a “yeah very unlikely to work” or “seems promising, try it”. I have read an interesting paper that suggest filtering the sentences with a domain dictionary created from the training set before passing it to a pre-trained model. These kind of ideas is what I am looking for!

Unfortunately manually labeling the ticket data to get a benchmark is not something I can do, or that of course would be my first test too.

1