Comments

You must log in or register to comment.

External_Juice_8140 t1_ja8e485 wrote

Nobody can just answer this question.

Predict a set of data using the pre trained model. Then, predict the same set of data, manually by a human.

Compare the results. How accurate was the model compared to a human?

7

bluebolt789 t1_ja9mbyc wrote

Yeah I am not looking for a definitive answer, because as you said the only way to know for sure is to try and evaluate the performance.

I’m just trying to gauge whether it’s a “yeah very unlikely to work” or “seems promising, try it”. I have read an interesting paper that suggest filtering the sentences with a domain dictionary created from the training set before passing it to a pre-trained model. These kind of ideas is what I am looking for!

Unfortunately manually labeling the ticket data to get a benchmark is not something I can do, or that of course would be my first test too.

1

Donno_Nemore t1_jabbex8 wrote

The risks would be a misalignment between the training data and your business data. Business apps with clever acronyms that have strong associations with sentiment would be the major risk.

The value-add of such a system is unclear at best. The IT help desk ticketing systems I am familiar will all have a review/feedback component. The IT staff already know all the A-holes. Are you trying to empirically prove who the A-holes are?

2

bluebolt789 t1_jadtupr wrote

Yes, that’s what I was thinking too.

Thank you for your input!

1

professorlust t1_jacqyy8 wrote

Have you looked at any of the sentiment analysis work from the last 5 years?

1

bluebolt789 t1_jad0st0 wrote

I don’t get your point, could you be more specific please?

1

professorlust t1_jadgwa2 wrote

Sentiment analysis is pretty “standard” NLP ml at this point.

There’s literally 1000s of tutorials, medium articles, YouTube video etc on the topic.

That’s without getting into more academic/research focused articles

1

bluebolt789 t1_jadmcuq wrote

Yeah but most of it is supervised (not necessarily a neural network, even more classic approaches) or pre-trained on tweets/product reviews. I haven’t found anything pre-trained on it support tickets.

Am I missing something obvious?

1

2blazen t1_jadrdgu wrote

I think what he means is your question is beneath the sub's standards lol

You may have more luck googling specifically about cross domain sentiment analysis, asking chatgpt, or asking it on r/MLQuestions or r/learndatascience

2

bluebolt789 t1_jads2m6 wrote

Oh ok.

I didn’t think this was such an obvious question to ask since I don’t have labeled data, but I’ll take your advice…

Btw, since it’s an easy question, if you have any input (besides googling or cross posting) to give me, happy to hear it!

1