[D] What techniques can I use to tell if a problem is likely enough to be solved by ML so as to justify compiling the dataset?

Submitted by SnuggleWuggleSleep t3_10vgw7s in MachineLearning

I have a problem that if I solve it with ML, I'll make money, with an outside chance of it being a lot of money. Compiling a dataset will take significant work.

Are there any techniques that I can apply to let me know if this is going to be worth it? Perhaps there are certain hallmarks that a problem would have if it is likely to be solvable with available data? Maybe something I can do with a small initial dataset?

Thanks.

1

Comments

You must log in or register to comment.

[deleted] t1_j7hlj4r wrote

Without sharing much details about the specific problem its going to be difficult to give proper feedback/advice.

Some questions you can ask yourself:

- Can a human solve the problem? How skilled does the human have to be?

- Do you think you will need fancy architectures to train a model or is assembling the data the hard part and modelling will be easy? How easy? Basically the question is is assembling the data the risk or is modelling the risk?

- Have others tried? Why are you so convinced that you can make money solving the problem? If you are so convinced, then why have others not tried?

7

dfcHeadChair t1_j7ko3pf wrote

  1. What is your best guess at how much money you'll make?
  2. Divide that by your best guess at the amount of time, money, and effort it will take you to compile the dataset.
  3. Do the division and ask yourself if it's worth it.

The hard math is going to get you your answer. You may be able to do some fancy correlation mapping depending on the models you think will solve the problem and what data you will need. The trouble with the "shortcut" route is two-fold:

  1. It may take you longer that to do the three steps above.
  2. You might not get an accurate answer.
1

Feeling_Card_4162 t1_j7kp7rq wrote

This is a good way to get an idea of the financial benefit but it’s also important to think about the knowledge you’ll gain and how much other people would benefit from it when deciding whether to continue or not. There is more to determining if something is worth your time than just money.

2

dfcHeadChair t1_j7ku6j3 wrote

Yep I agree. If you learn a transferable skill that should be taken into account.

I was framing the problem in the same terms OP did.

1

mjaltthrowaway t1_j7oqeo1 wrote

I suppose the first question that comes to mind for me is: what problems exist in the vaccine world (besides poor customer sentiment) that ML/AI could potentially solve or enhance? Personalization? - to someone.

I suppose the first question that comes to mind for me is: what problems exist in the vaccine world (besides poor customer sentiment) that ML/AI could potentially solve or enhance? Personalization?

Maybe OP can use a similar method of analysis.

1

SnuggleWuggleSleep OP t1_j7qyhhs wrote

Reads like an ai that needs more training wrote this.

1

mjaltthrowaway t1_j7r9zon wrote

No it's real human. For some reason it didn't take in my first paragraph. What I was saying was that I own a high value domain related to vaccines and I was also thinking something along the similar lines of what OP posted.

And then the rest of my post was about starting from first principles and asking the simple question what problem does this machine learning website or app solve.

1