[D] To those of you who quit machine learning, what do you do now?

Submitted by nopainnogain5 t3_11ryvao in MachineLearning

I'm currently doing my master's degree and have been set on a DL-related career for a while. But recently I noticed it doesn't bring me joy.

Coming up with architectures that randomly work/don't work, tuning parameters, waiting for days till the model is trained... the level of uncertainty is just too high for me. Because of that, I don't feel productive working on it and I'm slowly considering switching to another IT field.

For those of you who quit machine learning (especially deep learning):

  1. What did you switch to?
  2. Are you satisfied with your new job? (Is it stressful/intellectually challenging? Is it possible to keep it 9-5?)
  3. How to ensure a smooth transition to that field?

Thanks in advance!

___
PS I know machine learning isn't all about deep learning, but in my current subfield (computer vision), mostly deep learning is used.

55

Comments

You must log in or register to comment.

[deleted] t1_jcawoko wrote

[deleted]

53

nopainnogain5 OP t1_jcax5q1 wrote

High chance you're right

10

I_will_delete_myself t1_jcblldw wrote

People can't do deep learning or AI without the tools to make them happen. Imagine how complicated the data collection methods at the scale of Terabytes of data and cleaning it. People also need to annotate data for it to work which requires software to get it done in a cost effective manner.

9

chef1957 t1_jcbzcfe wrote

I agree with this. I used to do Machine Learning but shifted to working at an open-source company (Argilla), to be more involved with data quality instead of overfitting and building models without enough data.

6

nopainnogain5 OP t1_jcc8xe3 wrote

How would you call your position? Is it software engineering / data engineering or yet something else?

3

chef1957 t1_jcenkpv wrote

It is more software engineering working on the core package and creating educational content (videos, presentations etc.) about getting from no data to a decent baseline model. Combining both, really helps to understand what people struggle with.

1

haljm t1_jcazfgr wrote

Try more applied fields!

Not in industry (PhD student), but used to do deep learning type research with a computer vision background. I'm now working on applying ML for computer systems. At least 90% of my actual research work is building the system, collecting the data, and deciding how to frame the problem; for that last 10%, essentially anything reasonable will do. The catch is that if you don't have a solid ML background, you might completely miss that last 10% and never realize!

It's much more satisfying since if you're working on applications, you likely have potential downstream use cases lined up (that's what collaborators are for!). You're also probably the only person working on this specific problem formulation, so you don't need to worry about beating SOTA and squeezing out that last 5% -- your data-driven algorithm is 2x better than a traditional approach, so does it really matter?

35

nopainnogain5 OP t1_jcc9his wrote

In case I'd like to dive into something along these lines, how such positions tend to be called?

3

haljm t1_jccw6ef wrote

I'm probably not the most qualified since I'm a PhD student, but in my experience it's generally an ML position where the description requires you to have some domain knowledge or a position title in the application domain that specifies that you're doing ML.

3

prettyyyyprettyygood t1_jce8ye3 wrote

I think "Machine Learning Engineer" or even just "Data Scientist". Most of the jobs out there are probably exactly like this. In fact there's a shortage of people who want to do the 'boring' stuff, compared to people who want to be researchers. If you are good at MLops and implementing solutions that you know will work, you're super valuable.

1

Mikkelisk t1_jcc4spe wrote

> Coming up with architectures that randomly work/don't work, tuningparameters, waiting for days till the model is trained... the level ofuncertainty is just too high for me.

Good news! You say you work in computer vision. There's a high chance that in practice you'll mostly use off-the-shelf solutions and most of your actual time will be spent gathering data:)

16

PsyEclipse t1_jcd8tig wrote

I actually went the other way. My background is FEDGOV hurricane scientist who now works at a popular weather app after teaching myself the ML part. We do a lot of statistical modeling in meteorology already (mainly bias correction and downscaling), and I made the decision to move away from dynamics-based research to statistics-based applications since it's about to take over our field as well. I'd argue meteorology is one of the original big data problems, but I guess that's a topic for a different day.

Anyway, there are a bunch of meteorology companies that are using AI/ML for a variety of things. Since you said you were doing computer vision, identifying things in satellite imagery (Sentinel-2, LandSat, maybe even GOES) is very popular right now. It doesn't get you away from the core issues of tuning models, but maybe a different focus might work for you...?

I would just warn you that the pay in the field of meteorology sucks relative to BIG TECH.

7

tcho187 t1_jccfe8j wrote

You can pivot to an adjacent role within ML like MLOps or Data Engineering. That’s what I did. I didn’t like waiting an entire day for a model to run and then having to fix it late at night so I can do another iteration throughout the night. So now I build machine learning platforms which is more traditional software engineering and comes with predictable outcomes. Your knowledge about ML is still valuable.

6

nopainnogain5 OP t1_jccjypz wrote

Staying up late to run another iteration sounds very familiar haha. Are you happy with your change or did it turn out that the grass is greener on the other side?

3

tcho187 t1_jce1you wrote

I’m happy with the change. MLOps is an exciting field right now. There’s a lot of companies that need people who knows ML and software engineering.

2

Senior_Buy445 t1_jcc5jio wrote

I know exactly what you are talking about. Perhaps you could train a model to tell you what to do next? We have folks all the time believing that ML/DL will solve any problem… :-)

5

LeDebardeur t1_jchfnr0 wrote

1 - Data engineering and DevOps
2 - It's way less stressful than ML because you have really clear requirements ( I need to get data from a source in a certain target with those constraints ). This sometimes can be challenging due to business requirements (Time, consistency, and monitoring those pipelines) but I find it better than go into a project where I don't even know if it will be feasible or no.
3 - I was a good programmer before I got to ML, so for me it was like I switched back to what I used to do, so it was not a big deal. ( My curriculum was a lot of software engineering / managing networks and pure dev)

3

nightshadew t1_jcc56x4 wrote

You can do a lot in other subfields. Even basic churn predictions are something valuable for a lot of firms. Are you sure you’re not swept by the DL hype yourself?

2

chhaya_35 t1_jce7fx3 wrote

I jumped from ML ( 2 years) to MLoPs ( 1 year) then to backend engineering (1 year) . Now I am planning to come back to ML. Mostly worked with CV was bit bored since we mostly used off the shelf models, but other endeavours turned out to be more boring for me . It's my personal opinion, there's nothing wrong with the fields. I don't find it intellectually challenging. For me it was bit stressful because it seems there were lot of systems in place which created complexities. Coming back to ML, because I feel it satisfying ( although it has its cons) but I do enjoy it more than the other fields that I have tried.

2

spadel_ t1_jcqdxi4 wrote

I went into a quant research position at a prop trading firm and am very happy about that decision. While unfortunately I have not been using any deep learning so far, it is a lot of stats and machine learning. Also there are some interesting applications of physics informed neural networks, which I want to look into at some point. It is definitely fun to work on problems now that just need to be solved in a creative way instead of continuously having to come up with new research ideas.

2

neato5000 t1_jccszzz wrote

I've had jobs that were similar to what you describe. My current job contains less by way of tiny tweaks to massive DL models and more feature engineering and engineering in general which suits me better.

My slightly warm take is that DL at the coal face in industry feels very random, very time consuming, and as a result a bit demoralising. More power to you if you have the knack for it, and enjoy it, it's just not super my bag.

1

darkshenron t1_jcctjv4 wrote

A lot of NLP specialists are having an existential crisis since chatgpt and now gpt4 have shown to solve pretty much any problem you throw at it. Some of us have used our knowledge of productising large language models to move into MLOps and data engineering where the outcome is more deterministic

1

nopainnogain5 OP t1_jccvvcj wrote

What would you say are the key skills I should sharpen to work as a data engineer? (ofc I can google it, but I'm curious how it looks like from your experence)

1

GFrings t1_jcd941i wrote

Manage people who still do machine learning. Motivating scientists isn't unlike the constant iterative grind of matching the right reward function to the right model.

1

Snoo58061 t1_jcdrz12 wrote

Never quite made it to working on ML professionally. This week I'm a 'Data Engineer'.

1

nopainnogain5 OP t1_jcdshkg wrote

I like how you phrased it "this week"

1

Snoo58061 t1_jcdtg08 wrote

Well I started of doing my time in the Data Warehouse. I was hoping I could retire to the Data Lakehouse. Now it's being drained by a Data Pipeline and the rest is slowly floating off into The Cloud.

Amusingly they recently changed my team name to Data Integration and Engineering. The DIE team.

4

SnooPears7079 t1_jcaxru5 wrote

Would you say “coming up with architectures that randomly work / don’t work” is a shortcoming of your understanding or of the field in general?

I’m asking because I’m thinking about doing the opposite switch right now - ML interests me deeply and I’m currently in standard cloud development.

0

pwsiegel t1_jcbjf82 wrote

It's a property of the field in general - there is very little theory to guide neural architecture design, just some heuristics backed by trial-and-error experimentation. Deep learning models are fun, but in practice you spend a lot of your time trying to trick gradient descent into converging faster.

22

currentscurrents t1_jcclq02 wrote

The whole thing seems very bitter lesson-y and I suspect in the future we'll have a very general architecture that learns to reconfigure itself for the data.

5

nopainnogain5 OP t1_jccd7vm wrote

I was wondering if this has something to do with lack of experience. And from what I've heard indeed the more you experiment with the models, the better you understand what helps when, to some extent.

The thing is, a neural network still remains a black box, as the number of parameters is too big to fully understand what is happening. It is an empirical study mostly. So you choose your architecture, test, change hyperparameters, test, change the architecture, test, change some other parameters, test, and so on. You can't be sure your model will work properly right away and it may take lots of iterations. With larger models which take long to train it might be overwhelming.

Of course, it might be different in your case, you can start with some toy examples, and if you still like it, I'd recommend playing with larger networks.

1

loadage t1_jccdzk2 wrote

That was my first thought too. I'm about to finish my masters program and I spent the first half thinking that it was just hyperparameter tuning, until I sat down and learned the math and theory. Now it's so much more interesting and explainable. That random tuning is now much more calibrated from experience and understanding the theory. (As of now), I could easily make a career out of this, because it's not random and simple optimization. Plus, the field is so hot right now, that it's unreasonable to assume that what data scientists do now is what they will do in 5, 10, or 20 years

0

H0lzm1ch3l t1_jccmlwq wrote

I dislike Computer Vision. It just annoys me somehow.

−7