Viewing a single comment thread. View all comments

globalminima t1_iwe6j08 wrote

Reply to comment by sgt102 in [D] I Wrote a book "Managing Machine Learning Projects" by sgt102

There is no mention of monitoring, maintenance or retraining - does chapter 9 go into this? This is a big blind-spot if it's not there (and where most of the problems happen for inexperienced ML engineers)

13

sgt102 OP t1_iwhfc9e wrote

Chapter 9 addresses (to some extent) logging and monitoring, and goverance - which is a lot to do with how the model should be managed in life....

I've worked in projects where the model was ungoverned and went wrong and no one noticed for a long time... and that caused damage. I also got called in to sort out a project where the team retrained the model every week... and every week they overfitted it on new data. I think knowing what the models should do, being able to say that they are doing that and then having a clear way of deciding what to do if they aren't (ie. someone in charge) is the base of maintaining them... what's your pov though?

2