I've never been at a workplace which used any of the structures you mention. Honestly, model development is fairly straightforward from the project management and software development perspectives. The clever bit is the statistics/machine learning, and the parts requiring the most care are data acquisition (problem definition, statistical sampling, ...), model validation (error resampling, testing for important sub-populations, ...) and deployment (verifying the deployed model, ...). Most serious analysts I know use something that resembles CRISP.

Data Science is different from development, the agile methodologies don't apply because it's more split into 2-3-4 stages:

1. a discovery stage, where you hone in on the question you want to ask, where you get sample data from that simplified the actual data you'll work on
2. Which ML algorithm or strategy is close to answering your question with the sample data you have
3. Setup training, validation and test samples. Then validation that they are representative of you real data
4. Run the model and iterate over the process to improve results. Maybe use hyper parameters optimisation to come up with best results for your lose function.
5. Present your result for peer review
6. Refactor your model for performance and deployment

There is a lot of data science preamble before you get to a peer review. So quick feedback loops are different compared to software development. The discovery phase is more about understanding the data and extracting the appropriate features that should be tested. It's mostly about applying stats to your data, that then gives you hints about which ML modeling to choose from. See this article on stats: https://towardsdatascience.com/10-machine-learning-methods-that-every-data-scientist-should-know-3cc96e0eeee9

The developer stage is more at the tail end where you look at refactoring the algorithm to make it as fast and explainable as possible. Maybe also add a feedback loops in production to check for model drift, that's where your agile frameworks would potentially be used.

Kanban. Trying to do scrum for ML ends up pretty goofy as most backlog items will be spikes.

I'm a TL of a fairly small ML team. There's no perfect solution but I can try to explain a lil bit what we do.

We have 2 separate processes, what it's been described as product discovery so the (R in R&D) and the productionizing (the D in R&D).

For the research part, in scrum it's called spikes. Honestly, we use kanban (so we don't commit to sprints) although we have periodic checkpoints on how the research is going.

Honestly before starting the research, sitting down with the stakeholders and understanding whats we are trying to solve is key. I would not suggest starting a project with a very vague description on what you might want to solve, this typically leads an horror show based on "he said she said we understood". But let's say you have a "well defined" idea of what value you want to create. Then you move into kanban (create the research ticket bla bla bla). In my company we document everything, the ideas that we had, what worked what didn't, metrics, error analysis, anything that you might need in the end to make a decision on which specific solution you might need. This part is what "most people" assume ML in companies who have never done ML in a company think ML it's about. Basically you might have a notebook, a shitty poorly optimized script, perhaps if you have some standards you'll have a dockerized demo. In any case, as per the methodology goes, kanban is flexible enough to fit anything you want in research.

Now we move into let's write real code that scales. This is software development. You got a model, but you have to serve it, might be real-time or batched or a combination, probably you got a DB that stores results or an API whatever. That's pretty standard, and depending on the company might fit into more kanban or sprints. What we do is fit everything into a different kanban.

At the end of the day, it's a matter of checking what works for you and your company. Even if you were just writing pure software, not a single company does the same scrum, everyone adapts it to their needs and people. In my experience, having short research cycles, perhaps a week or two, evaluate the progress, and decide what to keep doing fits pretty much into the idea of a sprint, but typically in sprints you want to deliver some value, which will not be the case for most sprints, hence we transitioned to kanban.

​

On how to convince PMs and managers, well, I got no clue. IMO you can't be a PM for a ML team without actual ML knowledge. There's no way you can come up with solutions or ideas if you don't know the potential that ML has.

​

Happy to discuss tho :D

What are those things.

Scrum sounds like scum and scram.