I do not recall seeing any highly mathematical papers at ICML this year. What you are proposing might be better received at a conference like AISTATS, perhaps.

Ah, that's a bit of a shame. I recall seeing a talk called "Towards a Mathematical Theory of Machine Learning" by Weinan E at ICML 2022 (whom I'm citing!), but I'm guessing that's not indicative of the conference as a whole.

My wife does pretty theoretical statistical learning work and has had success with JMLR

Here are two relevant conferences:

• DeepMath
• Mathematical and Scientific Machine Learning

This is great! Thanks :)

You're very welcome!

COLT is the best ML theory-oriented conference that I'm aware of.

Theoretical papers are welcome at the major conferences, but not that common.

NeurIPS has some pretty technical mathy papers too, right? How does it stack up against others mentioned here?

Among A*, COLT is probably the best venue, with ICML being a great highly visible fit too. NeurIPS will need justification and a great intro, whereas ICLR is would need experimental proofs.

Overall heavily mathematical papers, when properly contextualized and given intuitive understanding of proofs tend to be very popular.

ICML, NeurIPS, ICLR all have theory papers. But theory people tend to complain that they often get anti-theory reviews. It's possible since the reviewers are so random that you might end up with strict empiricists. COLT on the other hand is pure theory. JMLR is also the most theory heavy journal in machine learning. Optimization people sometimes veer towards SIAM journals, while stuff closer to statistics would be fit for statistics journals like AoS, Bernoulli, JRSS etc.

[removed]

[deleted]

Heartening to hear that a wide range of options are open to me. Thanks a bunch!

There are actual math journals such as simods for stuff like that. If you like Weinan E you can take a look at the venues he publishes in.

You're not going to find theory-dense papers at major ML conferences. Most of the reviewers don't bother going through them and people usually find theory boring compared to "super duper cool" architectures that lead to 0.1% increase in performance. Like the top comment said, COLT is a good place to start.

I would echo the others here and say that, depending on the focus of your paper, the big conferences do take maths/theory papers (NeurIPS, ICML, ICLR, COLT, also AIStats and UAI depending on your topic) and JMLR for longer papers. But all of the conferences are both very competitive and have a large random component in what gets accepted… it may also be worth looking at workshops at these conferences to see if anything fits better. Less “prestigious” but also easier to get into and more likely to be reviewed by a suitable/friendly referee.

What’s the topic of your research?

Also AIStats

> But theory people tend to complain that they often get anti-theory reviews.

How the turn tables! A couple of decades ago - the boot was on the other foot.

Approximation theory traditionally looks at the structure of function spaces under addition; but approximation spaces under composition are underexamined. Studying approximation spaces under composition may quantitatively explain the outperformance of neural networks, reveal links to dynamical systems, and suggest related architectures.

(Edit: Following the work of Weinan E, Chao Ma, Lei Wu, Ronald DeVore, Gitta Kutyniok, et al.)

I remember trying to publish a theory paper (statistical learning theory) in ICML and got criticized by two reviewers that complained the paper had no experimental justification (despite being pure information theoretic lower bound of any learnt algorithm which was impossible to justify experimentally??) and my professor and I doubt they understood what was happening.

The third reviewer was extremely knowledgeable in this area and we truly appreciated their comments which definitely helped better the paper.

Frankly, none of the major venues can reliably evaluate mathematical work. I have seen some reasonable evaluations of work which use some simple objects/concepts which are new to the ML community, but those are rare. There is quite a lot of statistics published, e.g. at COLT. But very little to no math. For that, you're better to go to math venues ime.

>Theoretical papers are welcome at the major conferences, but not that common.

Agreed, which the caveat that 'theoretical' here rarely means more than 'statistics' / 'optimization'. Math is basically non-existent in ML.

>Overall heavily mathematical papers, when properly contextualized and given intuitive understanding of proofs tend to be very popular.

Strongly disagree. MLers have a very limited appreciation of 'math'.

>COLT is a good place to start.

Although it basically limited to optimization/stats, not math.

>NeurIPS has some pretty technical mathy papers too, right?

Examples?

Aistats man!

SIMODS is the journal you are looking for. Be warned, the math here is often of a different flavor/level than what's discussed at conferences.

I was confused why you would mention him in particular before seeing that OP mentioned him before lol. Yes I agree with this advice

no idea but thanks for asking! I'm finding real good stuff in the answers

I know, right? The responses have been great so far.

Hey, I’m actually interested in statistical learning theory in grad school. What kind of math prerequisites do I need to be able to research statistical learning in a phd program?

Ah, it's by SIAM! Definitely of interest to me, I'll be sure to check the proceedings out.

A basic course in statistics and the desire for the PhD grind.

Dude cmon stop playing. I’ve read some of these papers and the math is crazy. What math courses do you need

I kindda agree that you don't necessarily need a "math course" other than the usual requirements of CS undergrads. You somewhat pick up the rest on the way. I asked the same thing to theory grad students and they said the same thing. One fella that had a math BS actually said he didn't find his undergrad experience to be terribly useful, which kindda says a lot. Taking actual learning theory classes and reading textbooks will be necessary though.

That doesn’t make sense tho. Math majors have more theory than cs majors. Statistical learning definitely requires high levels of mathematics, like functional analysis. It’s a huge area in statistics.

The thing is, you can't learn everything in advance. And you don't need everything all the time. Some works in learning theory might be mathematically very deep, but taking a few undergrad math courses definitely won't prepare you for those. Although they might help you develop mathematical thinking. But as per the knowledge itself, I don't think taking math major courses is the most efficient way to do it.

So then your saying a cs major is the best major? All they learn how to do is code? How would you not need mathematical maturity to even make contributions to statistical learning theory. Like I would even argue a statistics major is more prepared than cs cause they have the background in stats with computing experience as well. Cs is just a software dev major.

Because you do learn theory in undergrad CS...? On the contrary, no course teaches you how to code.

Yeah you learn systems design, not functional analysis, measure theory, and actual mathematics to do the derivations in statistical learning theory. Statistical learning theory is mathematics. Not just coding.

You learn algorithm analysis, computational complexity theory, discrete mathematics, automata, cryptography and whatnot. Do these seem like coding to you?

None of those are useful for creating new statistical learning methods or pushing the boundaries of statistical learning as functional analysis, measure theory, real analysis, and statistics. Like cryptography is useless for developing new regularized regression methods, who gives a shit about complexity theory? Like you guys think ML theory and statistical learning is a CS branch. Like you guys coin the term machine learning and think it’s a branch of CS… very far from the truth. Mathematicians and statisticians have been running circles around you guys doing this for decades. Know your place

Wait what? The core of learning theory is algorithm analysis and complexity theory! Please take any learning theory textbook or course first before making such groundless judgements. God the freakin definition of "PAC learnable" is algorithm-theoric.

It’s literally called statistical learning theory

Yes that's why you need at least "undergraduate" statistics knowledge.

Lol that’s so false. Undergrad stats is not nearly enough, not even undergrad math. You can get away with just knowing how to code a little bit but it’s way more math and stats. Cs majors are just trained to be software engineers and nothing else. I’m a math and stats major and run circles around them in AI courses because they don’t have any technical depth mathematically.