Viewing a single comment thread. View all comments

Matsarj t1_ixiaoj1 wrote

Reply to comment by dpineo in [R] Category Theory for AI,AI for Category theory by FresckleFart19

I'm someone with a PhD in a field that uses category theory extensively and I now work in DS. I'm finding a lot of ideas in this post unmotivated, I guess I'm pretty bearish on CT being applied to ML. Can you explain what problems you see that category theory will be used to solve?

13

yldedly t1_ixigekm wrote

I stumbled on this thesis some time ago, where the author formulates a category of causal model, where arrows are structure-preserving transformations between models. Seems like it would be useful for causal model discovery.

9

Matsarj t1_ixijut0 wrote

That looks interesting, and I'm definitely not saying there's no intersection between CT and DS. There's some cool things I've seen with CT and probability theory recently. But to me it often seems like theory in search of a problem, and a far cry from functor becoming a common word among ML practitioners.

3

LazyHater t1_ixipeej wrote

CT is kinda good for ML if you have a complex topology of solution spaces. When programmers try to implement categories from a naive view, instead of applying sophisticated categorical constraints on their models, I definitely feel a sort of way about it. With that said, LLMs with analytic modules should be able to do categorical constructions in the not too distant future, which will be nice as hell. Optimizing functors might be a thing someday too, but its definitely not there mathematically yet

Im bullish on deriving (co)homologies using ML but it will be some time before we get there i think.

2

Matsarj t1_ixirp1l wrote

I guess I'm separating purely categorical applications from TDA applications, which I agree things like persistent homology will probably be useful.

2

LazyHater t1_ixiuzpu wrote

Persistent homology for real data is developing plenty of classical techniques, but we probably need a very good LLM with some very good HoTT to derive non-simplicial (co)homology for some given category -> some given abelian category

1

Matsarj t1_ixiwioj wrote

This sounds really interesting. Can you expand here or link to any resources related to this? I'm most interested in where you would apply these cohomology theories.

1

LazyHater t1_ixj3e2t wrote

Resources are quite scarce I'm afraid. Emily Riehl and company are working on (inf,1) categories to establish homotopy between derived functors, for applications in univalent foundations. For a computer algebra system or proof assistant, type equivalence is required to abstract away implementation details. To actually compute homotopy equivalence, it's better to compute cohomology equivalence, but simplicial cohomology is often too expensive to compute. So it's an open problem whether we can optimize a derived homology functor between a derived (enriched) functor and an abelian (enriched) category (which still lacks proper definition afaik). But its a goal I heard at a HoTT talk once to get non-simplicial cohomology of types instead of computing homotopy (which is computationally impossible at scale). Feel free to steal and spread this idea but it's kinda original and speculative.

tl;dr application is computing homotopy equivalence of types at a reasonable expense

3

Matsarj t1_ixjliwb wrote

So I'm pretty familiar with homotopy theory but don't know any type theory, homotopy or otherwise. What does determining whether types are homotopy equivalent get you in terms of ML applications?

1

LazyHater t1_ixjq0v8 wrote

In "laymans" terms, it gives you a) an environment for ML models to verify their proofs and b) and rich space for ML to study relations between different fields of mathematics, logic, philosophy, ethics, and everything else by default at that point.

propositions are implementations of types so just being able to say when propositions are equivalent in like a rigorous way is good for science anyways.

3

Phoneaccount25732 t1_iybqxxl wrote

Does category theory continue to be as insanely mind-blowing once you actually understand some of it?

1

LazyHater t1_iyeaom6 wrote

Yes and no. The fundamental ideas, once they start to sink in, show clear parallels between vastly different fields of analytic thought. The more you understand the framework though, the more its limitations can be concerning. Dependence on the axiom of choice, for example, and the naturality of choice in the field itself, leads some to speculate that if contradiction can be chosen true, the theory's implementation (with the vast majority of categorical proofs appealing to choice) is completely broken.

It's overwhelming at times how applicable category theory is from the right perspective, but underwhelming how its implementation in set theory can be expected to pan out.

tl;dr: category theory is dope but aoc is sus

1

dpineo t1_ixj7ot5 wrote

Sure. I see the potential of CT as being a language for expressing, reasoning about, and ultimately designing, AI/ML architectures abstractly.

In software development, we have the concept of "design patterns" that provide a common vocabulary with which we can describe common recurring patterns in software design at an abstract level. It cuts past the implementation details and allows us to focus on larger concerns, such as the composition and coupling of components, and the flow of information. This maturity in software development has allowed us to grow past brute forcing spagetti-code programs to developing robust enterprise-sized systems.

I believe that AI/ML is still in it's spagetti-code infancy. We have no idea how to build and compose AI/ML components into a system a disciplined way. To scale up to larger and more complex AI/ML systems, we're going to need to step back and look at AI/ML architectures more abstractly the way that software did with design patterns. I think CT may be able to help with that.

2