Viewing a single comment thread. View all comments

canyonkeeper t1_ir9nb50 wrote

It’s not research if it’s not replicable /not s

22

Veedrac t1_irbdxgp wrote

It blows my mind that this sub has basically become ML Flat Earth. There's no legitimate way people here actually think this research can't or won't replicate. “Oh but I can't personally put my foot on the moon play with this exact model exactly today” is such a fake argument.

7

ECEngineeringBE t1_ir9q399 wrote

Damn, that must mean that all those experiments they run at CERN aren't research because I can't replicate them in my kitchen.

−11

xrailgun t1_ira22vy wrote

Shit straw man take.

14

ECEngineeringBE t1_irac32j wrote

How so?

I could say the same about that "shit gatekeeping take"

1

xrailgun t1_irc7qic wrote

In case you're serious, physics papers are crammed full of mathematical derivation first to logically support their hypotheses, then include all relevant conditions and parameters such that IF/WHEN you get access to the collider, key in the same, you could replicate them.

In ML, mathematical support still exists to varying degrees, but without sharing the source code, even if you had access to Google's/OpenAI's/Nvidia's billion dollar hardware, you can't replicate it.

2

eposnix t1_ircvtgw wrote

Wait, what? I mean, maybe not 100%, but there are metric fuckloads of open source implementations of closed source models replicated by just the method in the paper.

1

FriendlyRope t1_iragqww wrote

He does has a point, if there is no independent verification of an experiment (I.e. replication, or at least independent inspection of an experiment) the experiment can not be said to be trusted. For example the results shown could be "cherry picked" or the test data could be contaminated by training data.

2

ECEngineeringBE t1_irahdyd wrote

Sure, but just because you can't replicate it, doesn't mean that nobody can. We already had Facebook's paper on video generation a week ago, and we also have stability AI saying that they're planning their own model.

And also, just because the results can't be fully trusted (due to high barrier of replicability), does not mean that the publication isn't "research".

5

throwawayguy91 t1_irbfj1m wrote

thats the whole reason ATLAS and CMS work independently from each other

1