Viewing a single comment thread. View all comments

ECEngineeringBE t1_ir9q399 wrote

Reply to comment by canyonkeeper in [R] Google announces Imagen Video, a model that generates videos from text by Erosis

Damn, that must mean that all those experiments they run at CERN aren't research because I can't replicate them in my kitchen.

−11

xrailgun t1_ira22vy wrote

Shit straw man take.

14

ECEngineeringBE t1_irac32j wrote

How so?

I could say the same about that "shit gatekeeping take"

1

xrailgun t1_irc7qic wrote

In case you're serious, physics papers are crammed full of mathematical derivation first to logically support their hypotheses, then include all relevant conditions and parameters such that IF/WHEN you get access to the collider, key in the same, you could replicate them.

In ML, mathematical support still exists to varying degrees, but without sharing the source code, even if you had access to Google's/OpenAI's/Nvidia's billion dollar hardware, you can't replicate it.

2

eposnix t1_ircvtgw wrote

Wait, what? I mean, maybe not 100%, but there are metric fuckloads of open source implementations of closed source models replicated by just the method in the paper.

1

FriendlyRope t1_iragqww wrote

He does has a point, if there is no independent verification of an experiment (I.e. replication, or at least independent inspection of an experiment) the experiment can not be said to be trusted. For example the results shown could be "cherry picked" or the test data could be contaminated by training data.

2

ECEngineeringBE t1_irahdyd wrote

Sure, but just because you can't replicate it, doesn't mean that nobody can. We already had Facebook's paper on video generation a week ago, and we also have stability AI saying that they're planning their own model.

And also, just because the results can't be fully trusted (due to high barrier of replicability), does not mean that the publication isn't "research".

5

throwawayguy91 t1_irbfj1m wrote

thats the whole reason ATLAS and CMS work independently from each other

1