TMills t1_j8eh61k wrote

It doesn't need to be sota in an absolute sense, but it should be interesting in an empirical way. If the model is small, it needs to benchmark against other small models. If it's efficient it should compare against other efficient models. If you just like it aesthetically, or think it's clever, then you need to think about what that cleverness buys you and evaluate it in that dimension.


TMills t1_ittkwez wrote

In addition to some of the other great comments here, it's helpful to adjust your prior beliefs about new treatments we are testing. Even with promising pre-trial data, many will not work in humans, most that do have only slight benefits, and only rarely do we see large benefits. And almost all have some potential side effects. So whatever the control is (placebo or standard of care), it's often not the "worse" arm.