Comments

You must log in or register to comment.

ukamal6 t1_j57miqb wrote

First first-authored paper here, got accepted with a spotlight!

18

CupcakeCleric t1_j57ofaz wrote

Welp, rejected. To be fair, I was hoping for an AC stroke of luck, which by definition is rare. Guess I'll drink something and start working on the ICML submission.

6

tfburns t1_j57q0pu wrote

I'm not seeing area chair meta reviews or decisions on OpenReview yet. Anyone see them on their papers? I got an email only.

1

memming t1_j59enjb wrote

Spotlight vs notable-top-25% distinction - From the email:

This year, we are disassociating the presentation style (e.g., Accept (oral), Accept (spotlight)) from the AC recommendation. What this means is that your paper will receive 2 "tags":

Tag for AC’s recommendation

  • notable-top-5% was traditionally called Accept (oral)
  • notable-top-25% was called Accept (spotlight).
  • Poster accept

Tag for presentation style

  • In-person presentation
  • Digital presentation

Why 2-tags? We decided to host all oral presentations only in-person. While all accepted papers will be offered digital presentation (i.e., video will be available for asynchronous viewing), notable-top-5% and notable-top-25% papers will also be offered an in-person presentation. Poster accept will be offered in-person poster session as well.

1

AdMassive9465 t1_j59pyqi wrote

Personally, as a reviewer, this year's ICLR reviewing system was a trainwreck.

  • Reviewers had 12 days to review ~5 papers. Twelve days to review 5 papers submitted at (arguably) one of the best venues of this (complex) domain.
  • Of the 16 expected reviews (for these 5 papers), only 8 had been delivered before the deadline (5 were mine). 5 were submitted on the day of the deadline. 3 were submitted in the "buffer" period before the opening of the rebuttal phase.
  • A paper was withdrawn by the authors after the opening of the rebuttal phase. Surprisingly -- for this paper -- a reviewer proposed to reject the paper and, among the motivations, they stated that the paper should "compare" with a paper that had been accepted 1 month before the submission deadline of ICLR'23, and that was supposed to be presented at a conference held in mid 2023.
  • Of the other 16 reviewers assigned to the other papers I reviewed, only 3 wrote one post (aside from the review). I even explicitly asked the support of other reviewers w.r.t. an obscure point of a paper, and received no response (this paper was rejected).
  • I championed one paper, which was ultimately rejected by the AC who had great misconceptions of the paper as a whole. None of the reviewers chimed in, and my attempts at changing the AC's opinion were in vain (the AC was not "wrong", but they were just looking at the paper from the wrong perspective). I feel very sorry for the authors since they had very little they could do: they interacted just with myself, and I was pleased by the paper; and now they got a reject due to concerns expressed _after_ the closure of the rebuttal phase (and raised just between myself and the AC).
  • Out of the 5 papers I reviewed, only one was accepted.

I think that the worst part, however, is that all of the above will be "meaningless" given the ridiculous amount of submissions to ICLR'23.

Frankly speaking, this is discouraging as a reviewer who "tries their best" to help research.

^(Edit: fixed numbers after double-checking)

22

ukamal6 t1_j59rey7 wrote

About the trainwreck review part, I totally agree with you. Even after the deadline, I was getting random (I am saying random because I had almost close to zero expertise in those fields, so I had to decline all those requests) emergency review invitations which made me understand the overall situation. One thing that made me really confused was: why did they take so much time (almost 2 months, after the author discussion period was over) to send out the decision? I was really not sure why the meta-review process deserved this much time (~60 days) when the actual review time was just 12 days! Any thoughts about this?

3

AdMassive9465 t1_j59s6n6 wrote

Agreed. I don't know what happened behind the curtains, and I hope that whatever happened was done for a good reason.

Yet, I have a bad aftertaste: I cannot stop thinking that my effort was useless. Truly, I could have written shit-reviews such as "no novelty, the paper ultimately describes a method related with neural networks, which have been studied for decades" (jk) and forget about it.

6

spionski t1_j5ac1t4 wrote

This is very interesting to read. For the paper you championed, was that discussion between you and the AC public, i.e. visible to authors? I had a similar experience from an author perspective. We had really constructive discussions with all the reviewers, and were fairly optimistic we'd get in, but then the meta-review was pretty disconnected from all of that. Unfortunately no idea what may have happened between AC and reviewers, so it's a bit of a mystery from our perspective. It's frustrating to have all these in-depth discussions, but then no opportunity to clear up even basic misunderstandings in the meta-review, where it actually matters. Having a discussion period with the AC would have made such a difference, and given the timeline it seems it would totally have been feasible.

3

AdMassive9465 t1_j5aliqj wrote

It was not: in theory, it should have been a discussion that involved "only" the reviewers and the AC. However, none of the reviewers (besides me) participated---thereby resulting in a 1:1 discussion with the AC (whose remarks could not have been addressed in any "revision" of the paper, since this occurred in December).

Interestingly, there is even a "gap" between the Reviewer and AC guidelines:

  • according to the AC guidelines, the authors could "participate in the discussion" occurring between November 18th and December 12th...
  • ...but this was not stated in the reviewer guidelines, which stated that, after November 18th, the only role of the reviewer was to have the "virtual discussion with the AC for borderline papers".

I was unaware that the authors could still "participate" after November 18th, so I did not even think that it was possible to involve them in the discussion (had I known that the other reviewers would have done nothing, I would have certainly involved the authors)

3

CarelessBar2844 t1_j5g54gw wrote

Rejected after an 866 score (up from 856). the 8 score reviewer had given a 2 line review and the meta reviewer said their review was discounted (I expected this).The sad part is that in both neurips (score 656) and iclr (score 866), our paper was borderline/weak accept from all reviewers (and one 8 score this time), but the meta reviewer comes up with new problems with the paper, without giving us a chance to respond.

Any way, onto the grind for the next conferences, I hope I will get a first author top conference paper this year!

3

byan19 t1_j65x7kr wrote

Got rejected, due to a reviewer changing his score from 8 to 5.

2