Comments

You must log in or register to comment.

Aswarin t1_iv3cirx wrote

I had one reject (but their comments read as though they just hated the idea/lack of novelty) and three marginally below the accept line. One of my reviewers pulled out but I've got no idea what my score is as it says NA currently. Overall the reject annoyed me just touting its not novel but looking at the comments of the others hopefully I can change some minds.

5

big_mantis t1_iv3dmz8 wrote

I got 2 8s and 3 3s so a pretty big split in opinions lol. This is my first time submitting to an AI conference so not sure where that leaves my chances but not too optimistic about getting accepted.

16

CupcakeCleric t1_iv3fo6m wrote

Two low-confidence weak accepts and one high-confidence reject. The last one made two serious mistakes (and one of them is undergrad-level CS stuff). I doubt I'll be able to change the reviewer's mind. fml

12

Blasphemer666 t1_iv3g9rk wrote

I am sorry this is my first time submitting it.
How should I read the reviews?

We look at the recommendation (score)?
So how good is accepted?

6

Sufficient_Flight876 t1_iv3jb5i wrote

8, 3, and 3. First time submitting a ML conference and I'm curious if reviewers can update their score freely during the discussion or they can only update once at the end

4

Blasphemer666 t1_iv3l3oc wrote

Replying to Sufficient_Flight876 (#446,909)

I am also quite curious, my sum is the same as yours (5,6, and 3).
But I have no high score, idk how to read these scores correctly.

So how do they decide what papers are accepted after final scores are out?
Average > 5.0?

1

tfburns t1_iv3m9qy wrote

FYI, if you are curious about your chances given your ICLR scores, you can check some scores (and resulting decisions) from previous years:

2022: https://guoqiangwei.xyz/iclr2022_stats/iclr2022_submissions.html

2021: https://github.com/evanzd/ICLR2021-OpenReviewData

And someone even made a small calculator based on 2019 data: http://horace.io/willmypaperbeaccepted/

Edit: Someone has also scraped this year's data, so you can see where your paper lies in the distribution: https://docs.google.com/spreadsheets/d/1INZI9epkfBkPOlKuJFaffUCOKDns87Iqg4zovHnf-zs/edit#gid=554805545

21

tfburns t1_iv3mvh5 wrote

Replying to Sufficient_Flight876 (#446,909)

>8, 3, and 3

Very big split! In one past case of mine, I found the higher-rated one sneaked their rating down to go along with the group without saying anything. Hope that doesn't happen for you.

>I'm curious if reviewers can update their score freely during the discussion or they can only update once at the end

AFAIK they can change at any time by "editing" their original review comment. Ideally they also add some text/reply to indicate that they changed their score and why.

Whatever the outcome, congrats on your first submission!

1

tfburns t1_iv3n3s5 wrote

Replying to big_mantis (#446,685)

>2 8s and 3 3s

For a total of 5 reviews? Not only a big split but also a large number (both of my submissions only have 3 reviews, and average across all papers is apparently 4.1 at present).

Congrats on your first submission :)

9

tfburns t1_iv3nbla wrote

Replying to CupcakeCleric (#446,786)

>The last one made two serious mistakes (and one of them is undergrad-level CS stuff)

That sucks!

Good luck, though. I think the confidence ratings are not always too meaningful. There might be a chance you can correct the mistake and/or AC notices and decides in your favour.

5

tfburns t1_iv3no12 wrote

Replying to Blasphemer666 (#446,808)

>How should I read the reviews?

I suggest reading them in full multiple times, then trying to write a synthesis of the opinions and main critiques. After that, you need to figure out how will you respond to the main critiques: will you try to fix them (if so, how?) or are those critiques incorrect or mistaken in some way (if so, how?).

​

>So how good is accepted?

No one can tell you for sure at this stage, and ultimately it's a case-by-case decision. However, you can see what past years' scores led to in terms of decisions here:

https://www.reddit.com/r/MachineLearning/comments/ymctqy/comment/iv3m9qy/?utm_source=reddit&utm_medium=web2x&context=3

5

tfburns t1_iv3o2on wrote

Replying to Aswarin (#446,647)

At the very least, since the reviews and your replies are public, you can try to correct the record re misconceptions, even if you don't end up changing certain reviewers' minds. And sometimes when replying to one reviewer you can convince a different reviewer. Good luck.

2

51616 t1_iv458zh wrote

4 8s for my first submission to a big conference!

12

micfilter t1_iv459nm wrote

  1. 5,5,3
  2. Not sure
  3. I have so many questions!
    1. Why? I'm an unaffiliated independent researcher, solo author, and I have never submitted to an AI conf before... so I have no idea what I'm doing.
    2. My main question is: a large concern of all 3 reviewers seems to be that I did not clearly state my contributions, nor did I contextual my contributions in prior literature.
      1. One way to address both of these would be to cite my prior workshop (also on arXiv) work.
      2. I didn't do this for the conference submission because there is significant overlap between the works, which would 100% deanonymize me. I even used the same title for the 2nd worksop paper as this conference paper (that was actually a 4am mistake to not update the conference paper title).
      3. So, I'm not sure if I should just go ahead and cite the workshop work using "a 3rd person narrative", or if this is a good question for the AC (see below), or totally n00b and kinda pathetic.
    3. Regarding asking questions for the AC, ICLR states: "In case you have any questions or concerns connected to your submission that are not addressed in the Author Guide / FAQ, your Area Chair is your first point of contact, and you can message them through the “official comment” feature on Open Review."
      1. Does anyone know if these "official comments" are viewable by the reviewers?
      2. How seriously should I pursue alt paths for finding the info before asking the AC?
3

51616 t1_iv45ny4 wrote

Replying to tfburns (#447,070)

Having 5 reviews i suppose is unusual. I guess initially the paper has 8/8/3/3 split, which might necessitate an additional review. Unfortunately, that one is another 3

6

LilHairdy t1_iv50upr wrote

I've got 8, 5, 5, 3

I didn't read them yet, because of anxiety. So far I only read reviews of other papers and feel tempted to defend them :P

3

wassup369 t1_iv5b7zb wrote

got 5,5,6,6. What are the chances of getting accepted?

1

johnnycrab_whoop t1_iv5gj70 wrote

Scores are 8, 5, 3. The third review is very thought-out and raised valid concerns, maybe it's possible to change the score. It's my first time submitting to ICLR, and I am very satisfied with the reviews, all three of them are quite sophisticated and helpful. But unfortunately I know that this is not the case for all submissions...

2

MD_rail t1_iv5tl9i wrote

8,6,6,6 seems hopeful.. maybe?

8

ukamal6 t1_iv6vwli wrote

Replying to MD_rail (#450,144)

My first submission here, and got the same score as well. Probably I will have to convince the 6 scoring reviewers and increase their ratings a bit.

1

calvinreeve t1_ivbaqh3 wrote

56815 with confidence 23224. Reviewer with score 1 claims to not know the area and that the paper is not understandable to "general experts". Is the fact we have 5 reviews an indication that the confused reviewer's comments will be discarded?

2

johnnycrab_whoop t1_ivbvzxl wrote

One can update the manuscript until November 18. Has anyone seen it stated somewhere if it's until the end of the day, or until end of November 17? Could not find it in the guide for authors, but maybe I am just blind. :)

1

Odd-Squirrel4324 t1_ivdlbhu wrote

Does the page limit of 9 apply to the revision during rebuttal phase? I appreciate any answer.

1

BeatLeJuce t1_iveapvr wrote

Replying to Unlucky-Attitude8832 (#448,457)

to be fair, "4" wasn't an option you had as a reviewer, so it was either 3 or 5, and 5 is "slightly below acceptance threshold". So if you feel like there is a flaw in the paper (even one you could recover from in rebuttals), "3" was the natural vote to give. Personally, for most papers I rated "3", the authors could come back from it if they manage to address my concerns properly.

2

oldppd t1_ivpm54o wrote

One reviewer seems positive and summarized that this paper is marginally above the acceptance, but he gave a 5 score. Should I notice him or something? Will this offend him? This is my first time submitting to ICLR, and the score is 6, 5, 5, 5. So this question is indeed important for me. Is there anyone experienced who can tell me about it? Thanks!

1

oldppd t1_ivt702h wrote

Replying to East-Beginning9987 (#491,475)

>no one is offended by anything I believe, if it's a valid and polite response. The alternative is to accept rejection, which I don't thing you would like

Thanks for your suggestion. I decide to add the notice of this in the end of the reply.

2

etherw t1_ixftszo wrote

After the discussion period, no reviewer has changed the score yet (still 8,6,5,5), what's the chance like (first-time submission)?

1

Healthy-Noise-4166 t1_ixj3oi3 wrote

Got 6, 6, 6 after rebuttal. What are the chances of being accepted?

1

SkeeringReal t1_ixuzpym wrote

What kind of scores do you need to get a spotlight presentation? Eg.g. you get something like 8,8,8,6, all with high confidence.

How are spotlights chosen? I guess they want a diverse set of topics right? Like if there's only one RL paper they may just choose that one?

Anyone here ever been on the committee which decided this stuff?

1