Comments

You must log in or register to comment.

tfburns t1_iv3m9qy wrote

FYI, if you are curious about your chances given your ICLR scores, you can check some scores (and resulting decisions) from previous years:

2022: https://guoqiangwei.xyz/iclr2022_stats/iclr2022_submissions.html

2021: https://github.com/evanzd/ICLR2021-OpenReviewData

And someone even made a small calculator based on 2019 data: http://horace.io/willmypaperbeaccepted/

Edit: Someone has also scraped this year's data, so you can see where your paper lies in the distribution: https://docs.google.com/spreadsheets/d/1INZI9epkfBkPOlKuJFaffUCOKDns87Iqg4zovHnf-zs/edit#gid=554805545

21

SkeeringReal t1_iydjn0h wrote

Is there an update on this for post rebuttal now? What's the new distribution?

1

big_mantis t1_iv3dmz8 wrote

I got 2 8s and 3 3s so a pretty big split in opinions lol. This is my first time submitting to an AI conference so not sure where that leaves my chances but not too optimistic about getting accepted.

16

tfburns t1_iv3n3s5 wrote

>2 8s and 3 3s

For a total of 5 reviews? Not only a big split but also a large number (both of my submissions only have 3 reviews, and average across all papers is apparently 4.1 at present).

Congrats on your first submission :)

9

51616 t1_iv45ny4 wrote

Having 5 reviews i suppose is unusual. I guess initially the paper has 8/8/3/3 split, which might necessitate an additional review. Unfortunately, that one is another 3

6

big_mantis t1_iv4fr2l wrote

Thanks! Yeah I was a little salty we got 5 lol, the average would've been a lot better without that extra 3

3

khaldrug0 t1_iv3f390 wrote

My first time also, I got 5 5 6 8, hopefully with a little bit of work can increase those

5

tfburns t1_iv3nt0t wrote

Congrats on your first submission! Sounds pretty close. Good luck.

2

Unlucky-Attitude8832 t1_iv4nisw wrote

I also got 2 8s and 2 3s, the guys gave 3 must be brutal lol

2

BeatLeJuce t1_iveapvr wrote

to be fair, "4" wasn't an option you had as a reviewer, so it was either 3 or 5, and 5 is "slightly below acceptance threshold". So if you feel like there is a flaw in the paper (even one you could recover from in rebuttals), "3" was the natural vote to give. Personally, for most papers I rated "3", the authors could come back from it if they manage to address my concerns properly.

2

CupcakeCleric t1_iv3fo6m wrote

Two low-confidence weak accepts and one high-confidence reject. The last one made two serious mistakes (and one of them is undergrad-level CS stuff). I doubt I'll be able to change the reviewer's mind. fml

12

tfburns t1_iv3nbla wrote

>The last one made two serious mistakes (and one of them is undergrad-level CS stuff)

That sucks!

Good luck, though. I think the confidence ratings are not always too meaningful. There might be a chance you can correct the mistake and/or AC notices and decides in your favour.

5

51616 t1_iv458zh wrote

4 8s for my first submission to a big conference!

12

MD_rail t1_iv5tl9i wrote

8,6,6,6 seems hopeful.. maybe?

8

ukamal6 t1_iv6vwli wrote

My first submission here, and got the same score as well. Probably I will have to convince the 6 scoring reviewers and increase their ratings a bit.

1

SuperTankMan8964 t1_iv6yhaj wrote

It's a decent rating, but can't say for sure since AC has all the power. Don't forget to properly do a rebuttal still.

1

Blasphemer666 t1_iv3g9rk wrote

I am sorry this is my first time submitting it.
How should I read the reviews?

We look at the recommendation (score)?
So how good is accepted?

6

tfburns t1_iv3no12 wrote

>How should I read the reviews?

I suggest reading them in full multiple times, then trying to write a synthesis of the opinions and main critiques. After that, you need to figure out how will you respond to the main critiques: will you try to fix them (if so, how?) or are those critiques incorrect or mistaken in some way (if so, how?).

​

>So how good is accepted?

No one can tell you for sure at this stage, and ultimately it's a case-by-case decision. However, you can see what past years' scores led to in terms of decisions here:

https://www.reddit.com/r/MachineLearning/comments/ymctqy/comment/iv3m9qy/?utm_source=reddit&utm_medium=web2x&context=3

5

Blasphemer666 t1_iv3o284 wrote

Thanks a lot for sharing extremely useful information. 🥹

3

tfburns t1_iv3o7ln wrote

You're welcome and good luck :)

3

Blasphemer666 t1_iv3oenu wrote

So will those posters also be included in the proceedings?

2

tfburns t1_iv3on4g wrote

Yes.

Edit: I don't believe the image of the poster itself is included, but rather the paper!

3

Aswarin t1_iv3cirx wrote

I had one reject (but their comments read as though they just hated the idea/lack of novelty) and three marginally below the accept line. One of my reviewers pulled out but I've got no idea what my score is as it says NA currently. Overall the reject annoyed me just touting its not novel but looking at the comments of the others hopefully I can change some minds.

5

tfburns t1_iv3o2on wrote

At the very least, since the reviews and your replies are public, you can try to correct the record re misconceptions, even if you don't end up changing certain reviewers' minds. And sometimes when replying to one reviewer you can convince a different reviewer. Good luck.

2

Final-Ad-671 t1_iv3wira wrote

The list of papers and scores is available at: https://twitter.com/shiqiang_w/status/1588714893309804545?s=20&t=f_DzcCrVtNx9jkEekd0dyQ

5

Blasphemer666 t1_iv3y64j wrote

There are 4,849 submissions, if acceptance rate is 30%, then it should be accepted if ranked before 1,454.

I am now around 2,800, should I just aim for the next conference? Lol.

11

SuperTankMan8964 t1_iv5ubwe wrote

No, it's all up to the AC. You can have three 8s and still get rejected, and three 3s and still get accepted. There are plenty of examples out there.

7

mo6phr t1_iv5gye3 wrote

“Quantum Machine learning” 1 1 1 1😳😳😳 Looks like it was written by a LLM

5

Sufficient_Flight876 t1_iv3jb5i wrote

8, 3, and 3. First time submitting a ML conference and I'm curious if reviewers can update their score freely during the discussion or they can only update once at the end

4

Blasphemer666 t1_iv3l3oc wrote

I am also quite curious, my sum is the same as yours (5,6, and 3).
But I have no high score, idk how to read these scores correctly.

So how do they decide what papers are accepted after final scores are out?
Average > 5.0?

1

khaldrug0 t1_iv4ou32 wrote

I think it's decided by the area chair, they read your paper and the reviews, then they decide whether to accept or reject

2

deschaussures147 t1_iv3m4yz wrote

They can do that freely during the reviewing period, but if you want an honest answer I won't put too much hope into the submission with your current score :D

1

tfburns t1_iv3mvh5 wrote

>8, 3, and 3

Very big split! In one past case of mine, I found the higher-rated one sneaked their rating down to go along with the group without saying anything. Hope that doesn't happen for you.

>I'm curious if reviewers can update their score freely during the discussion or they can only update once at the end

AFAIK they can change at any time by "editing" their original review comment. Ideally they also add some text/reply to indicate that they changed their score and why.

Whatever the outcome, congrats on your first submission!

1

Sufficient_Flight876 t1_iv3phf8 wrote

>AFAIK

Thank you all! Yeah I'll try my best for the rebuttal since reviewers at least liked the method... They complained the writing and some of the validation

2

micfilter t1_iv459nm wrote

  1. 5,5,3
  2. Not sure
  3. I have so many questions!
    1. Why? I'm an unaffiliated independent researcher, solo author, and I have never submitted to an AI conf before... so I have no idea what I'm doing.
    2. My main question is: a large concern of all 3 reviewers seems to be that I did not clearly state my contributions, nor did I contextual my contributions in prior literature.
      1. One way to address both of these would be to cite my prior workshop (also on arXiv) work.
      2. I didn't do this for the conference submission because there is significant overlap between the works, which would 100% deanonymize me. I even used the same title for the 2nd worksop paper as this conference paper (that was actually a 4am mistake to not update the conference paper title).
      3. So, I'm not sure if I should just go ahead and cite the workshop work using "a 3rd person narrative", or if this is a good question for the AC (see below), or totally n00b and kinda pathetic.
    3. Regarding asking questions for the AC, ICLR states: "In case you have any questions or concerns connected to your submission that are not addressed in the Author Guide / FAQ, your Area Chair is your first point of contact, and you can message them through the “official comment” feature on Open Review."
      1. Does anyone know if these "official comments" are viewable by the reviewers?
      2. How seriously should I pursue alt paths for finding the info before asking the AC?
3

Gnabenmeister t1_iv4vfke wrote

I think it is mentioned that citing your previous work should be done in 3rd person

2

LilHairdy t1_iv50upr wrote

I've got 8, 5, 5, 3

I didn't read them yet, because of anxiety. So far I only read reviews of other papers and feel tempted to defend them :P

3

johnnycrab_whoop t1_iv5gj70 wrote

Scores are 8, 5, 3. The third review is very thought-out and raised valid concerns, maybe it's possible to change the score. It's my first time submitting to ICLR, and I am very satisfied with the reviews, all three of them are quite sophisticated and helpful. But unfortunately I know that this is not the case for all submissions...

2

calvinreeve t1_ivbaqh3 wrote

56815 with confidence 23224. Reviewer with score 1 claims to not know the area and that the paper is not understandable to "general experts". Is the fact we have 5 reviews an indication that the confused reviewer's comments will be discarded?

2

cwkx t1_ivja2s8 wrote

If the 1 says they don't know the area (especially if it's a short review), most AC's will discard that rating.

1

wassup369 t1_iv5b7zb wrote

got 5,5,6,6. What are the chances of getting accepted?

1

SkeeringReal t1_ix6mq6i wrote

>https://twitter.com/shiqiang_w/status/1588714893309804545?s=20&t=f_DzcCrVtNx9jkEekd0dyQ

That puts you around the 38% mark, so I'd say the chances are not good.

1

johnnycrab_whoop t1_ivbvzxl wrote

One can update the manuscript until November 18. Has anyone seen it stated somewhere if it's until the end of the day, or until end of November 17? Could not find it in the guide for authors, but maybe I am just blind. :)

1

Odd-Squirrel4324 t1_ivdlbhu wrote

Does the page limit of 9 apply to the revision during rebuttal phase? I appreciate any answer.

1

Nameless1995 t1_ive35bn wrote

Probably. You can you use the unlimited appendix though.

2

East-Beginning9987 t1_ivss9ke wrote

I think it was written on the website that it is 9 pages fixed, even during the camera ready, so it is unlikely that rebuttal would allow anything extra

1

Odd-Squirrel4324 t1_ixawy4p wrote

Many thanks. Got 8865 after the discussion, is this good enough to get in? First submission to ICLR, finger crossed...

2

oldppd t1_ivpm54o wrote

One reviewer seems positive and summarized that this paper is marginally above the acceptance, but he gave a 5 score. Should I notice him or something? Will this offend him? This is my first time submitting to ICLR, and the score is 6, 5, 5, 5. So this question is indeed important for me. Is there anyone experienced who can tell me about it? Thanks!

1

East-Beginning9987 t1_ivss7mr wrote

no one is offended by anything I believe, if it's a valid and polite response. The alternative is to accept rejection, which I don't thing you would like

1

oldppd t1_ivt702h wrote

>no one is offended by anything I believe, if it's a valid and polite response. The alternative is to accept rejection, which I don't thing you would like

Thanks for your suggestion. I decide to add the notice of this in the end of the reply.

2

etherw t1_ixftszo wrote

After the discussion period, no reviewer has changed the score yet (still 8,6,5,5), what's the chance like (first-time submission)?

1

Healthy-Noise-4166 t1_ixj3oi3 wrote

Got 6, 6, 6 after rebuttal. What are the chances of being accepted?

1

SkeeringReal t1_ixuzpym wrote

What kind of scores do you need to get a spotlight presentation? Eg.g. you get something like 8,8,8,6, all with high confidence.

How are spotlights chosen? I guess they want a diverse set of topics right? Like if there's only one RL paper they may just choose that one?

Anyone here ever been on the committee which decided this stuff?

1

dasayan05 OP t1_ixx046o wrote

Spotlight/Oral are mostly case by case decision and totally up to the ACs. I don't think you can get a general rule or anything

2

Rolling_Pig t1_iv3qq9j wrote

  1. 8 6 6 5
  2. No..
−2

tfburns t1_iv3ror3 wrote

What makes you unsatisfied? Your scores seem on the positive side.

4

Rolling_Pig t1_iv3uxm0 wrote

Weaknesses reviewers claimed are not reasonable for me... And some of them absolutely misunderstood my paper even if they rated more than 6.

2

tfburns t1_iv3vin0 wrote

That sucks. Hopefully you can correct their misconceptions or at the very least set the record straight.

1