There is a list of people who decided to share the paper and reviews despite rejection:

https://openreview.net/group?id=NeurIPS.cc/2022/Conference#rejected-papers-opted-in-public

The poster sessions are still nice, though overcrowded. I didn't particularly enjoy the keynotes so far. But in general terms, it's still fairly academic.

Definitely not. I'd say a solid 70-80% are researchers, and a significant number of non-research industry folks are those who work on ML

There are very few non academic folks

Only the leap between the beginning of the conference hall, and the beginning of NeurIPS at the end of that hall. All the big papers so far have been known for long enough.

Most haven't happened yet, so I don't know. GResearch is throwing one at a boat this year, which for now sounds like the most exciting thing this conference as far as Parties go. I.e., FlowRider's not gonna show up again, those times are over, now it's mostly about trying to get people to mingle. I hope the PhD students still party it up amongst themselves, but if so, I wasn't invited.

Apart from that, the biggest news in the ML research social sphere is that the 42. annual Workboat Show will begin tomorrow on the other side of the convention center. Everybody saw their big posters and everyone is talking about them. It's this year's inside joke. If no-one acknowledges them in keynotes or the end-of-conference speech, I'll be heavily disappointed.

new number who 'dis?

On the plus-side, the Mug looks nicer than the 2019 ones. On the negative side, a lot of the usual faces are missing. E.g. haven't seen Yoshua Bengio, or Neil Lawrence, or LeCun, or Schmidhuber, or Schoelkopf, Ilya Sutskever or Ian Goodfellow or lots of other folk. Not that there aren't a ton of brilliant people regardless, but it's kind of weird (might have just missed some of them in the crowd, though).

Poster sessions are crowded like 2019, but it empties out fairly quickly (e.g. 1 hour after the start of the session there's noticable less people). No-one's social distancing and <30% of people wore masks.

No Sperbank models this year. Nvidia is also missing, which was a bit more unexpected.

no, the hot talk is WorkBoat, GPT-4 is vapor ware. But today is OpenAI's party, so if they're going to announce something, today would be the day, IMO.

The ticket was 1000 USD for industry people, which is a huge hike from previous years. Academic rate should've been much lower, but IDK. Then there's the hotel cost. Mine costs ~2k USD for the week, but I've heard people pay way less for an AirBnB. Travel costs depend too heavily on where you're coming from to give a decent estimate.

It's crowded, but that's always been the case. There's a few thousand people here. It's okay most of the time, although it is hard to talk to people during poster sessions, which is unfortunate.

Overall, I'm happy to be here again and meet my old friends and acquaintances that you make working in the field over time. I missed that. And the general sense of being around ML people 24/7, it's nice. On the other hand, I can't wait to have a real meal again and not just fast food all the time. I'd give it an 8/10.

So, maybe let's not put the cart in front of the horse, focus on getting through the PhD first. You sound super motivated, but just to be sure, these are your two most important next steps:

1. If you can, find a good advisor/mentor. They'll introduce you to people and help you develop your research skills. TBH i don't know if the current PhD student job market is more of a "take what you can get" or a "lots of open positions" thing. But if you have several options on where to join, go with the ones that you feel like you can vibe with: they should have experience with publishing on top tier conferences, but also willing to help you succeed. use the search function, lots of advice on this reddit for what makes a good advisor.

2. Decide on what kind of research you want to do/pick a topic. Would you rather do applied research or theoretical stuff? There's a whole spectrum from maths or learning theory to foundational (e.g. RL or deep learning) to applied (computer vision, NLP, ...). Find a topic that excites you enough that you can dedicate several years to learning it and excelling in a subfield of it. The classes you'll be taking and the conversations with people at your lab/conferences should give you a flavor of what's out there. There's a bit of luck involved with picking a research direction that proves to be relevant, so advise from advisors/mentors helps a ton at that particual stage. One of the most important lesions you should learn in your PhD is how to find and approach good research questions.

Those are the two most important things to optimize your PhD success, which in turn optimizes hireability. In general your best bet at an industry research position is to do work that is meaningful enough that someone notices, in a field that the company (or a reasearch team with headcount) cares about. What that is depends on where you want to go and which field you want to work in. Definitely try to collaborate with industry, apply for internships or similar programs, or try to find other ways to collaborate while you're still in your phd. But like I said: I'd focus more on enjoying my PhD first, everything else should merely be a regularizer. Find a topic that interests you and the rest will follow.

Yes!

I haven't checked

The “deals” are maybe $1 cheaper than the price listed on amazon.com.

The swag is much better at neurips, though they do have open bars on the floor, so that's nice.

Underrated: I haven't read all of the Outstanding Papers yet, but I'm looking forward to digging into "Is Out-of-distribution Detection Learnable?" and "A Neural Corpus Indexer for Document Retrieval". (though arguably oustanding papers aren't underrated ;) ).

Cohere: Aidan wasn't there, which was a bit said, would've enjoyed meeting him again.

Well, I don't know at what level you're at, but I'm assuming you're undergrad and will keep this high level:

Well, we have models that have some understanding of text (e.g. GPT-3), and some notion of images (anything since ResNet or even AlexNet). Mostly in the vague sense that when we feed text or images into these "encoder" networks, they spit out a "representation vector" (i.e., a bunch of hard to decrypt numbers). We can feed those into "decoder" networks that do sensible things with those vectors (e.g. tell you that this vector is most likely of class "husky" at this and that position, or produce text that is the logical continuation of whatever textprompt you give it). We can train huuuuuuuuuuuuuge models like that (Billions of parameters to learn, probably cost 10^6$ to train for the first time). Very recently (last 1-2 years) we've learned to combine these two models (e.g. CLIP). So you can feed in text and get out an image (e.g. Stable Diffusion), or feed in text and image and get out whatever the text said to get out of the image (e.g. Flamingo).

That's roughly where we are in terms of big picture. Currently, we're working on better ways to train these models (e.g. by requiring less to no supervised data), or find out how they scale with input data and compute, or get smaller models out of the big ones, or whether to name the big ones "foundational" or "pretrained", or find creative ways to use or improve e.g. Stable Diffusion and similar models for other applications like reading code, as well as bunch of other stuff. No idea what the next big idea will be after. My hunch is on memory (or rediscovering recurrent nets).

Edit: this was extremely Deep Learning centric, sorry. There's of course other stuff going on: I don't follow Reinforcement Learning (learning over time with rewards) at all, so no clue about that, though it's arguably important for getting from ML to more general AI. Also, there are currently lots of issues being raised wrt. Fairness and Biases in AI (though I have seen almost no papers on it this year, why is that?). And more and more, people start to reason about "causality", i.e., how to go from correlation of things to causation between things... lots of other stuff outside my bubble.

I've seen people have a lot of success by doing a PhD in their respective field (e.g. Chemistry, Biology, Hydrology, ....) that applies ML to their field, and then slowly change their research to ML (e.g. going from "predicting stuff about atoms" to "using super state-of-the-art ML to predict stuff" to "adapting state-of-the-art stuff in ways that might be useful to other ML people and publishing at NeurIPS". You'll need to discuss this with your advisor though, and ideally find one who's willing to support you (or at the very least doesn't mind if you try).

I'd usually go to NeurIPS to present my own work, but this year I don't have a paper here. I came to mingle with other researchers, and catch up with old friends (people from my previous jobs/labs, people who left my current team, ex-interns, my advisor, co-authors from previous papers, random party acquaintances from parties at previous conferences, ....), make new ones, and see what other people are working on.

JAX is a huge step up from Tensorflow, even though I aboslutely don't understand why it takes Google so many iterations to eventually land on a PyTorch API. Jax might be close enough that they'll give up trying, but I feel like it still falls short in being as good as Pytorch. But Google will definitely continue using it, and they're one of the main drivers of AI research. So at least in that area it'll see non-trivial adoption. I still think Jax is inferior to PyTorch, so there's no reason to switch (the better support for TPUs might be a selling point, so if Google gets serious about pushing those onto people, there might be an uptick in Pytorch->Jax conversion).

The productization story is way worse than in PyTorch, and even within google, production still uses TF. Until that changes, I don't think Jax will make big inroads in industry.

Research Scientist. If you wan to do a more research elated role, I'd suggest you find one. They're certainly out there, just talk to companies who are hiring (see the NeurIPS sponsor's page). I can't tell you how hard/easy it will be to get in if you don't have any recent, relevant publications in the area, though. I'd expect that at least early-stage startups might not care, but you won't have time to do research there, either.

1. The recruiters who are there are still intent on hiring and getting to know people. But the big corps have shifted from "we're hiring everything and everyone" to "if you're outstanding in an area we care about, we'd love to have you". I haven't talked much to recruiters, but it seemed like they were still trying hard to find interested people.

2. Can't say. HR people were interested in me, though my badge clearly says that I already work in industry.

3. Haven't seen much happening in that sphere (which does not imply that it isn't out there).

That's not my area of expertise, so I don't really have any advice, sorry.

ask me anything

I wouldn't get my hopes up on anything big on that front. Sure, they could train a more compute efficient model (c.f. Chinchilla), but in general, it'll be incremental work, not ground breaking. I'd be surprised if OpenAI actually dedicated a lot of resources to improving GPT-3, it would not be their style. There's comparatively little to gain in terms of new breakthroughs, IMO.

it gets confused with nipples

Some people were bothered by the political incorrectness, so it was renamed in 2017

So obvious joke first: no, I don't agree because that's a continuous Random Variable and you're asking for a point estimate. badum tss

No but seriously, no-one can remotely predict scientific advances 10 years into the future... I don't have a good notion of what consciousness for an AI would look like. The definition Chalmers gave today ("experiencing subjective awareness") is a bit too wishy-washy, how do you measure that? But broadly speaking I don't think we'll have self-aware programs in 10 years.

I agree. Chalmers points out that consciousness doesn’t require human-level intelligence and may be a much lower bar, especially if consciousness exists as a spectrum or along multiple dimensions. If you’re willing to admit the possibility that there’s something that it’s like to be a bat, or a dog, or a fish, then it seems plausible that there could be something that it is like to be a large language model with the ability to genuinely understand language beyond a surface level. Chalmers seems to think we are getting close to that point, even if e.g. Lamda isn’t quite there yet.