londons_explorer
londons_explorer t1_jegw0va wrote
Reply to comment by Ulfgardleo in [News] Twitter algorithm now open source by John-The-Bomb-2
Parts of this code dump are for recommendations and ranking.
londons_explorer t1_je4l8q4 wrote
Reply to comment by tjcanno in U.S. renewable electricity surpassed coal in 2022 by altmorty
Batteries... or transmission (get power from another place 1000 miles away where it is windy right now) or hydro (store up water and use it only when other sources fail), or smart EV's (which charge only when there is spare power in the grid, and perhaps put some power into the grid at times of peak demand), or Heat reservoirs (heat peoples homes with heat pumps when there is spare power, and have big tanks full of a liquid that can store hotness or coolness for release into the home later when desired.
We can use one or all these solutions. We'll probably end up using a mix, decided by market forces.
londons_explorer t1_je367bb wrote
Reply to Hobbyist builds ChatGPT client for MS-DOS by CrankyBear
HTTPS on tiny devices is do-able... Check out the HTTPS client for the esp8266 for example. 'BearSSL'.
I don't think it is actually secure because the platform can't generate truly random numbers, but that doesn't matter for usecases like this.
londons_explorer t1_jdzwcfo wrote
Reply to comment by keepthepace in [N] OpenAI may have benchmarked GPT-4’s coding ability on it’s own training data by Balance-
Problems like this are never 100% novel.
There are always elements and concepts of the problem and solution that have been copied from other problems.
The easiest way to see this is to ask a non-programmer to come up with a 'programming puzzle'. They'll probably come up with something like "Make an app to let me know when any of my instagram friends are passing nearby and are up for hanging out".
Compare that to a typical leetcode problem, and you'll soon see how leetcode problems are really only a tiny tiny corner of what is possible to do with computers.
londons_explorer t1_jdqegui wrote
Reply to comment by Living-blech in China to introduce early 6G mobile applications by 2025, putting the country on track to rolling out commercial services by 2030 by Vailhem
I think you'd struggle to find anywhere in China without 5G - it's pretty universal.
londons_explorer t1_jdo4kj3 wrote
Reply to comment by farmingvillein in [D] Do we really need 100B+ parameters in a large language model? by Vegetable-Skill-9700
Think how many hard drives there are in the world...
All of that data is potential training material.
I think a lot of companies/individuals might give up 'private' data in bulk for ML training if they get a viable benefit from it (for example, having a version of ChatGPT with perfect knowledge of all my friends and neighbours, what they like and do, etc. would be handy)
londons_explorer t1_jdn0t7k wrote
Paper after paper has shown that bigger model outperforms smaller model.
Sure, you can use tricks to make a small model work better. But apply those same tricks to a big model, and it works even better.
londons_explorer t1_jcy0jf1 wrote
Reply to comment by londons_explorer in [P] TherapistGPT by SmackMyPitchHup
Cool tech demos can't exist in any remotely medical field for this reason.
I think that's part of the reason that medical science progresses so slowly compared to other fields.
londons_explorer t1_jcy050l wrote
Reply to comment by SmackMyPitchHup in [P] TherapistGPT by SmackMyPitchHup
This is the kind of service you need to either run 'underground' - ie anonymously, or you need to go get all the right legal permissions and certificates in place.
Otherwise you'll end up with massive fines and/or in prison when one of your customers sends a long chat about depression and then commits suicide. At that point, authorities won't overlook the fact you aren't properly licensed.
londons_explorer t1_jcpzan9 wrote
Reply to [D] Choosing Cloud vs local hardware for training LLMs. What's best for a small research group? by PK_thundr
I would make 'fake' data which isn't hipaa protected and do most of your work on that.
Then do a final fine-tuning on the HIPAA data on some rented servers. Your HIPAA data probably isn't more than a few hundreds of billion words anyway, so a fine-tuning should be quite quick and cheap to do a few full passes of the dataset.
londons_explorer t1_jcj8p9y wrote
Reply to [R] RWKV 14B ctx8192 is a zero-shot instruction-follower without finetuning, 23 token/s on 3090 after latest optimization (16G VRAM is enough, and you can stream layers to save more VRAM) by bo_peng
Can we run things like this through github.com/OpenAI/evals?
They have now got a few hundred tests, which is a good way to gauge performance.
londons_explorer t1_jcer3ci wrote
Seems this can be cleanly split into 'unrolling' and 'ink recognition'.
Unrolling at first seems like the easy bit... But it could be made complex if there are fragments of material which have internally become detached and fallen
londons_explorer t1_jc38zev wrote
Reply to comment by dojoteef in [R] Stanford-Alpaca 7B model (an instruction tuned version of LLaMA) performs as well as text-davinci-003 by dojoteef
I think it's a bug in the code - when you click "I agree" it doesn't send any network requests.
londons_explorer t1_jc1g3ee wrote
Reply to Twitter’s $42,000-per-Month API Prices Out Nearly Everyone | Tiers will start at $500,000 a year for access to 0.3 percent of the company’s tweets. Researchers say that’s too much for too little data by Hrmbee
There are many companies who would be willing to pay far more than this for the data.
Things like investment firms who want to know what is going on and react in real time.
The real solution is to have a delayed data feed - everything more than a week old is available for free. If you want data 15 mins delayed, pay $$. If you want data 1 second delayed, pay $$$$. If you want data immediately, pay $$$$$$$$.
londons_explorer t1_jbdrv43 wrote
Reply to [D] I'm a dentist and during my remaining lifetime I would like to take part in laying groundwork for future autonomic robots powered by AI that are capable of performing dental procedures. What technologies should I start to learn? by Armauer
What do you have to dedicate to this?
Time? Money? How much of each?
If it's just your time, I would start with hobby/kit robotics stuff, perhaps remote controlled (ie. Nothing smart) and show it doing dental work on plastic models of teeth with real tools. Then make a YouTube channel about your work, successes and failures.
That YouTube channel will hopefully get the next generation interested in actually doing the task properly.
If you have serious money to dedicate to the cause, I would try to start a startup, hiring a robotics expert, and someone who has previously worked in the medical devices field (there are soooooo many laws - navigating the legal landscape is probably trickier than making a robot do a filling). Obviously you should also go get VC funding wherever possible, but by putting in a chunk of your own money that will be far easier.
londons_explorer t1_jbdrgbx wrote
Reply to comment by wittfm in [D] I'm a dentist and during my remaining lifetime I would like to take part in laying groundwork for future autonomic robots powered by AI that are capable of performing dental procedures. What technologies should I start to learn? by Armauer
The overlap between ML and robotics is increasing day by day...
londons_explorer t1_jam8409 wrote
Reply to [D] OpenAI introduces ChatGPT and Whisper APIs (ChatGPT API is 1/10th the cost of GPT-3 API) by minimaxir
It was an interesting business decision to make a blog post announcing two rather different products (ChatGPT API and Whisper) at the same time...
ChatGPT is a best-in-class, or even only-in-class chatbot API... While Whisper is one of many hosted speech to text solutions.
londons_explorer t1_jam6r8g wrote
Reply to comment by LetterRip in [D] OpenAI introduces ChatGPT and Whisper APIs (ChatGPT API is 1/10th the cost of GPT-3 API) by minimaxir
Don't you mean the other way around?
londons_explorer t1_jam6oyr wrote
Reply to comment by LetterRip in [D] OpenAI introduces ChatGPT and Whisper APIs (ChatGPT API is 1/10th the cost of GPT-3 API) by minimaxir
Aren't biases only a tiny tiny fraction of the total memory usage? Is it even worth trying to quantize them more than weights?
londons_explorer t1_ja2m7hi wrote
Reply to [R] [N] VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion. by radi-cho
If I'm understanding this paper correctly... This technique doesn't work if there are any moving objects in any of the camera scenes?
londons_explorer t1_j9xw3l6 wrote
Reply to comment by a_common_spring in TIL about Janet Parker, the last person to die of smallpox in 1978. She worked above one of the last labs in its last months of permission to study the virus. The day Janet's viral strain was confirmed, Henry Bedson, the doctor in charge of the lab, took his own life. by w0mpum
It is highly likely the lab was the source.
And he was the boss of the lab.
Whether it was him personally being careless, or some fault in the procedures of the lab he oversaw doesn't really matter.
londons_explorer t1_j9ft1x2 wrote
Reply to comment by gwern in [D] Maybe a new prompt injection method against newBing or ChatGPT? Is this kind of research worth writing a paper? by KakaTraining
> That's not true, and has already been shown to be false by Sydney going off on users who seemed to doing harmless chats.
The screenshoted chats never include the start... I suspect at the start of the conversation I suspect they said something to trigger this behaviour.
londons_explorer t1_j835xx0 wrote
Reply to comment by Insecure--Login in [D] Are there emergent abilities of image models? by These-Assignment-936
You search the training image database for pictures of koalas with wine glasses... And there won't be many examples in there, and you check each one.
londons_explorer t1_j7u38tk wrote
Shadows and the way light interacts/reflects/refracts seem to be emergent behaviour of diffusion image models.
Ask for "A koala next to a glistening wine glass", and you'll probably get cool optical effects on the koala that the model has never seen before.
londons_explorer t1_jegw8tx wrote
Reply to comment by CommunismDoesntWork in [News] Twitter algorithm now open source by John-The-Bomb-2
The claims are plausible accidents from a technical perspective. It's very possible for a system which does blocklists to choke up on the longest Blocklist it has ever seen and fail to add new things to the list.