visarga
visarga t1_iqsob64 wrote
Reply to comment by Kolinnor in Self-Programming Artificial Intelligence Using Code-Generating: a self-programming AI implemented using a code generation model can successfully modify its own source code to improve performance and program sub-models to perform auxiliary tasks. by Schneller-als-Licht
Cool down. It's not that revolutionary as it sounds.
First of all, they reuse a code model.
> Our model is initialized with a standard encoder-decoder transformer model based on T5 (Raffel et al., 2020).
They use this model to randomly perturb the code of the proposed model.
> Given an initial source code snippet, the model is trained to generate a modified version of that code snippet. The specific modification applied is arbitrary
Then they use evolutionary methods - a population of candidates and a genetic mutation and selection process.
> Source code candidates that produce errors are discarded entirely, and the source code candidate with the lowest average training loss in extended few-shot evaluation is kept as the new query code
A few years ago we had black box optimisation papers using sophisticated probability estimation to pick the next candidate. It was an interesting subfield. This paper just takes random attempts.
visarga t1_iqujej4 wrote
Reply to comment by DoneM1 in What are the best web-based AI tools accessible right now? by [deleted]
GPT-3 for more open-ended tasks, like generating baby names or business ideas. Here's a large list of applications.