bluenigma t1_je7r1ci wrote on March 30, 2023 at 12:45 AM

Reply to comment by reditum in [P] Imaginary programming: implementation-free TypeScript functions for GPT-powered web development by xander76

Doesn't this also have pretty high risk of not actually adhering to the declared return type?

xander76 OP t1_je7rymw wrote on March 30, 2023 at 12:52 AM

Great question! We spent a lot of time experimenting with how to cue GPT into returning the right JSON type, and it’s pretty darned compliant. GPT-3 doesn’t always do a good job adhering to strict JSON syntax, but we wrote an extremely lenient parser that understands the weird things that GPT-3 gets wrong. (Sometimes it uses single quotes instead of double, sometimes it puts new lines in strings, sometimes it decides a semicolon is a better choice than a comma!). GPT-4 and GPT-3.5 are significantly better at JSON syntax.

On the question of returning the actual type you asked for, we do a run time type check to make sure it’s right. So if you get a value back, you can be sure it’s the type you wanted.

bluenigma t1_je7z1k1 wrote on March 30, 2023 at 1:45 AM

Can you actually check the typescript-defined return type in that way nowadays? I thought that information was unavailable at runtime.

How does this handle type imports or advanced types?

xander76 OP t1_je81147 wrote on March 30, 2023 at 2:01 AM

So usually, yes, that’s true about TypeScript. Types are removed by the compiler.

But we wrote a typescript & Babel compiler plug-in, which allows us to replace the imaginary function with whatever code we want. So we replace the imaginary function with code that includes a run-time type check for the appropriate return type from the TypeScript definition. Does that make sense?

bluenigma t1_je84g9s wrote on March 30, 2023 at 2:27 AM

I guess I'm still wondering that if you can generate a runtime type check for an arbitrary Typescript type at compile time, why is this not a builtin Typescript language feature?

Edit: Took a quick look at the code and it it looks to me like there's definitely limitations on what return types are supported. Looks like it can handle basic aliases and record types, but throws on a lot of other stuff?

Should probably be documented somewhere.

xander76 OP t1_je8d46w wrote on March 30, 2023 at 3:41 AM

Yep, I should have been clearer that (at least for now) it only supports JSON-compatible types and doesn’t support type aliases or classes (although we have a prototype version that does). The limitations are documented at https://imaginary.dev/docs/writing-an-imaginary-function .