pathwaycom/arc-task-gen ? reverse-engineered prompt
Reverse engineered prompt
Build me a tool that generates fresh ARC AGI style puzzle tasks that feel like the public ARC AGI 1 set, so I can use them for model evaluation without reusing the benchmark itself.
I want it to produce tasks in the usual ARC JSON format with train and test examples, and I want the generated tasks to be distribution matched to the public evaluation set as closely as possible. It should also include a way to label and inspect tasks, and a simple script to visualize them so I can check the output by eye.
Please make it easy to run from the command line, and include clear instructions for generating a batch of tasks and saving them out as tasks.json files. If there are any details about the ARC format or the public dataset that matter, look them up in the docs online if needed and make the project work end to end.
Are you gonna build this?
make sure you review the code using coderabbit