Abstract
Type systems and type inference systems can be used to help text and code generation models like GPT-3 produce more accurate and appropriate results. These systems provide information about the types of variables, functions, and other elements in a program or codebase, which can be used to guide the generation of new code or text. For example, a code generation model that is aware of the types of variables and functions being used in a program can generate code that is more likely to be syntactically correct and semantically meaningful. We argue for the specialization of language models such as GPT-3 for automatic program repair tasks, incorporating type information in the model's learning process. A trained language model is expected to perform better by understanding the nuances of type systems and using them for program repair, instead of just relying on the general structure of programs.