Official research release for the CodeGen1 models (2B, 6B, 16B) for Program Synthesis as presented in ICLR 2023:
Title: CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
Authors: Erik Nijkamp*, Bo Pang*, Hiroaki Hayashi*, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong (* indicates equal contribution)
The models are available on the HuggingFace Hub.
Program synthesis in the form of auto-regressive sampling can be performed as follows:
importtorchfromtransformersimportAutoTokenizer, AutoModelForCausalLMtokenizer=AutoTokenizer.from_pretrained("Salesforce/codegen-2B-mono") model=AutoModelForCausalLM.from_pretrained("Salesforce/codegen-2B-mono") inputs=tokenizer("# this function prints hello world", return_tensors="pt") sample=model.generate(**inputs, max_length=128) print(tokenizer.decode(sample[0], truncate_before_pattern=[r"\n\n^#", "^'''", "\n\n\n"]))@article{nijkamp2022codegen, title={CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis}, author={Nijkamp, Erik and Pang, Bo and Hayashi, Hiroaki and Tu, Lifu and Wang, Huan and Zhou, Yingbo and Savarese, Silvio and Xiong, Caiming}, journal={ICLR}, year={2023} }