Stability AI launches StableCode, an LLM for code generation

10/08/2023

Stability AI, a generative AI startup known for its Stable Diffusion text-to-image generation model, has announced the release of its new open large language model (LLM) called StableCode. StableCode is designed to assist users in generating programming language code and is available at three different levels: a base model for general use cases, an instruction model, and a long-context-window model that supports up to 16,000 tokens. The model is trained on data from the open-source BigCode project, with additional filtering and fine-tuning by Stability AI. Initially, StableCode will support development in programming languages such as Python, Go, Java, JavaScript, C, markdown, and C++.

StableCode’s long-context-window feature distinguishes it from other models, allowing for more specialized and complex code generation prompts. The longer context window enables StableCode to analyze medium-sized code bases that include multiple files, providing users with tailor-made suggestions for their specific code bases. StableCode also utilizes an approach called rotary position embedding (RoPE) for position outputs in its transformer neural network, rather than the ALiBi (Attention with Linear Biases) approach used by other models. Stability AI is looking to collaborate with the developer community to explore the applications and possibilities of StableCode in the generative developer space.