StarCoder is a decoder-only language model capable of generating code while demonstrating state-of-the-art results over other open source LLMs.
StarCoder has been trained on over 80 programming languages like Python, Java, Javascript, etc. while also being complimented by Github issues and PRs for learning the mapping between natural conversation and programming language.
The architecture consists of 15.5 billion parameters and is trained using multi-query attention along with flash attention to speed up the computation.
It offers 2 variants: StarCoderBase & StarCoder. Additionally, it also matches or outperforms the OpenAI code-cushman-001 model, making it one the best ai tools for code generation. It's available on hugginface at .
Please like this video and subscribe to @the.ai-loop if you find the content to be resourceful.
