StartChatAlpha Colab:
In this video I look at the Starcoder suite of models, how they were made and how they work. This collection has been developed through a collaboration of Hugging Face and other contributors, with an emphasis on open-source code modeling. In particular, the base models have been trained with 15 billion parameters and for a trillion tokens. A distinctive feature of StarCoder is its ability to generate continuous code and also fill in gaps in existing code, which I discuss in more detail later.
As I dive deeper into the models, I explore the applications of StarCoder, including a VS code plugin, which enables the model to operate in a similar fashion to Copilot, and a model that detects personally identifiable information (PII) – a highly useful tool for businesses that need to filter sensitive data from documents. I also delve into the different versions of the model, such as the base model and StarCoder StarChat Alpha, a fine-tuned version of the base model that's more adept at text generation. One highlight is the model's impressive sequence length of 8,000 tokens, rivaling GPT-4's base model.
My Links:
Twitter –
Linkedin –
Github:
00:00 Intro
00:27 StarCoder Blog
02:52 Tech Assistant Prompt
05:00 Different Models
07:53 StarCoder Space: Code Completion Playground
09:13 StarChatAlpha
12:16 Code Time: StarChatAlpha Running Locally
