Small language models (SLMs) are becoming increasingly popular because they are more efficient and cost-effective than larger language models (LLMs). SLMs are easier to adopt, more energy-efficient, and cheaper to train and deploy than LLMs. They are also more resilient to cyber-attacks and have a smaller carbon footprint. Additionally, SLMs meet regulatory requirements regarding copyright, reliability of data, and data privacy. While not meant to replace LLMs, SLMs offer easier scalability for industry-specific applications. Experts predict SLMs will have a significant impact on the market by 2025. Research from Apple has shown that using FlashAttention, a technology that improves efficiency, and choosing the right hardware and training schemes can further optimize SLMs. This is particularly helpful for institutions with limited resources. In th ...