The rapid expansion of artificial intelligence (AI) across various industries is prompting public sector organizations to embrace this transformative technology. However, these institutions encounter unique constraints related to security, governance, and operational stability that differentiate them from their private sector counterparts. As public sector executives express concerns about data security—reported by a Capgemini study to affect a significant percentage of leaders globally—it’s clear that the nature of government data, coupled with stringent legal requirements, necessitates a specialized approach to AI deployment. Han Xiao, Vice President of AI at Elastic, emphasizes the importance of control over sensitive information, indicating that government agencies face substantial restrictions on data movement and usage, complicating their efforts to integrate AI effectively.
Given these operational challenges, the large language models (LLMs) commonly used in the private sector may not be suitable for public agencies. Instead, small language models (SLMs) present a viable alternative. These models, which typically utilize billions rather than hundreds of billions of parameters, can be deployed locally to enhance data security and operational efficiency. SLMs not only mitigate the complexities associated with managing large models but also address the constraints of limited internet connectivity often experienced by government bodies. An empirical study has shown that SLMs can match or even outperform LLMs in specific tasks, thus enabling public sector entities to leverage sensitive data more effectively without compromising security or performance.
Moreover, SLMs can significantly improve the way government organizations search and manage vast amounts of unstructured data, such as technical reports and procurement documents. By employing advanced techniques like smart retrieval vector search, these models can index varied data formats, providing tailored responses that comply with legal standards. Xiao asserts that the potential of AI in the public sector extends far beyond simple applications like chatbots; it can revolutionize data interpretation, support executive decision-making, and enhance public access to services. By focusing on the efficiency and effectiveness of SLMs rather than the sheer size of AI models, public sector agencies can optimize their AI strategies while adhering to budgetary constraints and regulatory requirements.
Source: Making AI operational in constrained public sector environments via MIT Technology Review