The rapid advancements in AI have transformed how organizations, including those in the public sector, approach problem-solving, decision-making, and service delivery. While large language models (LLMs) like GPT-4 have garnered significant attention for their broad capabilities, small language models (SLMs) are the real workhorses—efficient, secure, and cost-effective.
Together, they form a comprehensive AI solution: SLMs handle structured, mission-critical tasks in the front, while LLMs bring the power and flexibility to tackle broad, complex challenges in the back.
The AI mullet: why bad hair fashion is the perfect way for public sector agencies to implement AI solutions
I was sitting at a dinner recently that Invisible gathered for thought leaders from government and industry, and a question was posed: “Should we be relying on SLMs or LLMs for public sector applications?” The conversation first covered the natural topics:
- Implementing a series of SLMs for the government with smaller corpuses of dataDefined questions and use cases
- Speed to deployment
- Lower costs for compute
Then we ranged into the far-reaching data landscapes of government, with needs that include:
- Compiling and aggregating data across decades
- Make sense of things like imagery collected by hundreds of satellites, including literally millions of images across decades of work with varying missions and outcomes.
- Analyzing the millions of documents that the government has within its repositories
- Analyzing hundreds of thousands of policy documents from which Federal Regulations are composed
As it turns out, the government has the proverbial wild west of data—vast opportunities that are largely underutilized. It was at this point in the conversation that our genius idea was born: the AI Mullet, where SLMs are the business in the front and LLMs are the party in the back. And the conversation flowed effortlessly with a great analogy of terrible fashion 😊. (Please enjoy the analogy as a slightly tongue in cheek way of looking at a deeply complex challenge.)
SLMs: the business in the front
When I think about how to work with customers on implementing AI at scale in the public sector, the solutions must be aligned to their missions, with tangible use cases that show real speed to outcomes. Talking to public sector leaders, there is a real desire to make AI an outcome, reducing the burden on our civil servants, military leaders, and intelligence professionals.
Small Language Models (SLMs) are lean, specialized AI models optimized for efficiency and precision. They require minimal compute power, making them ideal for real-time applications, on-premise deployments, and edge computing scenarios. In the public sector, SLMs can serve as the structured backbone of AI operations, ensuring compliance, security, and cost efficiency.
LLMs: the party in the back
Large Language Models (LLMs) bring the breadth and depth of AI-driven insights. These models can generate human-like text, summarize information, and offer wide-ranging contextual knowledge. Their expansive capabilities allow them to handle dynamic, open-ended inquiries and strategic analysis—perfect for when the public sector needs to think big, innovate, and engage with citizens in new ways.
The AI mullet in action: hybrid AI at work in the public sector
The combination of SLMs and LLMs creates a powerful synergy that ensures efficiency in day-to-day operations while allowing for flexibility and innovation when required. Below are key areas where this hybrid AI approach delivers value.
Structured efficiency with domain-specific knowledge (SLMs)
Public sector agencies operate in highly specialized environments. SLMs, which can be fine-tuned to handle regulatory frameworks, public policies, and legal documents, provide precise, context-aware recommendations and keep everything running smoothly upfront.
Think about the military using an SLM trained on personnel frameworks and records to evaluate top performing soldiers, marines, sailors, airmen, and guardians. Such an SLM could evaluate how individuals were trained at specific points in their careers. It can help identify how did training correlates to performance, whether certain courses should be expanded or deprecated.
SLMs can be built to evaluate weather data, farming data, immigration—the list goes on and on. These models would have two express purposes for our public sector professionals:
- Eliminate repetitive manual work
- Increase productivity
Imagine not having to sift through report after report to fill in data fields required for logistics models. We could accelerate the public sector work without constantly expanding headcount.
Security and compliance up front (SLMs), creativity and scale in the back (LLMs)
Government agencies must adhere to strict security regulations. SLMs ensure that sensitive data stays locked down in a controlled environment. Meanwhile, LLMs can handle large-scale analytics, broad research inquiries, and citizen engagement efforts without exposing secure data.
LLMs have been accredited at multiple security levels across governments to handle these background challenges of wide ranging data solutions. Using SLMs as a front door to security represents an interesting way to get to specific data without exposing all of the data to individuals who may not need access. It’s another way to think through attribute access controls (ABAC) without the onerous time constraints that have plagued these systems in the past.
Scalability and cost optimization
Running LLMs at full throttle can be expensive. By offloading routine, structured tasks to SLMs, agencies can reduce dependency on high-cost computing while keeping LLMs available for complex problem-solving and strategic initiatives. In the world of data as well as high end hair fashion, cost always gets a vote!
Citizen engagement: Fast responses with depth
SLMs can efficiently handle frequently asked questions and pre-trained interactions, ensuring quick and reliable responses. LLMs step in when deeper, open-ended conversations arise, creating a seamless experience that balances speed with nuance. As with cost, speed to response is key for wider engagement of the public. As technology continues to close the gap within citizens' everyday lives for instantaneous information, responsive government is a key area for citizen engagement.
Operational efficiency and real-time processing
For critical, time-sensitive operations like emergency response, SLMs provide instant, actionable insights. Once the dust settles, LLMs offer in-depth analysis, reporting, and long-term strategic insights. As we think about the range of public sector, it is a massive market with vastly different use cases across civil, defense, intelligence, and state and local municipality governments. There is no one-size-fits-all application for this diverse set of groups. A hybrid set of models adapted for and specialized to the needs of the public sector is key.
Making the AI mullet work for public sector AI
For agencies looking to embrace this structured yet flexible AI strategy, a few best practices can maximize impact:
- Govern AI responsibly to ensure ethical use, bias mitigation, and compliance
- Invest in adaptable infrastructure to support both SLMs and LLMs without creating bottlenecks
- Prioritize interoperability to enable seamless integration across departments and use cases
- Engage stakeholders early to align AI capabilities with organizational goals and citizen needs
A balanced approach to public sector AI
The future of AI in governance isn’t about choosing between structured efficiency and expansive intelligence—it’s about embracing both. SLMs keep things practical and reliable, while LLMs inject the agility and depth needed for forward-thinking public sector innovation.
Together, they form the ultimate AI mullet: business in the front, party in the back, and a smarter, more responsive government for all.
