The arrival of efficient open-source Large Language Models (LLMs) like LLAMA 3 from Meta provides a unique opportunity to address the long-standing challenges faced by GovTech startups. When talking about Generative AI (genAI) for different GovTech use cases typically, three common challenges emerge:
- Cost efficiency
- Data Ownership
- Getting immediate value
In an ideal scenario, any GovTech startup needs to show that money spent on it is used efficiently with minimal overhead, that there are no data policy conflicts or breaches, and that its usage is justified and will bring immediate value, not in some distant future.
Last year, genAI technology showed its potential, but it would most likely be costly, incurring significant utilization fees or require the rollout of cloud-based GPU to run. It also had risks in regard to who owned the data and how that data would be used later. While other industries are slowly navigating these data challenges, in GovTech, the data comes from cities, government bodies, or other entities funded by the government, where data ownership is not always clear cut.
Open-source LLMs dramatically change this picture. You can run them in a very cost efficient way on existing infrastructure or build new infrastructure which is CPU-based, without the need of powerful–and substantially more expensive–GPUs. You can establish a collaboration model with the entity who owns the data. You can also create a derivative model with enriched data via Retrieval Augmented Generation (RAG). With recent advancements in LLMOps pipelines this has become simpler, faster, and more cost effective. With the right infrastructure in place, you can demonstrate use cases with government entity data in a matter of days, showing potential, real world applications.
These developments mitigate the risks of using generative AI in GovTech scenarios, opening up the possibilities of using LLMs and their powerful functionality. GovTech startups do not need to build strong Machine Learning teams and invest a lot of money upfront because they can offload that effort to genAI and LLM infrastructure providers, integrating capabilities as needed.
This also creates a unique opportunity for government entities directly. They can proactively collaborate with technology vendors like Engenai to quickly and easily create entity-specific LLMs, incorporating government data. For example, models that are tuned for traffic data are different from those that work with school budgets or with public works requests. Instead of contending with dozens of models from each vendor, each requiring their own security reviews and ongoing auditing and management, these government-owned models can be utilized by GovTech partners, providing consistent, auditable, and interoperable generative AI offerings.
