The Digital Unit of BT Group has declared the opening of an innovative internal platform that will enable the organization to leverage large language models (LLMs) from suppliers like Anthropic, Meta, Claude, Cohere, and Amazon. The GenAI Gateway is a vital tool that BT Group will use as it integrates AI into its operations. It was developed in partnership with AWS and makes use of AWS Professional Services, Amazon Bedrock, Amazon SageMaker to provide secure, private access to a variety of natural-language processing and large language models.
While ad hoc use of LLMs is appropriate for testing and development, it is not a good fit for large-scale implementation; greater attention to cost control, security, and privacy is required.
Additionally, LLM performance must be watched for unanticipated errors (such “hallucinations”) and model decay over time (the point at which LLMs cease acting in a predictable manner). In the event that additional problems arise, the GenAI Gateway also protects BT Group from being “locked in” to a specific LLM.
Because the GenAI Gateway platform allows for per-use case budget tracking, it will incentivize BT Group engineers to select the appropriate model for the given use case at the most competitive price.
By consolidating platforms, BT Group can save redundant effort and resources while expanding the use of generative AI. Centralized administration of infrastructure, security configuration, and application programming interfaces (APIs) lowers the chance of error and the expense of keeping different LLMs for each use case.
The AWS-deployed GenAI Gateway, like every element of BT Group’s modular digital architecture, can only be accessed through secure APIs. GenAI Gateway leverages two fully managed services: Amazon Bedrock, which provides a single API access to a selection of high-performing foundation models from top AI firms such as AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon; and Amazon SageMaker, which combines a wide range of tools to enable high-performance, low-cost machine learning for any use case.
Enterprise search, chat history, FinOps billing per use case, fast security, and the utilization of several corporate data sources are all supported by the platform. Central privacy controls secure data in accordance with Group policy and applicable regulations.
These include separate tenants for each use case, the use of Personal Identifiable Information filters, the location of the data within the UK, and the isolation of trained models from one another.
The GenAI Gateway has safeguards built in to reduce the possibility of jailbreaks and harmful interactions. These guardrails block away queries that are not relevant to a particular application, ensuring that ethical and performance constraints are included from the beginning.
Gen AI Gateway is one of the several key enablers we are deploying to enable BT Group as an AI-enabled enterprise, and we will also use our “data fabric” data management platform to help enforce governing policies for how data can be used, as well as to manage access control and data sovereignty restrictions.
The first beta use cases for GenAI Gateway are available right now. A pilot being conducted by Openreach aims to streamline processes and increase productivity for its teams and clients of communications providers by compiling technical notes on Ethernet and full fiber jobs. There is also a live second use case that supports contract analysis for the business, legal, and procurement teams of the group.
Key Comments
Fabio Cerone, GM EMEA Telco at AWS, said: “The LLMs is showcasing how enterprises can effectively deploy generative AI at scale and speed. It’s been a brilliant, pioneering opportunity to collaborate and work backwards from the customer to provide a way to accelerate deployment of generative AI use cases into production with embedded security and compliance. The GenAI Gateway will trigger the flywheel effect in the adoption of generative AI, delivering quicker results for BT Group and its customers.”
Deepika Adusumilli, Managing Director, Data & AI, BT Group’s Digital Unit said: “AI is helping us reimagine the future of our company. We believe that where our data is a constant, we need flexibility with our LLMs. GenAI Gateway allows us to tap into this powerful new set of technologies at scale, in a way that is safe, responsible, flexible and scalable, delivering the ambition we have for AI to unlock the human potential within BT Group, today and in the future.”
