Following the commercial release of tools like ChatGPT, it has become clear that generative AI technology will have a marked impact on higher education. In the midst of widespread discussions on how generative AI can best be leveraged in teaching and research contexts, universities are exploring how they can provide secure access to this technology for faculty, staff, and students. In our recent blog post on university custom AI platforms, we highlighted three institutions—the University of Michigan, Harvard University, and the University of Tennessee Knoxville—that have made generative AI tools available to their communities. These examples revealed some of the key questions and challenges universities are facing in relation to generative AI use on their campuses, including protecting data security, selecting vendors, and ensuring access to the technology moving forward.

Since then, the University of California San Diego has launched TritonGPT, currently available in beta by invitation only. TritonGPT, a language model with a ChatGPT-like interface, was trained to answer detailed questions about UC San Diego’s policies, procedures, and campus life. Though TritonGPT is designed to serve a purpose very similar to the platforms at Harvard, Michigan, and Knoxville, UC San Diego’s approach stands out because instead of relying on Microsoft’s Azure OpenAI service, TritonGPT is hosted on local infrastructure and optimized for use in administrative operations.

In the following interview, we discussed TritonGPT with three individuals at UC San Diego who were central to the tool’s creation: Brett Pollak, director of workplace technology services; Adam Tilghman, analyst/architect with academic technology services; and Jack Brzezinski, senior AI architect. We hope what they shared will be useful for other universities weighing their options in terms of how to approach offering AI tools to their communities.

Note: The answers to some of these questions were crafted with the assistance of TritonGPT.

How and why did UC San Diego decide to develop TritonGPT? Who will it be available to, and why do you think it will be a valuable resource?

TritonGPT was developed by UC San Diego in response to our increasing reliance on commercial generative AI products like OpenAI’s ChatGPT and Google Bard for administrative tasks such as generating text for emails, summarizing documents, and assisting with documentation for processes and procedures.

We identified a unique opportunity to create TritonGPT, a system designed not only as an alternative to commercial generative AI tools but also as a specialized resource for efficiently addressing an extensive array of inquiries specific to UC San Diego. This includes questions related to academics, administration, finance, and research. TritonGPT leverages advanced indexing of key UC San Diego websites, enabling it to provide comprehensive and accurate responses to a diverse range of questions.

TritonGPT also includes a Job Description Writer, engineered to streamline the job description creation process for hiring managers. This tool uses a conversational interface that engages hiring managers in a dialogue, capturing the specific requirements and nuances of each role. The AI then crafts language that not only complies with established job card standards but also accurately reflects the unique characteristics of each position. This feature significantly reduces the time and effort involved in drafting job descriptions, ensuring they are both precise and tailored to the individual needs of the role.

Our team is in the process of creating a “Fund Manager Coach,” designed to connect similar roles across different departments that typically function independently. Recognizing the crucial role of fund managers in overseeing grants and managing departmental finances, this tool will enhance their understanding of UC San Diego’s financial policies and procedures. The “Fund Manager Coach” will provide personalized advice and targeted answers to specific questions. The potential is there to provide position-specific coaches for other positions that exist throughout the campus.

Currently in its pilot stage, TritonGPT is set to become accessible to all UC San Diego staff and students. We’re expanding our hardware to support the increased user base, ensuring TritonGPT can efficiently serve a wider audience.

TritonGPT is hosted on local infrastructure at the San Diego Supercomputer Center. What other campus and off-campus partnerships were involved in its creation? What do you see as the short- and long-term benefits of taking this approach?

The San Diego Supercomputer Center (SDSC) provides the infrastructure and support for hosting TritonGPT, while IT Services provides the software and domain expertise to train and refine the large language model.

The short-term benefits of hosting TritonGPT on local infrastructure include:

  • Cost control: By hosting on premise, our costs are contained to the infrastructure needed to host the model and associated components. We are not charged for licensing or token limits related to the large language model.
  • Improved security and control over data: By hosting TritonGPT on premise, UC San Diego has greater control over how its data is used and shared.
  • Fine tuning and flexibility: We are able to bring in other components to enhance the offering. This can range from bringing in multiple models that work together in addition to software to assist with chain of thought reasoning.

The long-term benefits of this approach include:

  • Enhanced collaboration and innovation: By partnering with the SDSC, UC San Diego can leverage the center’s expertise and resources to explore new applications and use cases for AI technology, fostering collaboration and innovation.
  • Cost savings: While the initial investment in hardware is higher, hosting TritonGPT on local infrastructure is forecasted to be about 40 percent cheaper over the next three years compared to similar cloud AI services.
  • Customization and adaptability: Our advancement in text-to-SQL generation, which allows for natural language interaction with structured datasets like those in our data warehouse, significantly enhances our integration and security capabilities. This positions us better for handling complex data interactions.

In addition to SDSC for hosting, we are partnering with Danswer, a startup company founded by two UC San Diego alumni. The Danswer platform started off as open source and dovetails nicely with our on-premise approach. Their software assists with rapid integration of the LLM with UC San Diego specific data sources in addition to providing a chat-like interface that is familiar for those that have used ChatGPT.

What have been the main challenges in developing this tool? Beyond just the specific case of UC San Diego, what do you think the main challenges are for universities that want to offer AI services to their communities?

Transitioning to hosting a large language model and related platforms was a smoother process for us, given our existing support for the Data Science and Machine Learning Platform for instructional purposes. However, managing community expectations presents a challenge. It’s one thing to advise on using external products like ChatGPT; it’s quite another to be accountable for the output of a system we support. Fine-tuning the model for accurate responses is a substantial task. Identifying the right expertise for optimizing the system—through customized prompts, content management, or integrating smaller models for ensemble learning—is challenging in this emerging field. It involves extensive research and experimentation to improve accuracy.

The other area of focus has been training staff how to work with GenAI models to get the most out of them. We’ve partnered with our Operational Strategic Initiatives team to develop some on demand and in-person AI trainings and workshops.

We’ve observed some apprehension within the community regarding GenAI. The potential for errors, misleading information, and bias in large language models is a significant concern. This necessitates careful consideration when deploying services reliant on these models for generating answers. We emphasize the importance of staff verifying the accuracy of AI-generated content before use or implementation. Our strategy involves a gradual rollout of these services, coupled with gathering feedback and continuous refinement.

What is your longer term vision for TritonGPT? Are you planning to offer other AI tools to your community in the future?

The AI industry is advancing swiftly, evidenced by recent developments: OpenAI has launched ChatGPT Enterprise; Microsoft and Google are incorporating their AI solutions into their platforms; and ServiceNow, already embedded with ML/AI features, has announced significant upgrades to its platform in the near future.

In addition to TritonGPT, our portfolio of AI services is expected to evolve significantly over the next six months. As many of our enterprise vendors are introducing new AI services, we will actively assess and integrate these offerings as funding availability and contractual terms permit.

We are also eagerly anticipating the opportunity to support the implementation of generative AI for instructional purposes, following the establishment of clear guidelines and directives from our academic administration.