UConn Chatbot Code

Model Loading and Setup

from transformers import T5Tokenizer, T5ForConditionalGeneration

# Load the saved model and tokenizer
model_path = "saved_model_t5_normal_complete"
tokenizer = T5Tokenizer.from_pretrained(model_path)
model = T5ForConditionalGeneration.from_pretrained(model_path)

This code imports the necessary libraries from HuggingFace Transformers and loads a pre-trained and fine-tuned T5 model along with its tokenizer from a saved directory.

Response Generation Function

def generate_response(prompt):
    input_text = "question: " + prompt 
    print(" ")
    input_ids = tokenizer.encode(input_text, return_tensors="pt")
    
    # Updated parameters for model.generate()
    output = model.generate(
        input_ids, 
        max_length=150, 
        num_return_sequences=1,
        num_beams=5,       # Using beam search with 5 beams
        temperature=0.7,   # Moderately random outputs
        top_k=50,          # Only consider top 50 tokens for each step
        top_p=0.95         # Use nucleus sampling with p=0.95
    )
    response = tokenizer.decode(output[0], skip_special_tokens=True)
    
    return response

This function takes a user prompt, prepares it for the model by adding a prefix, and then generates a response using the T5 model. The generation parameters are carefully tuned for optimal response quality:

num_beams=5: Uses beam search to explore multiple possible response paths
temperature=0.7: Adds some randomness to responses without being too chaotic
top_k=50: Limits token selection to the 50 most likely tokens
top_p=0.95: Uses nucleus sampling to focus on the most probable tokens

Interactive Chat Loop

# Test the model with user prompts
while True:
    print(" ")
    user_input = input("Enter your prompt (or 'quit' to exit): ")
    if user_input.lower() == 'quit':
        break
    
    generated_response = generate_response(user_input)
    print("Model's response:", generated_response)

This simple interactive loop allows users to input questions about UConn. The loop continues until the user types 'quit', with each prompt being processed by the model to generate an informative response.

Sample Output

Enter your prompt (or 'quit' to exit): when was uconn founded?
 
Model's response: As a UConn tour guide, I would explain that the University of Connecticut was founded in 1881 as the Storrs Agricultural School. The university was officially renamed the University of Connecticut in 1889. Over the years, the university expanded and evolved into the comprehensive research university it is today. UConn was one of the first public universities in the country to offer a wide range of academic programs and research opportunities. The founding of the university in 1881 marked the beginning of UConn's commitment to academic excellence and innovation.