Gemini AI Lab Assistant
A demo for CCA's Hybrid Lab on integrating AI models and databases into physical prototypes.
Project Overview
As the lab manager of CCA's interactive electronic art makerspace, I explore new technology and created a demo to teach students AI integration with Gemini and ChatGPT in physical prototypes.
Inspired by CCA student Colin Chan's project using ChatGPT with an old phone, I repurposed the device, adding new Gemini logic for the Raspberry Pi 5. Now, I'm integrating our lab's information so AI can assist with student resource questions.
Tools: Gemini AI, Python, Raspberry Pi, Physical Computing
Implementation
I referenced TechMakerAI's tutorial for setting up the Raspberry PI to talk to Gemini AI. This code listens from the microphone, translates speech to text, sends text to Gemini, and then translates text to audio. I integrated the phone's hook switch and keypad into the program.
Programming Diagram:
Check out the code on Github:
Python Script for Gemini AI Phone AssistantUsed an anolog phone's microphone, speaker, hookswitch, and keypad to interact with the program running on the Raspberry PI. I used a USB audio adapter to convert the mic & speaker's analog signals to digital.
The keypad uses a matrix row-column configuration to represent its buttons. It checks each column by pulling it low, sees which row goes low in response, and uses that row-column intersection to identify the pressed button.
Code segment for defining and listening to keypad:
# Phone Keypad
ROWS = [4, 17, 27, 22]
COLS = [5, 6, 13]
KEYS = [
['1', '2', '3'],
['4', '5', '6'],
['7', '8', '9'],
['*', '0', '#']
]
# Set ROW lines as input
row_lines = []
for row_pin in ROWS:
line = chip.get_line(row_pin)
line.request(consumer="row", type=gpiod.LINE_REQ_DIR_IN, flags=gpiod.LINE_REQ_FLAG_BIAS_PULL_UP)
row_lines.append(line)
# Set COL lines as output
col_lines = []
for col_pin in COLS:
line = chip.get_line(col_pin)
line.request(consumer="col", type=gpiod.LINE_REQ_DIR_OUT, default_vals=[1])
col_lines.append(line)
def read_keypad():
for col_index, col_line in enumerate(col_lines):
col_line.set_value(0) # Pull current column low
for row_index, row_line in enumerate(row_lines):
if row_line.get_value() == 0: # Key press pulls line LOW
time.sleep(0.1) # Debounce
col_line.set_value(1)
return KEYS[row_index][col_index]
col_line.set_value(1) # Reset column
return None
# Define key handlers
def handle_key_1():
speak_text("What inventory questions do you have?")
def handle_key_2(): print("You pressed 2")
def handle_invalid(): print("Invalid key")
# Dictionary for key actions
key_actions = {
'1': handle_key_1,
'2': handle_key_2,
}
# Perform action based on key press
def perform_action(key):
action = key_actions.get(key, handle_invalid)
action()
def keypad_listener():
while True:
key = read_keypad()
if key:
print(f"Key Pressed: {key}")
perform_action(key)
time.sleep(0.3) # Debounce
Final Experience