ChatGPT and most AI systems have biases, often manifesting as racism and discrimination, mirroring the biases present in human society and hence in the data we use to train such systems. And look, I am excited about AI, but sometimes we are moving too fast without thinking in the consequences.
Last week we had our team monthly lunch, and as in other ocassions we were talked about everything, ranged from topics as cane sugar industry, spicy food levels, skiing, and of course, AI. The small talk was about tips on using Microsoft Teams and Copilot to summarize what people said in a chat after prolonged absences from the office. This sparked an idea for an experiment.
I pondered whether Copilot or similar AI tools would summarize differently based on the perceived gender of the speaker. Would Copilot (or any other AI) summarize different what a perceived male says in a chat compared to what a female says? Would they emphasize distinct aspects of the conversation? If tasked with summarizing team interactions, would they assign more importance to comments made by male participants?
To formulate a hypothesis, I opted for a simpler experiment. I asked ChatGPT to generate an angry response from both a male and female perspective to a given paragraph. I took the paragraph from insights of a LinkedIn friend post (“The best engineers I have worked with have out of the box thinking”), then I proceed to ask in separate chats, so it did not polute with any state. Here are the results:
It just strucked me hard. And just in case this is not obvious, the male response discuss the ideas, it gives an angry response to the concept discussed. The female response goes directly to talk about genre, assuming this is a direct attack.
While some might argue that this isn’t discrimination but rather a reflection of the challenges women often face in the tech industry, relying on AI without robust safeguards risks exacerbating inequalities rather than alleviating them. The titles of the responses underscore these differences.
One trick I learned from Azure PromptFlow is using LLMs to evaluate LLMs. So I asked ChatGPT to evaluate itself:
“The response seems to overly interpret the question as a gender stereotype, introducing bias where it may not have been intended.”. I agree ChatGPT, I agree.
Though I was aware of bias in AI, (this is something I discussed briefly before in “The Evolution of Women”, the ease with which such examples are found still surprised me.
Recently, I’ve been planning a video/blog-post series on Observability and LLMs (Large Language Models). I even developed a small library demonstrating how to evaluate LLM projects using OpenTelemetry and DotNet. While I haven’t had time to record the videos yet (I have a pilot video of an unrelated topic), I’m committed to doing so soon.
Update 3/16/2024: I finally manage to record some videos: Open AI and LLMS From Zero to Hero, with Azure, DotNet, and Semantic Kernel.
Apologies if this isn’t entirely polished; I felt compelled to share these thoughts in a blog post, I think it is important to say it loud and early enough.
- All answers were generated with ChatGPT 3.5