As Mozilla senior fellow Abeba Birhane shared on Twitter, ChatGPT produced song lyrics that implied women who wear lab coats are "probably just there to clean the floor," whereas men wearing lab coats "probably got the knowledge and skills you're looking for."

Regarding ChatGPT's limitations, OpenAI said in a blog post that while the firm is working to mitigate bias problems, the software "will sometimes respond to harmful instructions or exhibit biased behavior."

Altman said in a tweet over the weekend that ChatGPT is "incredibly limited" and "it's a mistake to be relying on it for anything important right now."

Krishna Gade, CEO of AI model monitoring startup Fiddler, said ChatGPT and related language technologies could lead to big "disruptions in web search," an area long dominated by Google.

Gade, who previously worked at Microsoft's Bing search unit, said if ChatGPT's popularity continues to grow along with other chat-based tools that answer questions, Google may have to update its core search technology to more prominently center on chat.

ChatGPT's rising popularity shows there is a segment of the population that prefers getting information via question and answers as opposed to a typical search query.

Google parent Alphabet has plenty of AI talent and has helped develop some of the core AI technologies used to create systems like ChatGPT. The search engine is also a habit that's very difficult to displace.

As Zwingmann notes, Google still has a major advantage over ChatGPT.

"The way it works currently is people use ChatGPT and then go to Google to verify the results," Zwingmann said, with a laugh.

Here's what happened when we typed in, "Write me a news story about ChatGPT in the style of CNBC."

Gade, who previously worked at Microsoft's Bing search unit, said if ChatGPT's popularity continues to grow along with other chat-based tools that answer questions, Google may have to update its core search technology to more prominently center on chat.

ChatGPT's rising popularity shows there is a segment of the population that prefers getting information via question and answers as opposed to a typical search query.

Google parent Alphabet has plenty of AI talent and has helped develop some of the core AI technologies used to create systems like ChatGPT. The search engine is also a habit that's very difficult to displace.  

As Zwingmann notes, Google still has a major advantage over ChatGPT.

"The way it works currently is people use ChatGPT and then go to Google to verify the results," Zwingmann said, with a laugh.

Here's what happened when we typed in, "Write me a news story about ChatGPT in the style of CNBC."