You can count billionaire investor Steve Cohen among those who believes artificial intelligence is already making an impact on the business world.

The Point72 founder told CNBC's Andrew Ross Sorkin on "Squawk Box" that his financial firm has found ways for even the early AI models to save the company money.

"I'll give you one little anecdote. My CTO comes to me and says I can save the firm $25 million by using these LLMs to improve our efficiency," Cohen said, referencing his chief technology officer and the large language models like ChatGPT.

"Now, we're a nice sized firm. We're not a huge firm. So imagine what big companies can do. And that's just one thing, so it gives you a little bit of a look into what's possible," he added.

The excitement around AI has been one of the driving forces in the market rally of 2023 and early this year. The primary beneficiaries so far have been chipmakers like Nvidia and tech giants such as Microsoft that have direct business ties to these AI models, but another optimistic premise holds that the new technology will help all types of companies become more efficient.

Cohen called AI a "really durable theme" for investing and said that basically every company needs to be thinking about how it can change business.

"If you're a company and you're not thinking about this, you're going to wake up one day and go 'we're in trouble,'" he said.