Big Data, Artificial Intelligence and Machine Learning

Like every industry, the investment community goes through ‘hot’ trends that grab everyone’s attention and a lot people’s dollars.

One of the reason that I’ve attended Schwab’s annual conference for the past 16 years is that I like to see these trends in the form of sales pitches to advisors.

I didn’t pick up on this in the very early days, but it became obvious after the 2008 financial crisis when the number of vendors selling alternative investments boomed. Then, after stocks soared and alternatives mostly failed to deliver, ‘smart beta’ strategies (described in more detail here) came into vogue.

Now that there are hundreds (if not thousands) of smart beta mutual funds and ETFs, the new wave of products (and sales pitches) is focused on new technologies: big data, artificial intelligence and machine learning.

I don’t pretend to be an expert on these matters, but here’s how I understand each of these terms:

  1. Big Data refers to extremely large data sets that can be analyzed to reveal patterns, trends and associations that would be difficult to determine through traditional analysis.
  2. Artificial Intelligence refers the idea that computers and machines can perform tasks that are normally associated with human behavior, such as planning, understanding language, recognizing objects and sounds, learning and problem solving.
  3. Machine learning is a form of artificial intelligence, but instead of humans programming computers and machines to perform the tasks mentioned above, machine learning is the science of getting computers to act without being explicitly programmed by humans.

Goldman Sachs reports that the world’s 13,000 public companies produce two million pages of annual reports and 30,000 of phone earnings call each year.

While you could hire an army of analysts to read those reports and listen to those calls, you could also use technology to speed up the process and potentially increase the number and quality of insights from the information.

Technologists could use big data techniques to transform the raw data from the reports and calls into organized data sets that would allow human analysts to see trends that may not otherwise be noticed.

Using artificial intelligence, those analysts could train the computers to find those trends.  And, in the best of all worlds, using machine learning, the computers could find trends that the analysts wouldn’t have even considered.

More and more, we’re pitched funds and strategies that purport to use all of these techniques.  And, as you might expect, I’m skeptical of a lot of the claims and representations at this point.

In my view, using big data techniques to structure vast amounts of data is a necessity at this point.  All of the firms that we work with use big data techniques for trading purposes, trying to get best execution when they are buying and selling securities.  I view big data is a positive.

Artificial intelligence and machine learning?  I’m not so sure.

A good friend of mine works for a very large, cutting edge quantitative firm that is experimenting with artificial intelligence and machine learning.  He told me that at this point, it’s more rudimentary than everyone makes it sound.

More importantly, he said something very interesting, which was that the computer had developed several trading strategies that turned out to be very profitable.  The trouble, however, was that the computer couldn’t say exactly how it had developed these strategies.

So, even though, the computer had learned something potentially useful, it couldn’t teach the humans what it had learned, which meant that the humans couldn’t say – even in hindsight – what risks had been taken to earn those profits.  Danger, Will Robinson, Danger!

Personally, I’m not ready to trust computers so explicitly.  There may be a time when it makes a lot of sense, but I don’t think that we’re there yet.  I have the feeling that the technology folks that tell us that it’s right around the corner may be getting ahead of themselves, but who knows.

It’s a brave new world out there, and while technology a lot of exciting potential careful stewardship and human judgement will always be a necessity.