BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

The Rise Of AI Will Force Data Scientists To Evolve Or Get Left Behind

Following
POST WRITTEN BY
Rudina Seseri
This article is more than 7 years old.

The history of Artificial Intelligence is long, but it’s only been recently that technology companies and markets have begun to get excited about it… Why? Because now it works.

After a few decades of exploration of symbolic AI methods, the field shifted toward statistical approaches, that have as of late started working in a broad array of tasks due to the explosion of data and computing power, this in turn has led to machine learning and, most importantly, enabled deep learning.

This is great news for the tech industry. The downside is that there aren’t enough data scientists that understand deep learning. For those who do, there is a huge demand for their services. But the reality is that this is a fleeting moment in time, as the skill set of the data scientist will be rendered useless in 12-18 months. They will need to either evolve with new AI tools or become a new category of Machine Language Scientists.

The evolution of deep learning

Artificial intelligence has been around since the 1950s. First proposed as a concept by Alan Turing early in the decade, it was solidified as a field of study at the 1956 Dartmouth Conference, which used AI as a heading to encompass language simulation, neural networks modeled on human brains and expert systems. Though merely a concept then, AI was made possible by machine learning which used algorithms to examine data in order to make a decision or prediction.

While previously instructions had to be coded into a computer, machine learning, a 1980’s innovation, gave a system broad instructions which it could use to make snap judgments. For instance, a computer could “see” a cat by hand coding classifiers like edge detection for a cat shape to develop algorithms that could learn whether an object was a cat. Deep learning takes this idea but adds a huge neural network with millions of data points to allow a system to make an informed judgment. For instance, in 2012, Google researcher Andrew Ng used 10 million YouTube videos to “teach” a system what a cat looked like so the system could recognize a cat.

While the core concepts of deep learning have been around for a few decades, they didn’t gain a lot of traction because of technological barriers such as low computing power. In fact, academia itself only started focusing on deep learning after 2006 with the development of faster learning algorithms. The approach garnered deeper attention both in academia and the business world in 2012 with Google’s leap in deep learning particularly when it came to visual recognition at scale. Google’s perceived breakthrough brought new attention to deep learning. By 2015, Facebook applied deep learning to facial recognition with DeepFace, which automatically identifies and tags users in photos. Since then, the applications of deep learning have multiplied at a very fast pace, with better performance across (narrow) applications.

The talent gap

Whether it is basic machine learning or deep learning, the reality is that these recent advances in learning have created an unprecedented need for talent, creating a considerable gap between the demand and supply of data scientists, a highly trained segment of the workforce. In fact, even within the pools of data scientists, it’s only the youngest of generations who have been trained in the more advanced deep learning approaches, narrowing the available pool of talent even further.

As the AI wave becomes transformational across end-markets from enterprise to consumer platforms, from cybersecurity to robotics, the demand for data scientists is growing exponentially. The role of data scientists in this technology will assume a new level of importance and will evolve in a similar fashion to what we saw happen in the field of computer sciences with the development of computing.

Much like developers started using software for a broad array of applications, they started developing tools (e.g. programming languages, libraries, etc.) that would allow them to program at a higher level and tackle increasingly more complex problems.

As more sophisticated languages were developed, the number of programmers that needed to use lower level ones decreased, but the overall number of software developers needed just kept growing (and it still is!) This means that many previous developers had to continuously adapt to new languages, while also facing increasingly greater competition from new programmers, who just learned the new “easier” higher-level language.

Data science is following a similar path. As with software development, data scientists are automating lower level tasks and moving up in the “abstraction scale” and tackling a higher level and more complex tasks. In the process, they are also both automating tasks previously done manually by users/developers, and empowering them to create, in the case of developers, or use, in the case of users, increasingly powerful products.

In the short term, AI technologies are creating a real need for data scientists with demand far ahead of the supply of qualified talent. In the long run, as AI advances to establish and analyze causations as well as correlations, software rather than humans will perform these analyses. And, data scientists - at least the successful ones – will evolve from their current roles to becoming machine learning experts or some other new category of expertise, yet to be given a name.

Rudina Seseri is Founder and Managing Partner at Glasswing Ventures, a Boston-based venture capital firm, an Entrepreneur-In-Residence at Harvard Business School and an Executive-In-Residence for Harvard University’s Innovation-Lab.