Data Science, AI and Hype cycles

When in industry more than 50% of new roles are driven towards a specific skill set and when projections from various recruiting companies shows the world being short of certain skilled people and employers are scrambling to find certain type of resources in the market and are willing to pay a premium to get them on-board then it is a clear sign that we’re in a hype cycle.

The skill here is Data Science and resources Data Scientists.

New Keyword, New Hype Cycle

Those who have been in the industry long enough can recognize this. A decade back industry was going crazy for a similar skill known as Business Analysts, who are now found dime a dozen in the market (apologies if I’ve hurt someone’s sentiments, but you can’t escape the truth). I know certain organizations where the certain preference is being given to Data Modelers, Data Analysts & data Scientists instead of sitting Business Analysts. Orders are simple, upgrade or get left behind.

I dare to remind here, that I am by no means demeaning the efforts, all the data scientists have put in to get their studies, and PhD degrees, but I just want to highlight that in order to get the best outcomes of the analytics, data modeling and machine learning models built by data scientists, we have to put them in action with the existing systems and drive the actions based on the findings.

To achieve that, we need to see the models both data and algorithmic models as part of the overall system. Yes, we can get the best model built, but we practically can’t keep running those models manually and keep having a hard dependency on the data scientist who built the model to keep the model working and churning out the numbers.

The Positioning of Data Scientists in the market

I am very concerned about the trend that we as an organization and mainly the decision makers rely heavily on the data scientists to do the end to end work. While I myself have worked on multiple real-life machine learning project deployments and have worked with quite a few data scientists, but just like any other profession, all data scientists are not alike. Everyone has their strong points and favorite styles and types of models to work with.
As per my experience, a data scientist is fully versed with different types of models and possesses the ability to identify the best-suited models for a specified use case sill have limited knowledge of a bank’s or organization’s tech stack. A data scientist working in a startup pretty much as a green field and the opportunity to best establish the entire product architecture from scratch, but to fit a machine learning model into a legacy architecture is a lot trickier than what it seems.

You need someone who knows the existing system limitations and is progressive enough to scale up to the level of data scientist at least to talk some reasonable language and bring the best possible results to existing tech stack.

This exposes another limitation for data scientists too. As they are good with languages like Python, R and my SQL, not every organization is have these applications and databases natively present, so a separate application architecture needs to be put in place and then data science becomes part of the analytics module in a bigger picture or application architecture. There the data scientists find limited expertise as they may not be fully capable of building the right pipelines to existing systems. This is a job for engineering or development team that pretty much every organization has, so rather than putting a data scientist on a pedestal, organizations are better off blending them with the rest of the team best-suited a cross-learning can happen.

After having discussions with many organizations, around their initiatives in the data sciences and analytics, many companies are creating new departments of data analytics, which I feel will again end up creating silos within the organization. Data science need to be a horizontal service for all verticals rather than being a vertical of its own.

AI becoming a marketing buzzword

For any new technology to become successful and create a niche of itself or to transform the entire tech landscape, a key benefit or key value-add must exist that will keep bringing people back to the tech. Java introduced platform neutrality and .Net provided Platform neutrality with developer-friendly interfaces for coding. Both had their salient features. Using Data sciences to build AI / ML systems need to be use-case driven.

The current market hype is about throwing everything to AI and hoping something will stick. Also, the loss of jobs is creating a separate hysteria, even though the current systems are not yet mature enough to confidently say that they are better than humans to do the tasks. Only for a few certain tasks, we can say the maturity of the systems is acceptable enough to be relied upon.

People argue that Algo trading is going to replace human traders, we need to understand also grading existed for more than a decade now. Not everyone is a mathematical genius who can crack open the stock market and take the huge profits from it. Also, trading or CFT trading has been around and being attempted many times, in certain scenarios it succeeded but in certain it didn’t. This is common with all aspects of the life and various professions.

Media houses and online news sites are coming out with more click baits than real facts. for example, Newsweek coming out with an article in Jan-2018 "Robots can now read better than Humans, putting Millions of jobs at risk". In many cases they relegate the facts to side notes or tiny disclaimers, that will change the entire context of the news piece and no-one will read it. This is a sad truth. Magazines had to be sold and so are the newspapers, so expect more sensationalizing of the topic to happen in near future.

But, the case I wanted to highlight that the current maturity of the Artificial intelligence models is definitely not to the level of the hype is created. Yes, some models work well, but they do need human intervention in the form of validation or monitoring for the supervised learning & semi-supervised learning they have to go do. 

Companies started using AI everywhere. A normal MIS report now become intelligent report, a simple fraud management system with alerts is now touted as the next generation Intelligent Jarvis, and so forth. This makes good headlines in the newspapers or news websites but not necessarily great tech assets.

Conclusion

In the end, I would like to say that it is good that data is now taking the center stage to drive a lot of assessments and potentially hidden patterns that were elusive to us, but now we're overcompensating the processes by putting data scientists on a pedestal. Identify the right Use case and deploy the best models there. Let the value generation be the self-marketing of AI systems that you're putting through newly. Life would be much simpler that way.