The AI layer for the Enterprise and the role of IoT


According to Deloitte: by the “end of 2016 more than 80 of the world’s 100 largest enterprise software companies by revenues will have integrated cognitive technologies into their products”. Gartner also predicts that 40 percent of the new investment made by enterprises will be in predictive analytics by 2020. AI is moving fast into the Enterprise and AI developments can create value for the Enterprise. This value can be captured/visualized by considering an ‘Enterprise AI layer’. This AI layer is focussed on solving relatively mundane problems which are domain specific.  While this is not as ‘sexy’ as the original vision of AI, it provides tangible benefits to companies.


In this brief article, we proposed a logical concept called the AI layer for the Enterprise.  We could see such a layer as an extension to the Data Warehouse or the ERP system. This has tangible and practical benefits for the Enterprise with a clear business model. The AI layer could also incorporate the IoT datasets and unite the disparate ecosystem. The Enterprise AI layer theme is a key part of the Data Science for Internet of Things course. Only a last few places remain for this course!.


Enterprise AI – an Intelligent Data Warehouse/ERP system?

AI enables computers to do some things better than humans especially when it comes to finding insights from large amounts of Unstructured or semi-structured data. Technologies like Machine learning , Natural language processing (NLP) , Speech recognition, and computer vision drive the AI layer. More specifically, AI applies to an algorithm which is learning on its own.


To understand this, we have to ask ourselves: How do we train a Big Data algorithm?  

There are two ways:

  • Start with the Rules and apply them to Data (Top down) OR
  • Start with the data and find the rules from the Data (Bottom up)


The Top-down approach involved writing enough rules for all possible circumstances.  But this approach is obviously limited by the number of rules and by its finite rules base. The Bottom-up approach applies for two cases. Firstly, when rules can be derived from instances of positive and negative examples(SPAM /NO SPAN). This is traditional machine learning when the Algorithm can  be trained.  But, the more extreme case is : Where there are no examples to train the algorithm.


What do we mean by ‘no examples’?


a)      There is no schema

b)      Linearity(sequence) and hierarchy is not known

c)      The  output is not known(non-deterministic)

d)     Problem domain is not finite


Hence, this is not an easy problem to solve. However, there is a payoff in the enterprise if AI algorithms can be created to learn and self-train manual, repetitive tasks – especially when the tasks involve both structured and unstructured data.


How can we visualize the AI layer?

One simple way is to think of it as an ‘Intelligent Data warehouse’ i.e. an extension to either the Data warehouse or the ERP system


For instance,  an organization would transcribe call centre agents’ interactions with customers create a more intelligent workflow, bot etc using Deep learning algorithms.

Enterprise AI layer – What it mean to the Enterprise

So, if we imagine such a conceptual AI layer for the enterprise, what does it mean in terms of new services that can be offered?  Here are some examples

  • Bots : Bots are a great example of the use of AI to automate repetitive tasks like scheduling meetings. Bots are often the starting point of engagement for AI especially in Retail and Financial services
  • Inferring from textual/voice narrative:  Security applications to detect suspicious behaviour, Algorithms that  can draw connections between how patients describe their symptoms etc
  • Detecting patterns from vast amounts of data: Using log files to predict future failures, predicting cyberseurity attacks etc
  • Creating a knowledge base from large datasets: for example an AI program that can read all of Wikipedia or Github.
  • Creating content on scale: Using Robots to replace Writers or even to compose Pop songs
  • Predicting future workflows: Using existing patterns to predict future workflows
  • Mass personalization:  in advertising
  • Video and image analytics: Collision Avoidance for Drones, Autonomous vehicles, Agricultural Crop Health Analysis etc


These  applications provide competitive advantage, Differentiation, Customer loyalty and  mass personalization. They have simple business models (such as deployed as premium features /new products /cost reduction )


The Enterprise AI layer and IoT


So, the final question is: What does the Enterprise layer mean for IoT?


IoT has tremendous potential but faces an inherent problem. Currently, IoT is implemented in verticals/ silos and these silos do not talk to each other. To realize the full potential of IoT, an over-arching layer above individual verticals could ‘connect the dots’. Coming from the Telco industry, these ideas are not new i.e. the winners of the mobile/Telco ecosystem were iPhone and Android – which succeeded in doing exactly that.


Firstly, the AI layer could help in deriving actionable insights from billions of data points which come from IoT devices across verticals. This is the obvious benefit as IoT data from various verticals can act as an input to the AI layer.  Deep learning algorithms play an important role in IoT analytics because Machine data is sparse and / or has a temporal element to it. Devices may behave differently at different conditions. Hence, capturing all scenarios for data pre-processing/training stage of an algorithm is difficult. Deep learning algorithms can help to mitigate these risks by enabling algorithms to learn on their own. This concept of machines learning on their own can be extended to ‘machines teaching other machines’. This idea is not so far-fetched and is already happening, A Fanuc robot teaches itself to perform a task overnight by observation and through reinforcement learning. Fanuc’s robot uses reinforcement learning to train itself. After eight hours or so it gets to 90 percent accuracy or above, which is almost the same as if an expert were to program it. The process can be accelerated if several robots work in parallel and then share what they have learned. This form of distributed learning is called cloud robotics


We can extend the idea of ‘machines teaching other machines’ more generically within the Enterprise. Any entity in an enterprise can train other ‘peer’ entities in the Enterprise. That could be buildings learning from other buildings – or planes or oil rigs.  We see early examples of this approach in and Einstein. Longer term, Reinforcement learning is the key technology that drives IoT and AI layer for the Enterprise – but initially any technologies that implement self learning algorithms would help for this task


In this brief article, we proposed a logical concept called the AI layer for the Enterprise.  We could see such a layer as an extension to the Data Warehouse or the ERP system. This has tangible and practical benefits for the Enterprise with a clear business model. The AI layer could also incorporate the IoT datasets and unite the disparate ecosystem.  This will not be easy. But it is worth it because the payoffs for creating such an AI layer around the Enterprise are huge! The Enterprise AI layer theme is a key part of the Data Science for Internet of Things course. Only a last few places remain for this course!.

Data Science for Internet of Things course – Strategic foundation for decision makers

Data Science for Internet of Things course - Strategic foundation for decision makers 

To sign up or learn more email [email protected] The course starts in Sep 2016

We have had a great response to the Data Science for Internet of Things course. The course takes a technological focus aiming enabling you to become a Data Scientist for the Internet of Things. I also had many requests for a Strategic version of the Data Science for Internet of Things Course for decision makers.

Today, we launch special edition of the course only for decision makers.

The course is based on an open problem solving methodology for IoT analytics which we are developing within the course.

 Why do we need a methodology for Data Science for IoT?


IoT will create huge volumes of Data making the discovery of insights more critical. Often, the analytics process will need to be automated. By establishing a formal process for extracting knowledge from IoT applications by IoT vertical, we capture best practise.

This saves implementation time and cost. The methodology is more than Data Mining (i.e. application of algorithms) – but rather, it leans more to KDDM (Knowledge Discovery and Data Mining) principles. It is thus concerned with the entire end-to-end Knowledge extraction process for IoT analytics.

This includes developing scalable algorithms that can be used to analyze massive datasets, interpreting and visualizing results and modelling the engagement between humans and the machine. The main motivation for Knowledge Discovery models is to ensure that the end product will be useful to the user.

Thus, the methodology includes aspects of IoT analytics such as validity, novelty, usefulness, and understandability of the results(by IoT vertical). The methodology builds on a series of interdependent steps with milestones. The steps often include loops and iterations and cover all the processes end to end (including KPIs, Business case, project management). We explore Data Science for IoT analytics at multiple levels including Process level, Workflow level and Systems level.

The concept of a KDDM process model was discussed in 1990s by Anand, Brachman, Fayyad, Piatetsky-Shapiro and others. In a nutshell, we build upon these ideas and apply them to IoT analytics. We also create code in Open source for this methodology.

As a decision maker, by joining the course, you have early and on-going access to both the methodology and the open source code.

Please contact us to sign up or to know more [email protected]

Testimonials for our current course

 Jean Jacques Bernand – Paris – France

“Great course with many interactions, either group or one to one that helps in the learning. In addition, tailored curriculum to the need of each student and interaction with companies involved in this field makes it even more impactful.

As for myself, it allowed me to go into topics of interests that help me in reshaping my career.”

Johnny Johnson, AT&T – USA

“This DSIOT course is a great way to get up-to-speed.  The tools and methodologies for managing devices, wrangling and fusing data, and being able to explain it are taking form fast; Ajit Jaokar is a good fit.  For me, his patience and vision keep this busy corporate family man coming back.”

Yongkang Gao, General Electric, UK.

“I especially thank Ajit for his help on my personal project of the course — recommending proper tools and introducing mentors to me, which significantly reduced my pain in the beginning stage.”

karthik padmanabhan Manager – Global Data Insight and Analytics (GDIA) – Ford Motor Pvt Ltd.

“I am delighted to provide this testimonial to Ajit Jaokar who has extended outstanding support and guidance as my mentor during the entire program on Data science for IoT. Ajit is a world renowned professional in the niche area of applying the Data science principles in creating IoT apps. Talking about the program, it has a lot of breadth and depth covering some of the cutting edge topics in the industry such as Sensor Fusion, Deep Learning oriented towards the Internet of things domain. The topics such as Statistics, Machine Learning, IoT Platforms, Big Data and more speak about the complexity of the program. This is the first of its kind program in the world to provide Data Science training especially on the IoT domain and I feel fortunate to be part of the batch comprising of participants from different countries and skill sets. Overall this journey has transformed me into a mature and confident professional in this new space and I am grateful to Ajit and his team. My wish is to see this program accepted as a gold standard in the industry in the coming years”.

Peter Marriott – UK –

Attending the Data Science for IoT course has really helped me in demystifying the tools and practices behind machine learning and has allowed me to move from an awareness of machine learning to practical application.

Yair Meidan Israel –

“As a PhD student with an academic and practical experience in analytics, the DSIOT course is the perfect means by which I extend my expertise to the domain of IoT. It gradually elaborates on IoT concepts in general, and IoT analytics in particular. I recommend it to any person interested in entering that field. Thanks Ajit!”

Parinya Hiranpanthaporn, Data Architect and Advanced Analytics professional Bangkok

“Good content, Good instructor and Good networking. This course totally answers what I should know about Data Science for Internet of Things.”


Sibanjan Das – Bangalore

Ajit helped me to focus and set goals for my career that is extremely valuable. He stands by my side for every initiative I take and helps me to navigate me through every difficult situation I face. A true leader, a technology specialist, good friend and a great mentor. Cheers!!!

Manuel Betancurt – Mobile developer / Electronic Engineer. – Australia

I have had the opportunity to partake in the Data Science for the IoT course taught by Ajit Jaokar. He have crafted a collection of instructional videos, code samples, projects and social interaction with him and other students of this deep knowledge.

Ajit gives an awesome introduction and description of all the tools of the trade for a data scientist getting into the IoT. Even when I really come from a software engineering background, I have found the course totally accessible and useful. The support given by Ajit to make my IoT product a data science driven reality has been invaluable. Providing direction on how to achieve my data analysis goals and even helping me to publish the results of my investigation.

The knowledge demonstrated on this course in a mathematical and computer science level has been truly exciting and encouraging. This course was the key for me to connect the little data to the big data.

Barend Botha – London and South Africa –

This is a great course for anyone wanting to move from a development background into Data Science with specific focus on IoT. The course is unique in that it allows you to learn the theory, skills and technologies required while working on solving a specific problem of your choice, one that plays to your past strengths and interests. From my experience care is taken to give participants one to one guidance in their projects, and there is also within the course the opportunity to network and share interesting content and ideas in this growing field. Highly recommended!

- Barend Botha

Jamie Weisbrod – San Diego -

Currently there is a plethora of online courses and degrees available in data science/big data. What attracted me to joining the futuretext class “Data Science for ioT” is Ajit Jaokar. My main concern in choosing a course was how to leverage skills that I already possessed as a computer engineer. Ajit took the time to discuss how I could personalize the course for my interests.

I am currently in the midst of the basic coursework but already I have been able to network with students all over the world who are working on interesting projects. Ajit inspires a lot of people at all ages as he is also teaching young people Data science using space exploration.

 Robert Westwood – UK – Catalyst computing
“Ajit brings to the course years of experience in the industry and a great breadth of knowledge of the companies, people and research in the Data Science/IoT arena.”

Overall, the syllabus covers the following themes in 6 months

Note that the schedule is personalized and flexible for the strategic course

i.e. we discuss and personalize your schedule at the start of the course

  • Principles
  • Problem solving with Data Science: Is an overall process of solving Data Science problems(agnostic of a language) and covers aspects such as exploratory Data analysis)
  • IoT analytics (includes analysis for each vertical within iot. This will be ongoing throughout the course including in the methodology)
  • Foundations of R: The basics of one Programming language ( R ) and how to implement Data science algorithms in R
  • Time Series – which forms the basis of most IoT data (code in R)
  • Spark and NoSQL databases: Code in Scala and implementation in Cassandra
  • Deep Learning
  • Data Science for IoT Methodology
  • Maths and Stats – (this will also be ongoing but will be a core module)
we also have (from day one) what we call foundation projects where you work in groups with projets where you already have code etc. so you apply the concepts in context of a real situation