Big Data Smart cities Open source algorithms – Predictive and Real time algorithms

Over the last week, speaking at the  Smart cities industry summit – speakers (you can download my presentation in the link below)  , I had many questions about the approach we are taking and especially around the ideas of Big Data Smart cities Open source algorithms


Here is some more information


  • In a nutshell, our goal is to go beyond the simplistic discussion of ‘What makes a city smart?’ to real, practical implementations via code/algorithms


  • Open data for Smart cities has been explored before. We extend the Open Data discussion to open source Big Data algorithms.


  • We work in close collaboration with a specific city (Liverpool) via Connected Liverpool but we adopt an open strategy – which means we share via open source and we welcome collaboration with cities and others


  • This is an early space which will evolve rapidly. At this point, our objective is more social (ex – to create an intellectual hub of technical expertise).



  • What is different now? The algorithms have existed for a while but the application in this context (Smart cities) is new.



  • Thus, while machine learning algorithms have existed for some time, the inclusion of Big Data principles is new .. Specifically, use of Apache Mahout and Real time Big Data techniques like Storm (which could be seen as a ‘real time Hadoop’), Berkeley Spark etc


  • How do we add this all together? A proposed Big Data Lifecycle showcase


  • The Big Data Lifecycle showcase comprises of:

-  Capturing data(sensors)

-   Store and augment data (using large earth observation datasets – see my previous blogs), APIs and

-    Services (algorithms, appstores)

Thus, this strategy extends the current Open data, APIs, Appstores discussion for Smart cities along

a)    Big Data (Apache Mahout and other Big Data techniques)

b)    Algorithms (Predictive and real time) and

c)    Earth observation data

By sharing as open source, we hope we can contribute and collaborate with other cities

Related posts:

1) My presentation on Smart cities industry summit – Creating Open source Smart city algorithms for Big Data by combining sensor data with satellite data .. 

2)  Big data for Smart cities – How do we go from Open Data to Big Data for Smart cities,

3)  Open source Big Data Smart City algorithms .

Image source: