Over the last week, speaking at the Smart cities industry summit – speakers (you can download my presentation in the link below) , I had many questions about the approach we are taking and especially around the ideas of Big Data Smart cities Open source algorithms
Here is some more information
- In a nutshell, our goal is to go beyond the simplistic discussion of ‘What makes a city smart?’ to real, practical implementations via code/algorithms
- Open data for Smart cities has been explored before. We extend the Open Data discussion to open source Big Data algorithms.
- We work in close collaboration with a specific city (Liverpool) via Connected Liverpool but we adopt an open strategy – which means we share via open source and we welcome collaboration with cities and others
- This is an early space which will evolve rapidly. At this point, our objective is more social (ex – to create an intellectual hub of technical expertise).
- Which algorithms? Three classes: 1) predictive algorithms including machine learning algorithms 2) real time algorithms and 3) applying ideas of Complexity theory to Smart cities ex Oxford Martin University program on complexity theory (thanks to Philip Sheldrake for recommending the Oxford link)
- What is different now? The algorithms have existed for a while but the application in this context (Smart cities) is new.
- Specifically, we are interested in Machine learning algorithms . The goal of machine learning is use example data or data from past experience to solve/predict a given problem (see this book on Introduction to machine learning for an overview). Machine Learning algorithms are used in many domains for example – to predict customer behaviour from existing data. Machine learning algorithms have been used in applications like Real-Time Traffic Congestion Prediction(pdf)
- Thus, while machine learning algorithms have existed for some time, the inclusion of Big Data principles is new .. Specifically, use of Apache Mahout and Real time Big Data techniques like Storm (which could be seen as a ‘real time Hadoop’), Berkeley Spark etc
- How do we add this all together? A proposed Big Data Lifecycle showcase
- The Big Data Lifecycle showcase comprises of:
- Capturing data(sensors)
- Store and augment data (using large earth observation datasets – see my previous blogs), APIs and
- Services (algorithms, appstores)
Thus, this strategy extends the current Open data, APIs, Appstores discussion for Smart cities along
a) Big Data (Apache Mahout and other Big Data techniques)
b) Algorithms (Predictive and real time) and
c) Earth observation data
By sharing as open source, we hope we can contribute and collaborate with other cities
Related posts:
2) Big data for Smart cities – How do we go from Open Data to Big Data for Smart cities,
3) Open source Big Data Smart City algorithms .
Image source: http://mahout.apache.org/images/mantle-mahout.png
