Data Model and Services
To present the required information in the Dashboard, we need to have a proper data at hand. In essence, we need to collect the data that we need to transform into something understandable or readable, regardless that we have information that fits the requirement immediately or we need to transform and integrate the data using Data Transformation and Load. That data is collected from different sources and in different ways – some of that data is static (it is collected once and it is rarely changed) and some of that data is chronological in its nature (it is bounded to a specific time or timeframe and is changed or published at regular times).
To successfully manage that data and to open that data to a different services (including Dashboards) we need a proper Data Model for the Cities. That Data Model enables us to be more specific and describe how we manage the data and how we describe the data that we have or connect to. Dashboards, other services and Applications are connecting to that data through the Data Model, and even if they can connect to the data directly, the most efficient and correct way to do that is using the Model. That approach give us several specific advantages like better control over the way how we publish and maintain the data, how we develop interfaces that enables other apps to use that data or how we make data portable or linked for the other data consumer.
Building services that consume that data through the Data Model is straightforward from there – we build and use Application Programming Interfaces that share data or we build the connectors that enable us to publish that data in any specific data formats, that are either human or machine readable. Good model also help us to build the applications that are fully or partially transferrable between data domains or data stores.
Data Stores
Data Stores are just a datasets that are holding a data for you – but since we have a variety of different data providers and different requests on the performance and information retrieval services, we need to develop additional stores that are keeping the specific data for your needs. Some of the data is the data that is coming from a relational data stores that are created and used by the applications, some of the data is coming from connectors that are enabling Internet of Things (devices) and some of the data is actually data that is not sitting primary at this location, but is linked and replicated to the dataset from other location around the world.
Open Data Store – one of the first services / applications that we provide with the Data Model is the Open Data Store. It actually collects, publish and maintains the data if the Open Data environment where data in the store is prepared to be consumed for the external stakeholders. This Data Store is based on standard based platform for open data management which is used in 70+ countries to drive their national open data portals. It is fully extendable and it is a primary data store that platform is using.
Big Data Store – this is the data store that is holding a large amounts of “big data” – sometimes data that was collected over a longer period of time and now it resides in larger volume, or data that is generated from the different devices, or sometimes called from “Internet of Things”. That type of data is very hard to process in some meaningful time, but today it is a known problem with a known solutions. So, we are using Internet based volumes that are holding large volumes of structured data that is analyzed using different techniques to produce snapshots of data that can be addressed in some way.
Connected Data Store – linked data is really popular these days, and instead of copying or moving the data into the sets that will be locally analyzed, we are actually moving links that describe the data (using the metadata).
Get Connected