1. Home
  2. Data Set Manager: Data Gate / CKAN federated

Data Set Manager: Data Gate / CKAN federated

×

Warning message

  • You can't delete this newsletter because it has not been sent to all its subscribers.
  • You can't delete this newsletter because it has not been sent to all its subscribers.
  • You can't delete this newsletter because it has not been sent to all its subscribers.

    Snap4City Overview: Data Gate, Resource Manager, Development Tools
click the image to open the tool (for registered users),  see the video

The DataGate/CKAN is a special module of CKAN open source portal for data set management and publication. CKAN supports the management of multiple collections, multiple organizations, etc. DataGate has been designed to collect/upload open data sets and process them automatically for data ingestion, field cleaning and regularizing, and for reconciliation of entities towards a Knowledge Base grounded on Km4City. The resulting cleaned data set is also accessible for further reuse and can be automatically published on the DataGate. It is a exceptional tool for shortening time to ingest and publish of datasets including for example:

  • referral data, statistical data, etc.;
  • open and not open data sets with geolocations;
  • descriptive data, that just describe the entities of the city and provide for them metadata, GPS and/or shapes

DataGate is also federated with other CKAN open data repositories in the world via the mechanism of Harvest: https://datagate.snap4city.org/harvest . So that thousands of data sets are accessible via DataGate and can be published in international networks of open data. The federation of new repositories / harvest can be requested to snap4city@disit.org by https://www.snap4city.org/drupal/contact 

Since it is based on CKAN, the solution also presents API for automated upload of Open Data Sets, and for the cross sharing of Open Data among different CKAN/DataGates. In Snap4City, the CKAN APIs are used for automatically managing and publish Data Sets in bulk by:

  1. extracting data collected from ETL and/or IOT, For example, the proposed solution allows to defined process that publish periodically all data of a given source: the collected sensors data of an area every day, etc. Go on Github of Disit Lab to download a large collection of ETL processes including those to work in automatic manner with DataGate publication and ingestion. 
  2. providing data sets produced as result of Data Analytics algorithms and solutions.

Please, note that some of the following links are accessible only for registered users.

Entry Login URL: https://datagate.snap4city.org/   or    https://datagate.snap4city.org/ssologin_handler

User Manual to download:

Collections of Datasets ingested by using the DataGate tool:

Most of the datasets have been automatically ingested by DataGate and its ETL processing data mining and are visible by accessing the following link:

  • Share