Test Case Title
|TC6.1 - Managing DataSets via DataGate: ingest, search, download, upload, annotate, share|
Upload data set for knowledge base integration. Data Gate automatically assist the user to collects, enriches, integrates, and automatically reconciles referral data towards the Knowledge Base, smart city API, MicroServices
Enrich data set with metadata/annotations
Ingest and share data from other CKAN portals
Publish data towards other CKAN portals
Accessing data via tabular inspecting tool
Connection with dashboards via HTTPS/HTTP, LDAP
Using a PC or Mobile with a web browser. Access to DataGate, availability of some examples of datasets. They have been provided into the web site and FTP according to the following story. Access to another CKAN to exchange data sets among them: DataGate/CKAN and the other CKAN. For AUTOMATED ingestion see the ETL processes.
The following functionalities are available only for specific Snap4city users with specific privileges.
Expected successful result
Please note that some of the following links could be accessible only for registered users.
DataGate can be used to:
- Collect data sets coming from operators with their manual upload;
- Collect data sets coming from other CKAN;
- Publish data sets to public and/or registered users.
The possible actions that a DataGate authenticated user can perform on own data are the following:
- view the list of present Datasets
- Visualize the list of Resources available in every dataset
- Download the files of a dataset
- Visualize resource data
- Add a new Dataset
- Modify the dataset metadata
- Add resources (files) in a Dataset
- Modify the metadata of a resource
- Modify and update data of a resource using the Enhance Data view, available only for tabular resources (csv, xlsx, xls), compliant with the DataGate template
The user manual is also accessible as file on Google Drive: Datagate_tool_User_Manual.pdf
- Go to the Snap4city portal
- Make Login:
Fig. – Data Gate user home page (user: snap4city).
- From your home page you can see your activities on Datagate (click o the ‘Activity Stream’ tab). For example, the snap4city user has scheduled a real time ETL (from the Resource Manager tool) sending data on the portal and also an has also realise and IoTApplication from nodeRED writing in datagate.
Fig. – snap4city activities.
- To have a simple description related to Datagate, click on the menu ‘About’
- To search for all the Organizations (or group of users, stakeholders, etc.), click on the menu ‘Organizations’
- To search for a dataset, click on the menu (in the bottom of the page) ‘Datasets’
Fig. – Datasets’ list.
- Search a Dataset:
- Case A) you can search the datasets using: i) the search form (e.g. write ‘wifi’)
Fig. – case A) Dataset list, based on the search done, from the ‘Datasets’ page.
- Case B) you can use the tags on the left block (e.g. click on: ‘Antwerp > Show more tags > wifi), then it is visible a dataset list, based on the search done
Fig. – case B) Dataset list: search with tags.
- Case C) This is an alternative modality of Case B to make search and obtaining the same result. Starting from the home page or from your user profile page, click on the ‘Organizations’ tab and select a city (e.g. click on the ‘Antwerp’), search a dataset (use the search form, e.g. writing ‘wifi’).
- Case D) You can Obtain the same results clicking on the ‘Organization’ Main menu, clicking on the ‘Antwerp image, and using the textual filter.
- Select a Dataset:
- Click on Explore>Download to download the file
- Click on Explore>Preview, to visualize the data
Fig. – Click on File name.
Fig. – Data visualization.
- Starting from the Data visualization page, you can modify your data, and the use the lateral scroll bar and go at the bottom of the page, then click on the ‘Download csv’ button (as alterative modality to download the file).
Fig. – Download button.
Example 2: Upload data set for Knowledge base integration, follow these steps:
- Go to the snap4city home page, make the Login (if you are not already logged in), click on the ‘Data Set Manager: Data gate’ menu (on the left column) and then on the ‘About’ main menu of the Data Gate page. Then click on the Template LINK to download the csv template.
Fig. m – About page, and the Link to download the csv DataGate Template.
- Create your csv file (Fig. o) based on the DataGate template.
Fig. n – List of the csv DataGate Template rules.
Fig. o – the ‘’ csv file created starting from the DataGate Template (only the mandatory fields are visible).
- Go to the home page and select the Tab ‘Datasets’ from the menu, click on the button, if you have the permissions, you wiil see the ‘Add Dataset’ button, put the dataset name (e.g. ‘belangrijk museum’), insert the metadata, tags and upload the csv file (e.g. ‘belangrijk museum.csv’), then click on the button ‘Next: Add data’. As metadata you can put:
- Dataset name
- Upload the csv file
- Description (for example can be useful put the link of the Publisher, (e.g. something like this: “Data taken from: 'http://datasets.antwerpen.be”)
- A set of tags, that can be useful to retrieve the dataset
- A Licence
- The Organization, is the city to which the data are referring (e.g. ‘Antwerp’)
- Privacy, means that the dataset is visible only for the creator
- Public, means that the dataset is visible for the DataGate users
- Source: 'http://datasets.antwerpen.be/v4/gis/museumoverzicht.csv'
- Others non-mandatory: Author, author e-mail, version, etc.
Fig. – Add a New Dataset: Menu/Tab Datasets > Add Dataset button.