download_from_data_amsterdam_catalog

Get data and metadata from data.amsterdam.nl, unzip if needed and put the file in a local directory. To test run this command line: download_from_data_amsterdam_catalog https://data.amsterdam.nl/#?dte=catalogus%2Fapi%2F3%2Faction%2Fpackage_show%3Fid%3D5d84c216-b826-4406-8297-292678dee13c data

usage: download_from_data_amsterdam_catalog [-h] url output_folder

Positional Arguments

url
Insert full url from main result page of dataset. For example: https://data.amsterdam.nl/#?dte=catalogus%2Fapi%2F3%2Faction%2Fpackage_show%3Fid%3D5d84c216-b826-4406-8297-292678dee13c&dtfs=T&mpb=topografie&mpz=11&mpv=52.3731081:4.8932945
output_folder Specify the desired output folder path, for example: app/data

functions

extract.download_from_data_amsterdam_catalog.download_all_files(metadata, download_directory)

Download all files from metadata resources list.

Args:
  1. metadata: json dictonary from ckan with all the metadata including the resources list of all files.
  2. download_directory: path where to store the files from the files, for example data.
Result:
Unzipped and created dir filled with all data in the download_directory, if this does not yet exists.
extract.download_from_data_amsterdam_catalog.download_metadata(url)

Download files from data catalog response id.

Args:
url: full data.amsterdam.nl url of the desired dataset, for example: https://data.amsterdam.nl/#?dte=catalogus%2Fapi%2F3%2Faction%2Fpackage_show%3Fid%3D5d84c216-b826-4406-8297-292678dee13c
Result:
All the Metadata from this dataset as a json dictonary, with the owner, refresh data, resource url’s to the desired files, etc.
extract.download_from_data_amsterdam_catalog.get_catalog_package_id(url)

Retrieve package id from full url from data.amsterdam.nl, for example: catalogus/api/3/action/package_show?id=c1f04a62-8b69-4775-ad83-ce2647a076ef

Args:
url: full data.amsterdam.nl url of the desired dataset, for example: https://data.amsterdam.nl/#?dte=catalogus%2Fapi%2F3%2Faction%2Fpackage_show%3Fid%3D5d84c216-b826-4406-8297-292678dee13c
Result:
Unique id number of package.
extract.download_from_data_amsterdam_catalog.parser()

Parser function to run arguments from commandline and to add description to sphinx.