Questions tagged [opendata]

Open Data generally refers to machine readable data which is made freely available (or available under licence) for public consumption. If your question is not closely related to programming, please consider to ask your question on https://opendata.stackexchange.com.

Open Data refers to data which has been made available for public consumption. In general open data is made available by governments or government controlled agencies, often referred to as Open Government Data (OGD), but in reality it can refer to any data made available by any agency, be they a corporate or government entity.

Definitions of Open Data in Literature

[1] Machine readable data made available by governments or government controlled agencies under open licence conditions that allow for further commercial or non commercial use of the data (Janssen and Darbishire, 2012).

[2] Data and information produced or commissioned by the government or government controlled agencies (Lee, 2012, p.1).

Open Data Formats

Open Data can be made available in any formats, but the most popular formats include: CSV, XML, XLS, XLSX, JSON, RDF, RSS, PDF.

Example Open Data Repositories

References

[1] Janssen, K and Darbishire, H (2012) Using open data: is it really empowering? In: W3C (World Wide Web Consortium), Using Open Data: policy modeling, citizen empowerment, data journalism. Brussels, Belgium 19-20 June 2012.

[2] Lee, D (2012) Open Data Overview [Online]. Available: http://www.iia.ie/resources/download/581/ [Accessed 19 May 2012]

137 questions
41
votes
4 answers

Free Large datasets to experiment with Hadoop

Do you know any large dataset to experiment with Hadoop which is free/low cost? Any pointers/links related are appreciated. Preference: At least one GB of data. Production log data of webserver. Few of them which I found so far: Wikipedia…
Sundar
  • 1,204
  • 1
  • 14
  • 17
12
votes
3 answers

API to Database?

Please presume that I do not know anything about any of the things I will be mentioning because I really do not. Most OpenData sites have the possibility of exporting the presented file either in for example .csv or .json formats (Example). They…
yannick
  • 185
  • 1
  • 1
  • 7
12
votes
2 answers

Downloading GeoJSON boundaries using SPARQL from publicly available data

I'm interested in downloading some boundary files from statistics.gov.scot, which is an official statistical repository for sharing statistical data that utilises SPARQL queries. Background Statistics.gov.scot provides access to GeoJSON boundaries…
Konrad
  • 17,740
  • 16
  • 106
  • 167
12
votes
4 answers

How to get the historical weather for any city with BigQuery?

BigQuery has NOAA's gsod data loaded as a public dataset - starting in 1929: https://www.reddit.com/r/bigquery/comments/2ts9wo/noaa_gsod_weather_data_loaded_into_bigquery/ How can I retrieve the historical data for any city?
Felipe Hoffa
  • 54,922
  • 16
  • 151
  • 325
6
votes
1 answer

openrefine flag changed rows

I'm using openrefine to cleanup an excel data set. I have about 70 operations and I've been cutting and pasting on different data sets. I maintain a record id and export to a new excel sheet. Then I reload the sheet using the record id. It works…
Sonicthoughts
  • 548
  • 1
  • 4
  • 16
6
votes
1 answer

How to add a new page in CKAN?

I'm using CKAN as my open data portal. It's written in Python using Pylons framework. It uses Postgres as the database. I just want to add a new page similar to about page to display the terms and conditions to the users. About page comes built in.…
Techie
  • 44,706
  • 42
  • 157
  • 243
4
votes
4 answers

How to map subjective data in the semantic web?

I've been looking at the freebase project for storing data. It seems to be a great place to store concrete, objective data like names, locations and dates. Is it a good place to store subjective data like opinions or ratings? Is there…
Kevin Williams
  • 2,588
  • 3
  • 24
  • 25
4
votes
3 answers

How to get rdf file from sparql endpoint

I am new in opendata and need some help. Wikipedia have their sparql endpoint in this url: http://dbpedia.org/sparql Now I need to write webservice to get some rdf file from dbpedia. What should I send to this endpoint to get rdf file ?
hudi
  • 15,555
  • 47
  • 142
  • 246
4
votes
1 answer

Which data format to publish data from Vantage Pro 2 weather station?

I'm looking for a standard data format which can be used to publish weather data collected by a Davis Instrument Vantage Pro 2 weather station. This is in order to comply to openData movement. Do you have any JSON/XML/other standard data format…
Édouard Lopez
  • 40,270
  • 28
  • 126
  • 178
4
votes
2 answers

Good online resource for common databases / open databases?

I am trying to find the best area to download SQL tables of common subjects like states, countries, zip codes. Maybe even public/government information like FDA or FCC information. Does such a resource exist and is it mature?
4
votes
1 answer

real time traffic api

Is there an API that lets me know of real-time traffic in an area/city/location? I don't have any location in mind, but it is OK to have access to an API that restricts the access of traffic to a particular location. Thanks Abhishek S
London guy
  • 27,522
  • 44
  • 121
  • 179
3
votes
1 answer

How can I get currency exchange rates in Snowflake? (Historic and updated daily)

I see the Snowflake Marketplace has multiple providers of Exchange Rates. Is there one free and updated daily?
Felipe Hoffa
  • 54,922
  • 16
  • 151
  • 325
3
votes
2 answers

Inexplicable formatting magic on API Endpoint

I'm writing a wrapper for the Deutsche Bahn's Fahrplan OpenData API. However, I cannot seem to produce the same result as a simple curl request as follows: >>>import requests >>>header = {'Authorization': 'Bearer…
deepbrook
  • 2,523
  • 4
  • 28
  • 49
3
votes
2 answers

Converting a large XML file to Excel

I want to convert the following XML file (WARNING: it's 40MB), http://www.rvo.nl/sites/default/files/open_data/dop_projecten.xml, to a more readable format. Any hints on how to do this? I have been trying to convert it to Excel, but it keeps…
Sem
  • 31
  • 1
  • 1
  • 2
3
votes
1 answer

Socrata/SODA API call via JQuery not returning the entire dataset

I am making a GET call to the following table: https://data.cityofnewyork.us/Social-Services/Halloween-Noise-Lat-Long/25yv-wyir It has 715 entries. When I make the GET call however, it only returns 463 entries. var map =…
Matt Vaccaro
  • 318
  • 3
  • 9
1
2 3
9 10