HOW TO Access Data in Data.Colorado.Gov

Introduction

This article describes how to access data in Colorado Information Marketplace (CIM), the State of Colorado’s official open data portal and the data resource for Go Code Colorado.

More on CIM

The Colorado Information Marketplace (CIM) is a platform provided by Socrata and licensed by Colorado’s Statewide Internet Portal Authority (SIPA) on behalf of the Governor’s Office of Information Technology and the Chief Data Officer who sponsors and administers CIM. CIM is provided by Socrata as software-as-a-service (SaaS). CIM can be found here: https://data.colorado.gov/.

Access Methods

There are three primary methods to access data in CIM.

  • Through CIM’s web application
  • Through the Socrata Open Data API (SODA)
  • In Microsoft Excel using an OData link

Access Through the Web Application

CIM’s web application enables you to search and browse datasets and views. You can browse datasets by view type (e.g., “datasets”, “maps”), by category (e.g., “demographics”), or by topic (e.g., “Go Code Colorado”). CIM contains multiple public datasets, but those datasets published specifically for Go Code Colorado will be “tagged” with both the “BIC” and “Go Code Colorado” topic keywords.

When you’ve found a dataset you’re interested in, you can use CIM web application to view the dataset itself. You can browse datasets in several views:

  • as a table
  • as a rich list
  • as single records

CIM’s web application provides several other features for exploring datasets:

  • Filter
  • Sort
  • Visualize (charts, map, etc.)
  • View metadata

You can export the datasets in several formats:

  • CSV
  • JSON
  • PDF
  • RDF
  • RSS
  • XLS
  • XLSX
  • XML

You could then import the datasets into your own data architecture/platform.

Access through the Socrata Open Data API (SODA)

The Socrata Open Data API allows you to programatically access open data resources from CIM. Click the link below and for an example on how queries are structured in SODA.

http://soda.demo.socrata.com/resource/earthquakes.json?$where=magnitude > 5.5

The SODA API documentation has been broken down into three user categories: App Developers; Data Publishers; Data Consumers.

APP Developers: Locating Open Data and APIs

Once you’re on CIM, scroll down to the data catalog and use the search box and browse filters to find datasets that interest you. Every dataset is accessible via the SODA API.

Finding your API endpoint

Every Socrata open dataset has a built-in SODA API. But how you find the API endpoint can vary a bit.

If you’re on a Socrata dataset, identifiable by the colorful buttons at the upper right, click on Export and then API and you’ll find the API endpoint under API Access Endpoint. Copy that and save it for later.

Building simple filters and queries

Filtering data via a SODA API is fairly straightforward. There are two primary mechanisms you can use to filter data: Simple Filters and SoQL Queries

Simple Filters

Filtering data is very straightforward. SODA APIs are self-describing – the schema and contents of the dataset itself determines how you can query it. Any field within the data can be used as a filter, simply by appending it to the API endpoint as a GET parameter.

Additional filters can be added, and the filters will be ANDed together. Read the detailed documentation on Filtering Datasets for more info.

SoQL Queries

The ”Socrata Query Language” (SoQL) is a simple, SQL-like query language specifically designed for making it easy to work with data on the web. The language is both powerful and easy to learn, and everything works via GET parameters. Many different functions are available via SoQL. Read the detailed documentation on SoQL Queries for more info.

Paging

For performance, SODA APIs are paged, and return a maximum of 1000 records per page. So, to request subsequent pages, you’ll need to use the $limit and $offset parameters to request more data. The $limit parameter chooses how many records to return per page, and $offset tells the API on what record to start returning data. Read the detailed documentation on Paging for more info.

Throttling and Applications Tokens

You can make a certain number of requests without an application token, but they come from a shared pool and you’re eventually going to get cut off.

If you want more requests, register for an application token and your application will be granted up to 1000 requests per rolling hour period. If you need even more than that, special exceptions are made by request. Use the Help! tab on the right of this page to file a trouble ticket.

Read the detailed documentation on Paging for more info.

SODA Features Summary

SODA provides an open, standards-based, RESTful application programming interface to access datasets in the CIM.
Every hosted dataset is readily and uniformly accessible.  You can read the data, but also perform more advanced operations such as searching and filtering. Combine datasets with other Web services to create new mashups and applications. Socrata Open Data API (SODA) specifications were released to the Open Data community under a Creative Commons license

SODA is supported by a developer site http://beta.dev.socrata.com/.

Access in Microsoft Excel through an OData Link

CIM provides an OData link for every hosted dataset. With OData, you can create a spreadsheet in Microsoft Excel to analyze or visualize a dataset. By using an OData link, the data will be refreshable. You can use the OData link with Microsoft Excel 2010, Microsoft Excel 2013 or Microsoft Power Query.

Key Events

Wed01

Kickoff 2017

February 01, 2017 @ 6:00 PM - 9:00 PM
Fri07

Challenge Weekend

April 07, 2017 @ 6:00 PM - 8:00 PM
Fri - Sun21 - 23

Mentor Weekend

April 21 @ 6:00 PM - April 23 @ 8:00 PM
Wed24

Final Event

MAY 24 @ 6:00 PM - 10:00 PM

Don't miss an update.

Email Sign Up