Knowledge Discovery from Sensor Data
(SensorKDD-2009)
LINKS
Challenge
Problem Description
For this inaugural SensorKDD Challenge, held in conjunction with the 3rd Workshop on Knowledge Discovery from Sensor Data (SensorKDD-2009), we pose a general and open-ended problem related to the workshop theme Climate Change, Energy Assurance, and Infrastructural Impacts. Specifically, we focus on the first component and provide a large climate dataset consisting of daily global air temperature and precipitation measurements for a 50-year period (1950-1999).
Your task is to detect significant changes in this data. Note that “change” in this context may be an abrupt change (i.e., an anomaly or outlier), a gradual change (i.e., a shift in distribution), an extreme event (i.e. heat wave, severe rainfall, drought) or something different altogether. We intentionally omit a precise definition here to invite creative problem definitions and solutions.
Data
The data consists of daily global air temperature and precipitation measurements, aggregated from heterogeneous sensors, for the period from 1950-1999 (see [1] for details). This data is also publicly available for download, for example from [2] or [3] (registration may be required).
Two options are available for downloading the data.
- NetCDF – Earth Science data is often stored in netCDF files, a highly compressed data format specifically designed for high-dimensional data. We invite anyone interested in studying climate to get familiar with this format by visiting the following page:
http://www.unidata.ucar.edu/software/netcdf/
There you will find a detailed description of the file format as well as a number of tools and utilities to manipulate netCDF files. We also provide a code sample for reading the data with the open-source statistical software package R (www.r-project.org) using the ncdf package; similar utilities are available for Matlab, e.g. NetCDF Reader or MEXNC.
Download: Temperature [628MB] – Precipitation [628MB]
- Raw (CSV) – To encourage broader participation, we also provide the data in a raw (comma-separated) format. As part of the processing we performed the following unit conversions:
- Temperature was converted from Kelvin (K) to Degrees Celsius (ºC)
- Precipitation was converted from kilograms per square meter per second (kg/m2/s)
to millimeters per day (mm/day)
Each line of data contains the time series for one grid location in the following format:
<latitude>,<longitude>,<day1>,<day2>,...,<dayN>
Download: Temperature [974MB] – Precipitation [692MB] – compressed with ZIP.
Submissions
We invite the submission of manuscripts addressing this challenge task following the format of the SensorKDD-2009 Workshop (maximum 9 pages, ACM standard format). Please include your definition of “change”. Entries will be evaluated subjectively based on their methodology and validation strategy.
- Submission Date:
April 20May 1, 2009 - Author Notification: May 21, 2009
- Submission of Camera-Ready Papers: May 25, 2009
- Full-Day Workshop at ACM SIGKDD Conference, Paris, France: June 28, 2009
Award
Best Challenge Entry Award.
Contact Information
For questions about the challenge or to inquire about additional climate datasets, please contact:
Karsten Steinhaeuser
steinhaeuskj@ornl.gov
Phone: +1-574-576-7597
Challenge Organizers
Karsten Steinhaeuser, University of Notre Dame, IN and Oak Ridge National Laboratory, TN, USA.
Auroop R. Ganguly, Oak Ridge National Laboratory, TN, USA.
Nitesh V. Chawla, University of Notre Dame, IN, USA.
References
[1] R. Kistler, et al. “The NCEP/NCAR 50-year reanalysis.” Bulletin of the American Meteorological Society, 82(2), 247-267 (2001).
[2] http://www.cdc.noaa.gov/data/reanalysis/reanalysis.shtml
[3] http://dss.ucar.edu/datasets/ds090.0/
Thanks to Our
Sponsors!
COPYRIGHT February 2009 | Contact: Olufemi A. Omitaomu