ex01-Read SST NetCDF data, subsample and save.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "\n",
       ".rendered_html {\n",
       "    font-family: \"proxima-nova\", helvetica;\n",
       "    font-size: 130%;\n",
       "    line-height: 1.5;\n",
       "}\n",
       "\n",
       ".rendered_html h1 {\n",
       "    margin: 0.25em 0em 0.5em;\n",
       "    color: #015C9C;\n",
       "    text-align: center;\n",
       "    line-height: 1.2; \n",
       "    page-break-before: always;\n",
       "}\n",
       "\n",
       ".rendered_html h2 {\n",
       "    margin: 1.1em 0em 0.5em;\n",
       "    color: #26465D;\n",
       "    line-height: 1.2;\n",
       "}\n",
       "\n",
       ".rendered_html h3 {\n",
       "    margin: 1.1em 0em 0.5em;\n",
       "    color: #002845;\n",
       "    line-height: 1.2;\n",
       "}\n",
       "\n",
       ".rendered_html li {\n",
       "    line-height: 1.5; \n",
       "}\n",
       "\n",
       "/*.prompt {\n",
       "    font-size: 120%; \n",
       "}*/\n",
       "\n",
       ".CodeMirror-lines {\n",
       "    font-size: 110%; \n",
       "}\n",
       "\n",
       "/*.output_area {\n",
       "    font-size: 120%; \n",
       "}*/\n",
       "\n",
       "/*#notebook {\n",
       "    background-image: url('files/images/witewall_3.png');\n",
       "}*/\n",
       "\n",
       "h1.bigtitle {\n",
       "    margin: 4cm 1cm 4cm 1cm;\n",
       "    font-size: 300%;\n",
       "}\n",
       "\n",
       "h3.point {\n",
       "    font-size: 200%;\n",
       "    text-align: center;\n",
       "    margin: 2em 0em 2em 0em;\n",
       "    #26465D\n",
       "}\n",
       "\n",
       ".logo {\n",
       "    margin: 20px 0 20px 0;\n",
       "}\n",
       "\n",
       "a.anchor-link {\n",
       "    display: none;\n",
       "}\n",
       "\n",
       "h1.title { \n",
       "    font-size: 250%;\n",
       "}\n",
       "\n",
       "</style>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%load_ext load_style\n",
    "%load_style talk.css"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Read SST NetCDF data, Subsample and Save\n",
    "\n",
    "This notebook carries out some basic operations:\n",
    "* Open a data file\n",
    "* Check variables\n",
    "* Indexing to subsampe a variable\n",
    "* Save data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Load basic libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np  \n",
    "from netCDF4 import Dataset # http://unidata.github.io/netcdf4-python/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Set input NetCDF file info\n",
    "\n",
    "You can download data using wget or aria2c under Linux. In this case, the data is already downloaded and put into the folder of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#!wget ftp://ftp.cdc.noaa.gov/Datasets/ncep.reanalysis.derived/surface_gauss/skt.sfc.mon.mean.nc\n",
    "ncfile = 'data\\skt.mon.mean.nc'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Extract variables\n",
    "\n",
    "Open a NetCDF file and print the file handler, then you can find the information of its variales from last several lines.\n",
    "\n",
    "Just like:\n",
    "* platform: Model\n",
    "* Conventions: COARDS\n",
    "* dimensions(sizes): lon(192), lat(94), time(687)\n",
    "* variables(dimensions): float32 **lat(lat), float32 lon(lon), float64 time(time), float32 skt(time,lat,lon)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<type 'netCDF4._netCDF4.Dataset'>\n",
      "root group (NETCDF3_CLASSIC data model, file format NETCDF3):\n",
      "    title: 4x daily NMC reanalysis\n",
      "    history: Tue Jul  6 00:05:45 1999: ncrcat skt.mon.mean.nc /Datasets/ncep.reanalysis.derived/surface_gauss/skt.mon.mean.nc /dm/dmwork/nmc.rean.ingest/combinedMMs/skt.mon.mean.nc\n",
      "renamevars Fri Dec 18 12:16:41 1998 from airsst.mon.mean.nc\n",
      "/home/hoop/crdc/cpreanjuke2farm/cpreanjuke2farm Mon Oct 23 21:04:20 1995 from air.sfc.gauss.85.nc\n",
      "created 95/03/13 by Hoop (netCDF2.3)\n",
      "    description: Data is from NMC initialized reanalysis\n",
      "(4x/day).  It consists of T42  variables interpolated to\n",
      "pressure surfaces from model (sigma) surfaces.\n",
      "    platform: Model\n",
      "    Conventions: COARDS\n",
      "    dimensions(sizes): lon(192), lat(94), time(687)\n",
      "    variables(dimensions): float32 \u001b[4mlat\u001b[0m(lat), float32 \u001b[4mlon\u001b[0m(lon), float64 \u001b[4mtime\u001b[0m(time), float32 \u001b[4mskt\u001b[0m(time,lat,lon)\n",
      "    groups: \n",
      "\n"
     ]
    }
   ],
   "source": [
    "fh     = Dataset(ncfile, mode='r') # file handle, open in read only mode\n",
    "print(fh)\n",
    "fh.close() # close the file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "fh     = Dataset(ncfile, mode='r') # file handle, open in read only mode\n",
    "lon    = fh.variables['lon'][:]\n",
    "lat    = fh.variables['lat'][:]\n",
    "nctime = fh.variables['time'][:]\n",
    "t_unit = fh.variables['time'].units\n",
    "skt    = fh.variables['skt'][:]\n",
    "\n",
    "try :\n",
    "    t_cal = fh.variables['time'].calendar\n",
    "except AttributeError : # Attribute doesn't exist\n",
    "    t_cal = u\"gregorian\" # or standard\n",
    "\n",
    "fh.close() # close the file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Access the first and the last value of latitude"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "88.542"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lat[0] # Caution! Python’s indexing starts with zero"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-88.542"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lat[-1] # gives the last value of the vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Select a subregion\n",
    "\n",
    "* Lat: -50 ~ -90\n",
    "* Lon: 0 ~ 360"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lat_so = lat[-21:-1]\n",
    "lon_so = lon\n",
    "skt_so = skt[:,-21:-1,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Save subregion data\n",
    "\n",
    "save subregion data (several arrays) into a single file in uncompressed .npz format using np.savez."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.savez('data/skt.so.mon.mean.npz', skt_so=skt_so, lat_so=lat_so, lon_so=lon_so)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Surely, you can load these data back."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['skt_so', 'lat_so', 'lon_so']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "npzfile = np.load('data/skt.so.mon.mean.npz')\n",
    "npzfile.files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## References\n",
    "\n",
    "http://unidata.github.io/netcdf4-python/\n",
    "\n",
    "John D. Hunter. Matplotlib: A 2D Graphics Environment, Computing in Science & Engineering, 9, 90-95 (2007), DOI:10.1109/MCSE.2007.55\n",
    "\n",
    "Stéfan van der Walt, S. Chris Colbert and Gaël Varoquaux. The NumPy Array: A Structure for Efficient Numerical Computation, Computing in Science & Engineering, 13, 22-30 (2011), DOI:10.1109/MCSE.2011.37\n",
    "\n",
    "Kalnay et al.,The NCEP/NCAR 40-year reanalysis project, Bull. Amer. Meteor. Soc., 77, 437-470, 1996."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}