Showing posts with label NetCDF. Show all posts
Showing posts with label NetCDF. Show all posts

Wednesday, May 4, 2011

Installing NCView 2.0beta4 on Ubuntu 10.10

This post explains how I installed David Pierce's Ncview utility (version 2.0beta4) on Ubuntu 10.10. NB: Power users should scroll to the bottom for a summary of commands.

What is Ncview?

According to its author David Pierce of the Scripps Institute of Oceanography, Ncview is a visual browser for NetCDF files.

What is a NetCDF file?

A NetCDF file is a self-describing, machine-independent array file commonly used by the climate science community to store climate data. If you're interested in Ncview, you probably know about NetCDF files.

Why do we need Ncview?

NetCDF files can have dozens of dimensions, hundreds of variables, and millions of data points. Ncview can help a user get a quick picture of what the file holds, how to read it, and where to find the parts of the data in which he or she is interested.

Installing Ncview on Ubuntu 10.10

This post shows how to install Ncview 2.0 beta4 on Ubuntu (10.10) specifically. For instructions on how to install Ncview generally on Linux, see David Pierce's Ncview site.

Get Ncview source files

First we need to download the Ncview source files. Navigate to ftp://cirrus.ucsd.edu/pub/ncview/ and select the most recent package. As of this writing, the most recent package is ncview-2.0beta4.tar.gz, updated on 3/4/2010. Download this tar.gz file and open it with Ubuntu's Archive Manager.

Next, extract the files to your home directory. From Archive Manager, I just click on the "Extract" button, select my home directory, and click on "Extract." This creates the folder "ncview-2.0beta4" in my home directory.

Browse the readme and the installation instructions

In the newly created folder, take a look at the files README and INSTALL. These are general instructions that provide good background. This installation tutorial borrows heavily from these two files.

Installation overview

In a perfect world, i.e. one where all dependencies were met and all files on your local machine were exactly where the installation scripts expected them to be, the installation would follow three steps, entered from the terminal from within the ncview-2.0beta4 directory:
  1. $ ./configure
  2. $ make
  3. $ make install
As such, this is a good starting point, because it just might work. But on my machine, I get the following errors when I try the first step:
andy@andy-laptop:~$ cd ncview-2.0beta4/
andy@andy-laptop:~/ncview-2.0beta4$ ./configure
checking for nc-config... yes
Netcdf library version: netCDF 4.1.1
Netcdf library has version 4 interface present: yes
Netcdf library was compiled with C compiler: gcc
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for library containing strerror... none required
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking for X... no
------------------------------------------------------------------------------------
Error, the X libraries and development headers must be installed for ncview to work!
------------------------------------------------------------------------------------
More information: You are trying to compile ncview. The machine you are compiling on
probably already has the X windows *runtime* libraries installed, but to *compile*
a program you need more than just the runtime libraries.  You need what are usually
called the 'development headers', named because they are used when developing or
compiling X windows programs.  The best advice I can give you is to use your package
manager to look for a package whose name is something along the lines of x11-devel,
or xorg-x11-proto-devel, or something along those lines that indicates the package
contains the X windows development headers.  Install that package first, then try to
remake ncview.  
Note: If that still fails, even after you've installed the X windows *development*
headers, then you may be on a machine where the automatic configuration system is not
set up quite as it probably should be.  In that case, you might have to specify the
location of the X libraries and X headers manually.  For example, on some machines
the following will work:
   ./configure  --x-libraries=/usr/lib64 --x-includes=/usr/include/X11
                              ^^^^^^^^^^              ^^^^^^^^^^^^^^^^ these are what
you want to set to reflect the location of files such as libX11.so and X.h on your
particular system.

The script output tells us that it performed about a dozen checks which passed, then reached one it failed (the line "checking for X. . . no"). Then we get the error "the X libraries and development headers must be installed for ncview to work!" Lastly, we get a very thorough message from Ncview author David Pierce advising us how to meet the "X" dependency. As he notes, we need the X windows *development* headers.

Install xorg-dev dependencies

We need the X Windows development headers. The brute force way to do this on Ubuntu is to install the package "xorg-dev," which includes many (all?) development header files related to the X11 windows system. Here's how I did it:
andy@andy-laptop:~$ sudo apt-get install xorg-dev
[sudo] password for andy: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libdmx-dev libdmx1 libexpat1-dev libfontconfig1-dev libfontenc-dev libfreetype6-dev libfs-dev libice-dev libpciaccess-dev
  libpixman-1-dev libpthread-stubs0 libpthread-stubs0-dev libsm-dev libx11-dev libxau-dev libxaw7-dev libxcb1-dev libxcomposite-dev
  libxcursor-dev libxdamage-dev libxdmcp-dev libxext-dev libxfixes-dev libxfont-dev libxft-dev libxi-dev libxinerama-dev libxkbfile-dev
  libxmu-dev libxmu-headers libxmuu-dev libxpm-dev libxrandr-dev libxrender-dev libxres-dev libxss-dev libxt-dev libxtst-dev libxv-dev
  libxvmc-dev libxxf86dga-dev libxxf86vm-dev x11proto-bigreqs-dev x11proto-composite-dev x11proto-core-dev x11proto-damage-dev
  x11proto-dmx-dev x11proto-dri2-dev x11proto-fixes-dev x11proto-fonts-dev x11proto-gl-dev x11proto-input-dev x11proto-kb-dev
  x11proto-randr-dev x11proto-record-dev x11proto-render-dev x11proto-resource-dev x11proto-scrnsaver-dev x11proto-video-dev
  x11proto-xcmisc-dev x11proto-xext-dev x11proto-xf86bigfont-dev x11proto-xf86dga-dev x11proto-xf86dri-dev x11proto-xf86vidmode-dev
  x11proto-xinerama-dev xserver-xorg-dev xtrans-dev
The following NEW packages will be installed:
  libdmx-dev libdmx1 libexpat1-dev libfontconfig1-dev libfontenc-dev libfreetype6-dev libfs-dev libice-dev libpciaccess-dev
  libpixman-1-dev libpthread-stubs0 libpthread-stubs0-dev libsm-dev libx11-dev libxau-dev libxaw7-dev libxcb1-dev libxcomposite-dev
  libxcursor-dev libxdamage-dev libxdmcp-dev libxext-dev libxfixes-dev libxfont-dev libxft-dev libxi-dev libxinerama-dev libxkbfile-dev
  libxmu-dev libxmu-headers libxmuu-dev libxpm-dev libxrandr-dev libxrender-dev libxres-dev libxss-dev libxt-dev libxtst-dev libxv-dev
  libxvmc-dev libxxf86dga-dev libxxf86vm-dev x11proto-bigreqs-dev x11proto-composite-dev x11proto-core-dev x11proto-damage-dev
  x11proto-dmx-dev x11proto-dri2-dev x11proto-fixes-dev x11proto-fonts-dev x11proto-gl-dev x11proto-input-dev x11proto-kb-dev
  x11proto-randr-dev x11proto-record-dev x11proto-render-dev x11proto-resource-dev x11proto-scrnsaver-dev x11proto-video-dev
  x11proto-xcmisc-dev x11proto-xext-dev x11proto-xf86bigfont-dev x11proto-xf86dga-dev x11proto-xf86dri-dev x11proto-xf86vidmode-dev
  x11proto-xinerama-dev xorg-dev xserver-xorg-dev xtrans-dev
0 upgraded, 69 newly installed, 0 to remove and 0 not upgraded.
Need to get 8,853kB of archives.
After this operation, 26.6MB of additional disk space will be used.
Do you want to continue [Y/n]? Y
This will download and install 27 megabytes worth of files, which we hope includes whatever it is that Ncview needs. Of course we could have tried to identify which handful of files ncview specifically needs and installed just those, but this is easier and faster.

Try ./configure again

After installing the xorg-dev packages, I run the ./configure command again and get the following (truncated) result:
andy@andy-laptop:~/ncview-2.0beta4$ ./configure
checking for nc-config... yes
Netcdf library version: netCDF 4.1.1
[...]
checking for X... libraries , headers 
checking for gethostbyname... yes
[...]
checking udunits2.h usability... no
checking udunits2.h presence... no
checking for udunits2.h... no
[...]
************************************************************************
Note: udunits2 support is NOT enabled, because I could not find the 
location of the udunits2 include file 'udunits2.h' or library file
'libudunits2.a'.  Ncview uses the udunits2 package to format date strings
with units of the form 'days since 1900-01-01'.  If you do not use
these udunits2-standard date formats, then don't worry about the lack
of udunits2 support.  If you DO use udunits2 format date strings, and
you want the udunits2 support, then you must tell me where to find
the udunits2 package by giving arguments to configure, as follows:
  ./configure -with-udunits2_incdir=include_directory -with-udunits2_libdir=library_directory
************************************************************************
checking /usr/local/include/ppm.h usability... no
checking /usr/local/include/ppm.h presence... no
[...]
************************************************************************
Note: the -frames option is NOT enabled, because I could not find the 
location of the PPM include file 'ppm.h' or library file
'libppm.a'.  Ncview uses the ppm package to dump out the frames viewed,
which is an easy way to make an mpeg video of the data if you want.
If you do not want this feature, then don't worry about the lack
of ppm support.  If you DO want this, then you must tell me where to find
the ppm package by giving arguments to configure, as follows:
  ./configure -with-ppm_incdir=include_directory -with-ppm_libdir=library_directory
************************************************************************
checking for a BSD-compatible install... /usr/bin/install -c
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating config.h
config.status: executing depfiles commands
 
----------- Configure Summary ----------
Compiler:
        CC                = gcc
 
UDUNITS:
        UDUNITS2_LIBS     = 
        UDUNITS2_CPPFLAGS = 
        UDUNITS2_LDFLAGS  = 
 
 
NETCDF:
        VERSION          = netCDF 4.1.1
        COMPILER USED    = gcc
        NETCDF_CPPFLAGS  = -I/usr/include
        NETCDF_LDFLAGS   = -L/usr/lib -lnetcdf
        NETCDF_V4        = yes
 
X:
        X_CFLAGS         = 
        X11_LIBS         = -lX11 
        XAW_LIBS         = -lXaw -lXt 
        X_PRE_LIBS       =  -lSM -lICE
        X_LIBS           = 
        X_EXTRA_LIBS     = 
A few things to note:
  1. This time the line "Checking for X ... " succeeded, so we know that installing xorg-dev worked;
  2. It started to fail again when looking for files called "udunits";
  3. Then it started to fail again when it couldn't find PPM files, which we are told are used for making movies of the data.
I want to be able to make movies of the data, so let's tackle the PPM issue.

Install the PPM dependency

The ./configure script output told us it's missing the files ppm.h, among others. Let's figure out what package that belongs to using the program apt-file. I don't have it installed, so I'll install it and then use it to find packages that include the file ppm.h. The following is my truncated commands and output:
andy@andy-laptop:~$ sudo apt-get install apt-file
[sudo] password for andy: 
Reading package lists... Done
[...]
Need to get 508kB of archives.
After this operation, 1,434kB of additional disk space will be used.
Do you want to continue [Y/n]? y
[...]
The system-wide cache is empty. You may want to run 'apt-file update'
as root to update the cache. You can also run 'apt-file update' as
normal user to use a cache in the user's home directory.
And then update the cache as it recommends (this took about 20 minutes with my Comcast Internet connection):
andy@andy-laptop:~$ sudo apt-file update
Downloading complete file http://us.archive.ubuntu.com/ubuntu/dists/maverick/Contents-i386.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 17.4M  100 17.4M    0     0  16574      0  0:18:23  0:18:23 --:--:-- 16731
[...]
Ignoring source without Contents File:
  http://extras.ubuntu.com/ubuntu/dists/maverick/Contents-i386.gz
Ignoring source without Contents File:
  http://dl.google.com/linux/talkplugin/deb/dists/stable/Contents-i386.gz
Now we want to use this program to find the packages with the file ppm.h:
andy@andy-laptop:~$ apt-file search ppm.h
gamgi-doc: /usr/share/doc/gamgi-doc/doc/formats/introduction/ppm.html
grass-doc: /usr/share/doc/grass-doc/html/r.out.ppm.html
gromacs-dev: /usr/include/gromacs/pppm.h
libnetpbm10-dev: /usr/include/libppm.h
libnetpbm10-dev: /usr/include/ppm.h
libnetpbm9-dev: /usr/include/libppm.h
libnetpbm9-dev: /usr/include/ppm.h
libtachyon-dev: /usr/include/tachyon/ppm.h
libtk-img-doc: /usr/share/doc/libtk-img-doc/html/img-ppm.html
tau-examples: /usr/share/doc/tau-examples/examples/opari/c++/ppm.h
tau-examples: /usr/share/doc/tau-examples/examples/openmp/c++/ppm.h
tau-examples: /usr/share/doc/tau-examples/examples/openmp/c/ppm.h
tbb-examples: /usr/share/doc/tbb-examples/examples/parallel_for/tachyon/src/ppm.h
I'm guessing it's the package libnetpbm that we want, so we'll install it:
andy@andy-laptop:~$ sudo apt-get install libnetpbm10-dev
[sudo] password for andy: 
[...]
Setting up libnetpbm10-dev (2:10.0-12.2) ...
Now we can try ./configure again (still heavily truncated output):
andy@andy-laptop:~$ cd ncview-2.0beta4/
andy@andy-laptop:~/ncview-2.0beta4$ ./configure
checking for nc-config... yes
Netcdf library version: netCDF 4.1.1
[...]
checking udunits2.h usability... no
[...]
checking /usr/local/include/ppm.h usability... no
checking /usr/local/include/ppm.h presence... no
checking for /usr/local/include/ppm.h... no
checking /usr/include/ppm.h usability... yes
checking /usr/include/ppm.h presence... yes
checking for /usr/include/ppm.h... yes
checking for ppm_writeppm in -lppm... no
checking for /usr/local/lib/libppm.so... no
checking for /usr/lib/libppm.so... no
checking for /lib/libppm.so... no
checking for /home/andy/lib/libppm.so... no
checking for ppm_writeppm in -lnetpbm... yes
checking for a BSD-compatible install... /usr/bin/install -c
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
 
----------- Configure Summary ----------
Compiler:
        CC                = gcc
 
UDUNITS:
        UDUNITS2_LIBS     = 
        UDUNITS2_CPPFLAGS = 
        UDUNITS2_LDFLAGS  = 
 
 
NETCDF:
        VERSION          = netCDF 4.1.1
        COMPILER USED    = gcc
        NETCDF_CPPFLAGS  = -I/usr/include
        NETCDF_LDFLAGS   = -L/usr/lib -lnetcdf
        NETCDF_V4        = yes
 
X:
        X_CFLAGS         = 
        X11_LIBS         = -lX11 
        XAW_LIBS         = -lXaw -lXt 
        X_PRE_LIBS       =  -lSM -lICE
        X_LIBS           = 
        X_EXTRA_LIBS     = 
This time it found the PPM files it wanted, and we're left with just one remaining error: the UDunits thing. Let's tackle that now.

Install UDUnits dependency

Let's use apt-file search again to find which packages we should install to get our missing files. From the output above, we see that we need the file "udunits2.h", among others. Let's search for it:
andy@andy-laptop:~$ apt-file search udunits2.h
libudunits2-0: /usr/share/doc/libudunits2-0/udunits2.html
libudunits2-dev: /usr/include/udunits2.h
So there's a package called udunits; let's see if there's a broader collection of packages, using aptitude:
andy@andy-laptop:~$ aptitude search udunits
p   libudunits2-0                                                 - Library for handling of units of physical quantities                    
p   libudunits2-dev                                               - Development files for the libunits physical units package               
p   udunits-bin                                                   - Utility for handling units of physical quantities
Let's install all three of these packages:
andy@andy-laptop:~$ sudo apt-get install libudunits2-0 libudunits2-dev udunits-bin

Try ./configure again

Now that we've met the UDUnits and PPM and X Windows dependencies, let's try configuring again:
andy@andy-laptop:~/ncview-2.0beta4$ ./configure
checking for nc-config... yes
Netcdf library version: netCDF 4.1.1
[...]
checking udunits2.h usability... yes
checking udunits2.h presence... yes
[...]
****************************************************************************
Udunits library version 2 support enabled.   
udunits2 dirs: include: .  library: .  libname: udunits2
****************************************************************************
[...] 
----------- Configure Summary ----------
Compiler:
        CC                = gcc
 
UDUNITS:
        UDUNITS2_LIBS     = -ludunits2
        UDUNITS2_CPPFLAGS = -I.
        UDUNITS2_LDFLAGS  = -L. -ludunits2 -lexpat -L. -ludunits2
 
 
NETCDF:
        VERSION          = netCDF 4.1.1
        COMPILER USED    = gcc
        NETCDF_CPPFLAGS  = -I/usr/include
        NETCDF_LDFLAGS   = -L/usr/lib -lnetcdf
        NETCDF_V4        = yes
 
X:
        X_CFLAGS         = 
        X11_LIBS         = -lX11 
        XAW_LIBS         = -lXaw -lXt 
        X_PRE_LIBS       =  -lSM -lICE
        X_LIBS           = 
        X_EXTRA_LIBS     = 
Success. The configure summary shows that it is aware of Compiler, UDUNITS, NETCDF, and X.

Run the make command

The next step is to run the make command.
andy@andy-laptop:~/ncview-2.0beta4$ make
This runs on my machine with copious warnings, but it appears to complete without any fatal errors. Next run make install:
andy@andy-laptop:~/ncview-2.0beta4$ sudo make install

Test the installation

Try testing the installation by running the program:
andy@andy-laptop:~/ncview-2.0beta4$ ncview
Ncview 2.0beta4 David W. Pierce  3 March 2010
Success. That completes the installation of Ncview. It's now ready to be used to browse NetCDF files.

Summary of commands for power users

  1. Download the Ncview source files: ftp://cirrus.ucsd.edu/pub/ncview/ (select most recent package)
  2. Install X.org development headers:
    andy@andy-laptop:~$ sudo apt-get install xorg-dev
  3. Install LibnetPBM development headers:
    andy@andy-laptop:~$ sudo apt-get install libnetpbm10-dev
  4. Install the UDUnits packages:
    andy@andy-laptop:~$ sudo apt-get install libudunits2-0 libudunits2-dev udunits-bin
    
  5. Run the configure script:
    andy@andy-laptop:~/ncview-2.0beta4$ ./configure
  6. Run make:
    andy@andy-laptop:~/ncview-2.0beta4$ make
    
  7. Run make install:
    andy@andy-laptop:~/ncview-2.0beta4$ sudo make install
    
  8. Test the installation:
    andy@andy-laptop:~/ncview-2.0beta4$ ncview
    Ncview 2.0beta4 David W. Pierce  3 March 2010

Thursday, March 24, 2011

How to install RubyNetCDF on Ubuntu: The Short Version

NB: For a much more detailed explanation of how to install ruby-netcdf, see The Long Version.

Installing RubyNetCDF

I follow the RubyNetCDF installation instructions provided by the program creators. I am using Linux Ubuntu 10.04 (Lucid) with Ruby 1.8.7 on a Dell Latitude E6400. This probably won't work on Windows, except perhaps through Cyg-Win.

Install dependencies

The first steps are to install NArray and to install NetCDF.
  1. Install NArray
    % sudo gem install narray
  2. Now check to make sure it actually is working by jumping into an interactive Ruby session and creating an NArray object:
    $ irb -rubygems
    >> require 'narray'
    => true
    >> a = NArray[2,3,4]
    => NArray.int(3): 
    [ 2, 3, 4 ]
    >> exit
    
  3. Install NetCDF version 3:
    $ sudo apt-get install netcdf-bin libnetcdf-dev
  4. Check that the installation was successful:
    $ ncdump
That completes the first two steps to using RubyNetCDF's dependencies. Now we  download RubyNetCDF and install it.

Install ruby-netcdf as a gem

RubyNetCDF exists as a gem.

sudo gem install ruby-netcdf

but this will fail as follows:

Building native extensions.  This could take a while...
ERROR:  Error installing ruby-netcdf:
 ERROR: Failed to build gem native extension.

/usr/bin/ruby1.8 extconf.rb
extconf.rb:3: uninitialized constant Gem (NameError)

Gem files will remain installed in /usr/lib/ruby/gems/1.8/gems/ruby-netcdf-0.6.5 for inspection.
Results logged to /usr/lib/ruby/gems/1.8/gems/ruby-netcdf-0.6.5/gem_make.out

This failed because the file extconf.rb doesn't have the necessary rubygems call (require 'rubgems'). It might work if you're using Ruby 1.9. Note that it left the files in the directory where gems would go, so we can navigate to that directory and install it manually.

$ cd /usr/lib/ruby/gems/1.8/gems/ruby-netcdf-0.6.5
$ ruby -rubygems extconf.rb --with-narray-include=/usr/lib/ruby/gems/1.8/gems/narray-0.5.9.9

If you get the error extconf.rb:3: uninitialized constant Gem (NameError), then make sure you're including the argument -rubygems above. That's the equivalent of inserting at the top of the file the line
require 'rubygems'

The next step is to run the make commands:
$ sudo make
$ sudo make install

The final step is run a test to see if the installation of ruby-netcdf was successful, but mine got an error:
$ sudo make test
test.rb:3:in `require': no such file to load -- narray (LoadError)
 from test.rb:3
make: *** [test] Error 1

For this line to work with Ruby 1.8.7, it needs to have RubyGems initialized. Open the test.rb file, and add the line manually to the top. This is what the first few lines of my file look like after I insert the new line 3:
##require 'numru/netcdf' 
## // to test before make install -->
require 'rubygems'
require 'narray'
require '../netcdfraw'  
require '../lib/netcdf'

Now try running the test again:
$ sudo make test
.
.
.
test did not fail :-p (please ignore the warnings)
The message at the end, along with the friendly emoticon, tell us that the test succeeded. We have completed the installation of RubyNetCDF.

Confirm installation succeeded

We can confirm that everything is working by opening a Ruby session (but not in the protected gem directory)...
$ cd ~
$ irb
...and interacting with NetCDF.
irb(main):001:0> require 'numru/netcdf'
=> true
irb(main):002:0> file = NumRu::NetCDF.create("test.nc")
=> NetCDF:test.nc
irb(main):003:0> file.close
=> nil
irb(main):004:0> exit
That confirms that everything is working.
~Fin~

    How to install RubyNetCDF on Ubuntu: The Long Version

    What is NetCDF?

    The best introduction to NetCDF, originally developed for the Earth science community, comes from UCAR's intro page. NetCDF stands for Network Common Data Format. According to its home site, NetCDF is "a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data." It seems to be popular among research organizations dealing with large climate data. For more details information, see the NetCDF page at unidata.ucar.edu.

    Basically, NetCDF is a file format, like .xlsx for Excel files or .csv for commas separated value files. NetCDF files have the file extension .nc. NetCDF is also a set of software libraries. Once these libraries are on your computer, you can open the NetCDF files.

    We know how to open Excel files; how do we open NetCDF files?

    How do we open NetCDF files?

    There is no desktop application like Excel to open NetCDF files. Instead, they must be read directly with another program. We'd like to do it with Ruby. For example, in a Ruby environment, one can use the CSV class to open, read, modify, and write to CSV files. So we need a class for opening, reading, modifying, and writing to NetCDF files. That's what RubyNetCDF is for.

    What is RubyNetCDF?

    According to its creators, "RubyNetCDF is the Ruby interface to the NetCDF library built on the NArray library, which is an efficient multi-dimensional numeric array class for Ruby." So it's an interface, built on a library. What's an interface?

    An interface is just the user-facing part of an application that a user interacts with. A command-line is an interface; Microsoft Windows is a graphical user interface (GUI). So RubyNetCDF will be the interface that the user interacts with to engage with a NetCDF file.

    What's a library?
    In Ruby, a library is a piece of additional information that the Ruby language incorporates into itself in order to make more functionality available to the user. Why wasn't it included to begin with? Probably because its use is esoteric and only useful to a small number of users.

    Installing RubyNetCDF

    I follow the RubyNetCDF installation instructions provided by the program creators. I am using Linux Ubuntu 10.04 (Lucid) with Ruby 1.8.7 on a Dell Latitude E6400. This probably won't work on Windows, except perhaps through Cyg-Win.

    Install dependencies

    The first steps are to install Ruby, install NArray, and install NetCDF.
    1. Install Ruby
      1. I assume this is done since you already chose Ruby as the NetCDF interface
      2. Check your version by typing
        $ ruby -v
        at the command prompt. Mine is 1.8.7. NB: Ruby versions prior to 1.9 require explicitly requiring Rubygems before requiring other files. See the following example in Ruby 1.8.7:
        $ irb --simple-prompt
        >> require 'narray'
        LoadError: no such file to load -- narray
         from (irb):1:in `require'
         from (irb):1
         from :0
        >> require 'rubygems'
        => true
        >> require 'narray'
        => true
        >> exit
        
        Note that I had to include 'rubygems' before Ruby knew how to include the file 'narray'. In versions of Ruby later than 1.9, requiring 'rubygems' explicitly is not required.
    2. Install NArray 
      1. At the command prompt type
        $ sudo gem install narray
      2. I got a whole bunch of "No definition for ..." errors. I ignored them.
      3. Check the installation version by typing
        $ gem list narray
        Mine returns version 0.5.9.9. Now check to make sure it actually is working by jumping into an interactive Ruby session and creating an NArray object:
        $ irb --simple-prompt
        >> require 'rubygems' #this line not required for Ruby > 1.9
         => true
        >> require 'narray' 
        => true
        >> a = NArray[2,3,4]
        => NArray.int(3): 
        [ 2, 3, 4 ]
        >> exit
        
    3. Install NetCDF version 3. I'm running Ubuntu 10.04, so I go for the simplest solution possible:
      1. Go to Applications->Ubuntu Software Center
      2. Type "netcdf" (without the quotes) in the search field.
      3. Select "Programs for reading and writing NetCDF files" (netcdf-bin)
      4. Click "Install." Notice that it will automatically install the required dependency "libnetcdf4", which it calls "An interface to scientific data access to large binary data."
      5. Check that the installation was successful typing
        $ ncdump
        at the command line. The last line returned should give you the version number. Mine returns
        netcdf library version "3.6.3" of Dec 22 2009 06:10:17 $
    So that completes the first three necessary steps to using RubyNetCDF. Now we must download RubyNetCDF and install it.

    Download and install ruby-netcdfManually

    We'll do it manually, following the installation instructions.
    To begin, download the tar file. I let my browser put it in the default /home/andy/Downloads folder. From there I double clicked on the file ruby-netcdf-0.6.5.tar.gz, and used the default Ubunutu archive manager extract it. I extracted it to my home folder, /home/andy. Now I have a folder there called ruby-netcdf-0.6.5. Next I just follow the steps in the installation page:

    Navigate to the folder where you extracted the file:
    $ cd ruby-netcdf-0.6.5

    Then, type the following command:
    $ ruby extconf.rb
    This runs the ruby script called extconf.rb.When I run this command, I get the following error output:
    checking for narray.h... no
    ** configure error **  
       Header narray.h or narray_config.h is not found. If you have these files in 
       /narraydir/include, try the following:
    
       % ruby extconf.rb --with-narray-include=/narraydir/include
    
    *** extconf.rb failed ***
    
    The extconf script couldn't find the file narray.h (which is part of the NArray gem we installed earlier), so we need to tell the program where to find the file. Fortunately, it tells us how to do that, but first we need to actually find the file on our system. I did it in Ubuntu
    $ sudo find / -name narray.h
    Both methods gave me the path to the file: /usr/lib/ruby/gems/1.8/gems/narray-0.5.9.9. So now we use that in the command line, appending it to the first part:
    $ ruby extconf.rb --with-narray-include=/usr/lib/ruby/gems/1.8/gems/narray-0.5.9.9
    Now I get a slightly different error:

    $ ruby extconf.rb --with-narray-include=/usr/lib/ruby/gems/1.8/gems/narray-0.5.9.9
    checking for narray.h... yes
    checking for narray_config.h... yes
    checking for netcdf.h... no
        ** configure error **  
           Header netcdf.h or the compiled netcdf library is not found. 
           If you have the library installed under /netcdfdir (that is, netcdf.h is
           in /netcdfdir/include and the library in /netcdfdir/lib/),
           try the following:
    
           % ruby extconf.rb --with-netcdf-dir=/netcdfdir
    
    This time, it found the first file it was looking for (narray.h), and the second file it was looking for (narray_config.h), and then failed on the third file (netcdf.h). So we need to repeat the steps above to find the missing file, and then pass the path to the ruby script at the command line.

    First, find the file:
    $ sudo find / -name netcdf.h
    But this time it doesn't find anything. So what happened? We need to install an additional package, called libnetcdf-dev. The easiest way to do this in Ubuntu is through the package manager. System->Administration->Synaptec Package Manager, and search for "libnetcdf-dev." Install this file. Now try finding the file from the command line:
    $ sudo find / -name netcdf.h
    This time it should find the file. Mine is located at /usr/include/netcdf.h.

    Now that we know the file exists in our system, try running the RubyNetCDF installer again:
    $ ruby extconf.rb --with-narray-include=/usr/lib/ruby/gems/1.8/gems/narray-0.5.9.9
    This time, it should succeed. I get the following output:
    andy@andy-laptop:~/ruby-netcdf-0.6.5$ ruby extconf.rb --with-narray-include=/usr/lib/ruby/gems/1.8/gems/narray-0.5.9.9
    checking for narray.h... yes
    checking for narray_config.h... yes
    checking for netcdf.h... yes
    checking for main() in -lnetcdf... yes
    creating Makefile
    

    Now let Linux compile all the code and create the shared libraries it wants:
    $ make
    gcc -I. -I. -I/usr/lib/ruby/1.8/i486-linux -I. -DHAVE_NARRAY_H -DHAVE_NARRAY_CONFIG_H -DHAVE_NETCDF_H -I/usr/local/include -I/usr/lib/ruby/gems/1.8/gems/narray-0.5.9.9  -D_FILE_OFFSET_BITS=64  -fPIC -fno-strict-aliasing -g -g -O2  -fPIC   -c netcdfraw.c
    gcc -shared -o netcdfraw.so netcdfraw.o -L. -L/usr/lib -L/usr/local/lib -L/usr/local/lib/site_ruby/1.8/i486-linux -L. -Wl,-Bsymbolic-functions -rdynamic -Wl,-export-dynamic    -lruby1.8 -lnetcdf  -lpthread -lrt -ldl -lcrypt -lm   -lc
    

    Finally, install it:
    $ sudo make install
    [sudo] password for andy: 
    mkdir -p /usr/local/lib/site_ruby/1.8/i486-linux/numru
    /usr/bin/install -c -m 0755 netcdfraw.so /usr/local/lib/site_ruby/1.8/i486-linux/numru
    mkdir -p /usr/local/lib/site_ruby/1.8/numru
    /usr/bin/install -c -m 644 ./lib/netcdf_miss.rb /usr/local/lib/site_ruby/1.8/numru/
    /usr/bin/install -c -m 644 ./lib/netcdf.rb /usr/local/lib/site_ruby/1.8/numru/
    

    This should complete the installation of RubyNetCDF. The installation instructions recommend running a test:
    $ make test
    test.rb:3:in `require': no such file to load -- narray (LoadError)
     from test.rb:3
    make: *** [test] Error 1
    

    This tells us that it tried to run a ruby script called test.rb, and in line 3 it failed to execute a 'require' command on the file "narray." Adding the line
    require 'rubygems'
    to the top of this file will solve the problem and allow the test to run successfully.

    Alternative/better method: Install ruby-netcdf as a gem

    RubyNetCDF exists as a gem. If at the command line you type the command

    $ gem list ruby-netcdf --remote

    you'll see ruby-netcdf (0.6.5) listed. So I tried

    sudo gem install ruby-netcdf

    but that failed as follows:

    Building native extensions.  This could take a while...
    ERROR:  Error installing ruby-netcdf:
     ERROR: Failed to build gem native extension.
    
    /usr/bin/ruby1.8 extconf.rb
    extconf.rb:3: uninitialized constant Gem (NameError)
    
    Gem files will remain installed in /usr/lib/ruby/gems/1.8/gems/ruby-netcdf-0.6.5 for inspection.
    Results logged to /usr/lib/ruby/gems/1.8/gems/ruby-netcdf-0.6.5/gem_make.out

    Note that it left the files in the directory where gems would go, so we can navigate to that directory and install it manually.

    $ cd /usr/lib/ruby/gems/1.8/gems/ruby-netcdf-0.6.5
    Next, run the Ruby script with the second argument telling it where to find the NArray files (I use sudo just to preempt potential permission denied errors):
    $ ruby -rubygems extconf.rb --with-narray-include=/usr/lib/ruby/gems/1.8/gems/narray-0.5.9.9
    That produces the following output:

    checking for narray.h... yes
    checking for narray_config.h... yes
    checking for netcdf.h... yes
    checking for main() in -lnetcdf... yes
    creating Makefile
    

    If you get the error extconf.rb:3: uninitialized constant Gem (NameError), then make sure you're including the argument -rubygems above. That's the equivalent of inserting at the top of the file the line
    require 'rubygems'

    The next step is to run the make command, which lets Linux compile libraries and do a few other things:
    $ sudo make
    gcc -I. -I. -I/usr/lib/ruby/1.8/i486-linux -I. -DHAVE_NARRAY_H -DHAVE_NARRAY_CONFIG_H -DHAVE_NETCDF_H -I/usr/local/include -I/usr/lib/ruby/gems/1.8/gems/narray-0.5.9.9  -D_FILE_OFFSET_BITS=64  -fPIC -fno-strict-aliasing -g -g -O2  -fPIC   -c netcdfraw.c
    gcc -shared -o netcdfraw.so netcdfraw.o -L. -L/usr/lib -L/usr/local/lib -L/usr/lib/ruby/gems/1.8/gems/narray-0.5.9.9/ -L. -Wl,-Bsymbolic-functions -rdynamic -Wl,-export-dynamic    -lruby1.8 -lnetcdf  -lpthread -lrt -ldl -lcrypt -lm   -lc
    

    Finally, run sudo make install to finish:
    $ sudo make install
    /usr/bin/install -c -m 0755 netcdfraw.so /usr/local/lib/site_ruby/1.8/i486-linux/numru
    /usr/bin/install -c -m 644 ./lib/netcdf_miss.rb /usr/local/lib/site_ruby/1.8/numru/
    /usr/bin/install -c -m 644 ./lib/netcdf.rb /usr/local/lib/site_ruby/1.8/numru/

    The final step is run a test to see if the installation of ruby-netcdf was successful, but mine got an error:
    $ sudo make test
    test.rb:3:in `require': no such file to load -- narray (LoadError)
     from test.rb:3
    make: *** [test] Error 1
    

    What's going on here? It's calling the file test.rb, located in the subdirectory "/test." This fails because line 3 of the file test.rb is as follows:
    require 'narray'
    For this line to work with Ruby 1.8.7, it needs to have RubyGems initialized. This is usually done by requiring RubyGems at the top of the file:
    require 'rubygems'
    require 'narray'
    This is why we included the -rubygems argument above, but we can't do that here because we're not calling ruby, we're calling the Linux command make.

    To get around this, I resorted to common hackery: open the test.rb file, and add the line manually to the top. This is what the first few lines of my file look like after I insert the new line 3:
    ##require 'numru/netcdf' 
    ## // to test before make install -->
    require 'rubygems'
    require 'narray'
    require '../netcdfraw'  
    require '../lib/netcdf'
    ## <-- to test before make install //
    
    include NumRu
    
    Now try running the test again:
    $ sudo make test
    /usr/local/lib/site_ruby/1.8/i486-linux/numru/netcdfraw.so: warning: already initialized constant NC_NOWRITE
    /usr/local/lib/site_ruby/1.8/i486-linux/numru/netcdfraw.so: warning: already initialized constant NC_WRITE
    /usr/local/lib/site_ruby/1.8/i486-linux/numru/netcdfraw.so: warning: already initialized constant NC_SHARE
    /usr/local/lib/site_ruby/1.8/i486-linux/numru/netcdfraw.so: warning: already initialized constant NC_CLOBBER
    /usr/local/lib/site_ruby/1.8/i486-linux/numru/netcdfraw.so: warning: already initialized constant NC_NOCLOBBER
    creating test.nc...
    test did not fail :-p (please ignore the warnings)
    
    The message at the end, along with the friendly emoticon, tell us that the test succeeded. We have completed the installation of RubyNetCDF.

    Confirm installation succeeded

    We can confirm that everything is working by opening a Ruby session (but not in the protected gem directory)...
    $ cd ~
    $ irb
    ...and interacting with NetCDF.
    irb(main):001:0> require 'numru/netcdf'
    => true
    irb(main):002:0> file = NumRu::NetCDF.create("test.nc")
    => NetCDF:test.nc
    irb(main):003:0> file.close
    => nil
    irb(main):004:0> exit
    
    That confirms that everything is working.

    Conclusion

    To summarize, we completed four steps:
    1. We installed Ruby 1.8.7
    2. Then we installed NArray 0.5.9.9 
    3. Then we installed NetCDF 
    4. Then we installed Ruby-NetCDF 0.6.5 
    However, we are left with one problem: the ruby-netcdf gem is not listed when we ask RubyGems for a list of installed gems:
    $ gem list ruby-netcdf
    
    *** LOCAL GEMS ***
    
    
    However, the program works, so I will worry about this later.
    ~Fin~

    Lessons Learned

    • The gem version of ruby-netcdf (gem install ruby-netcdf) is different from the tarball version available from the website (http://ruby.gfd-dennou.org/products/ruby-netcdf/#download). Specifically, the files extconf.rb and extconf.rb.orig.
    • Installation instructions seem to assume Ruby 1.9 (which obviates RubyGems issue below).
    • If you're using Ruby 1.8, you must ensure that Ruby includes RubyGems. For details on how to do this, see the RubyGems documentation on this issue.
    • Installation instructions do not specify that the program requires NetCDF to have the additional package "libnetcdf-dev."
    • Even after successful installation of all parts, ruby-netcdf doesn't show up as a gem after a call to
      $ gem list ruby-netcdf