Long Paddock

 

A new beta version of the Long Paddock website is now available to view - click here for a preview

SILO Data formats

SILO Data Drill and Patched Point Data are available in a range of formats tailored to client requirements. The available formats are:

 

Data format Description Sample Data
Standard Format is widely used for biophysical modelling. It provides data for: daily rainfall, maximum and minimum temperature, pan evaporation, solar radiation, vapour pressure, and relative humidity at the times of maximum and minimum temperature. Data drill;
PPD  
Standard including ASCE reference evapotranspiration (tall crop ETo)

Similar to the Standard format, but also includes potential evapotranspiration (ETo). The ETo estimates are calculated using the ASCE Penman-Monteith formula with a default wind value of 2 m/s. The Penman-Monteith formula also requires radiation as an input. Radiation is derived from cloud oktas and hours of sunshine duration using the procedure documented in Zajaczkowski et al.(2013).

Note: this is the ASCE’s tall crop version of the Penman-Monteith equation.

Data drill;
PPD 

Standard including FAO56 reference evapotranspiration (short crop ETo)

Similar to the Standard format, but also includes potential evapotranspiration (ETo). The ETo estimates are calculated using the FAO Penman-Monteith formula with a default wind value of 2 m/s. The Penman-Monteith formula also requires radiation as an input. Radiation is derived from cloud oktas and hours of sunshine duration using the procedure documented in Zajaczkowski et al.(2013).

Note: this is the FAO’s short crop version of the Penman-Monteith equation.

Data drill;
PPD 
Morton's Similar to the Standard format, but also includes Morton's estimates of the following: (i) evaporation over shallow lakes; (ii) potential evapotranspiration over land; (iii) actual evapotranspiration over land; and (iv) wet environment areal evapotranspiration over land. Data drill;
PPD 
Evaporation*
combination
Similar to the Standard format, but includes an additional column (denoted Evap_Sp) containing either: (i) a synthetic estimate of pan evaporation (1/1/1889 to 31/12/1969); or (ii) observed (or patched) Class A pan evaporation (1/1/1970 to present). Data drill;
PPD 
Evaporation*
comparison
Similar to the Standard format, but includes an additional column containing a synthetic estimate of pan evaporation (1/1/1889 to present). This format differs from the "Evaporation combination" format in that all data in the additional column (denoted Span) are synthetic estimates. Data drill;
PPD 
Programmer's data for comparison Format contains most variables provided by SILO: the default variables included in the Standard format as well as:
  • FAO56 (short crop) estimate of potential evapotranspiration (using a wind speed of 2m/s);
  • synthetic evaporation;
  • Morton's estimates of evaporation/evapotranspiration; and
  • mean sea level pressure.

    This format may be useful for new clients wishing to evaluate our products.
Data drill;
PPD 
Programmer's data for comparison with ASCE Reference Evapotranspiration

Identical to the “Programmer's data for comparison” format, but also includes the ASCE’s tall crop estimate of potential evapotranspiration (ETo).

Data drill;
PPD

RAINMAN Format is for use with the RAINMAN application. It provides data for rainfall, maximum and minimum temperatures, Class A pan evaporation, vapour pressure and solar radiation. The data for each variable are in separate files. It also includes instructions in a Readme text file. Data drill;
PPD 
Monthly Format provides: (i) monthly totals for rainfall and evaporation; and (ii) monthly means for maximum and minimum temperatures, solar radiation and vapour pressure. Data drill;
PPD 
Rainfall only Format contains daily rainfall; data are not provided for any other variable. Data drill;
PPD 
 
APSIM Format is for use with the Agricultural Production Simulator (APSIM). Data drill;
PPD 
P51

Format is for use with the Grass Production (GRASP) model and the Model for Effluent Disposal and Land Irrigation (MEDLI) V2 model.

Note: this format is used by many biophysical modellers as it contains the core variables in a mimimal format.

Data drill;
PPD 
3PG Format is for use with CSIRO's Physiological Processes Predicting Growth (3PG) model. Data drill;
PPD 
CenW Forest Growth Model Format is for use with the CenW Forest Growth Model. Data drill;
PPD 
Century Format is for use with the CENTURY model. CENTURY is a general model of plant-soil nutrient cycling which has been used to simulate carbon and nutrient dynamics for different types of ecosystems including grasslands, agricultural lands, forests and savannas. Data drill;
PPD 
RUSTIC Format is for use with the Runoff, Storage & Irrigation Calculator (RUSTIC). Rainfall and Class A evaporation data are provided in separate files. Data drill;
PPD  

 * Please note, evaporation data are provided with a day-shift applied. For example: evaporation data for 31/01/2017 were actually reported on 01/02/2017 (day after). The shift to the previous day is applied because most (15 hours) of the recorded evaporation occurs in the previous day (measurement is taken at 9 am). The day-shift is applied to the Patched Point Data and Data Drill formats and also to the evaporation combination format (with the use of synthetic evaporation data). However, the observed evaporation data (class A pan) are interpolated without a day-shift applied and gridded datasets are not day-shifted.

Many of the data formats share a common structure consisting of two blocks of information:

1. Descriptive block.


Several formats include a header describing the data. The metadata in the header is structured as follows:

  1. Information for spreadsheet users (approximately 7 lines of text). The first line contains dummy data to enable a spread sheet program to sense the data format; and
  2. Information about the data requested: Approximately 20 lines of text containing:
    1. For DataDrill datasets:
      i. the latitude and longitude of the location; and
      ii. the user supplied reference (a single word, up to 10 characters long).
    2. For Patched Point datasets:
      i. the station details, including the latitude and longitude
    3. The elevation at the location of interest
    4. The date that the data were extracted from the database; an
    5. A link to the SILO webpage where the user can find documentation and notices.

2. Data block


The data block typically contains:


Note: It is strongly recommended that the metadata are retained in the file. If you are writing a program to import the data then the metadata and headings can easily be skipped by testing the first character in each line. If the 1st character is a 1 or 2 then it is a data line, otherwise it is metadata.

The date, variable labels and variable source labels are:

Column label Description Source label
Date Date represented using yyyymmdd, dd/mm/yyyy or d-mm-yyyy not applicable
Day

Day of year as a 3 digit integer, range 1-366

not applicable
T. Max Maximum temperature Smx
T. Min Minimum temperature Smn
Rain Rain (including other precipitation) Srn
Evap Class A Evaporation Sev
Radn Short wave solar radiation for a horizontal surface Ssl
VP Atmospheric water vapour pressure Svp
MSLPres Mean Sea Level Pressure Sp
RHmaxT Relative humidity at temperature T.Max, derived from T.Max and VP not applicable
RHminT Relative humidity at temperature T.Min, derived from T.Min and VP not applicable
Span Synthetic class A pan evaporation based on vapour pressure deficit and solar radiation Ssp
Evap_Sp or EvSp Observed Class A evaporation (1970-present), supplemented by synthetic pan evaporation (pre-1970) Ses
FAO-56 Reference potential evapotranspiration, calculated using the FAO56 formula not applicable
Mact Morton's estimate of actual evapotranspiration over land not applicable
Mpot Morton's estimate of potential evapotranspiration over land not applicable
Mlake Morton's estimate of evaporation over shallow lakes not applicable
Mwet Morton's estimate of wet environment areal evapotranspiration over land not applicable
VP Deficit Vapour pressure deficit not applicable
Rain Days Number of rain days in the month not applicable
Frost 0.0 Number of frost days at 0°C threshold not applicable
Frost 1.0 Number of frost days at 1°C threshold not applicable
Frost 2.5 Number of frost days at 2.5°C threshold not applicable
Avg Temp Monthly mean of daily temperature (defined as Tmin + 0.75 x [Tmax – Tmin]) not applicable
Avg TMax Monthly mean maximum temperature not applicable
Avg TMin Monthly mean minimum temperature not applicable
Avg Rad or Solar radtn Monthly mean solar radiation not applicable
Avg VP Monthly mean vapour pressure not applicable
Tot Rain Monthly total rainfall not applicable
Tot Evap Monthly total evaporation not applicable

 

The source code indicates how each datum was obtained:

Patched Point Dataset (PPD) source code values and meanings

Source code Meaning in Patched Point data Meaning in DataDrill data
0 Official observation as supplied by the Bureau of Meteorology Not applicable
13 Official observation recorded over a period exceeding the standard 24 hour observation period, that has been redistributed to daily data using daily observations from a comparable station Not applicable
15 As for 13, but redistributed using interpolated data (see code 25) Not applicable
23

Comparable station, official observation supplied by the Bureau of Meteorology

Not applicable
25 Interpolated from daily observations for that date  
26 Synthetic Class A pan evaporation, calculated from interpolated temperatures, radiation and vapour pressure  
35 Interpolated from daily observations using an anomaly interpolation method (CLIMARC data)  
75 Interpolated from the long term averages of daily observations for that day of year  

 

Additional notes specific to each format are provided with the sample data available in the table of formats shown above.

Notes on the CLIMARC interpolation method (code 35)

An anomaly interpolation technique was used to construct the gridded surfaces for variables other than rainfall, for all years prior to 1957. The surfaces were derived, in part, using observational data made available under the CLIMARC project. Prior to the availability of the CLIMARC datasets, there were insufficient data to construct gridded surfaces so long term mean data was typically used.

The user should note that CLIMARC interpolations:

  1. are used throughout the period 1889-1956;
  2. are based on relatively few observations. For example, there are data for about 60 CLIMARC stations compared to several hundred stations reporting temperature in the post-1957 period;
  3. show less variation than the post-1957 data; and
  4. are derived from relatively old data which in some cases contain uncorrected instrument biases.

Consequently the interpolated CLIMARC data may not be suitable for some studies e.g. climate change detection, extreme events, number of frosts etc.

We expect the interpolated CLIMARC data to be significantly better than long term averages. Furthermore, the CLIMARC data should preserve the daily relationships between different elements better than long term means. It should however be noted that in some areas and years, the CLIMARC data are no different to the previously supplied long term averages. For example, in Western Australia there are very few observations before 1907, and Ceduna didn't commence reporting until about 1940. Please consult the Technical Report (PDF, 2.5M, last updated 09:13AM, 24 June 2010)* for further information.

Notes on redistribution (codes 13,15)

Much of the data in Australia have been collected by volunteer observers at Post Offices, Police stations, etc.  Some of these workplaces only operate on weekdays. In this case, for example, the rainfall measured on Monday will be for the period since 9am Friday. SILO redistributes this rainfall back to the days when it probably fell according to the amount and days that rain fell at nearby stations.

Notes on comparable stations (codes 13, 23)

Sometimes when a station ceases recording and there is another one recording nearby, the data for the two stations will be compared to see if they can be treated as a single 'composite' station.  This might happen, for example, with a Post Office and Hospital in the same town. In this case, data for the PPD station that was requested will be indicated by a source code of 0, and data coming from the other 'comparable' station will be indicated with a source code of 23.  Sometimes the two stations will have recorded for an overlapping period and the comparable station used to redistribute data for the requested station. When this happens a source code of 13 is used.

Brief history of CLIMARC

The CLIMARC project was led by Nick Clarkson (Queensland Department of Primary Industries) with support from several state and federal departments. CLIMARC increased the amount of climate data, i.e. non-rainfall data such as temperature, for the period before 1957. When the Bureau of Meteorology commenced storing observations electronically, it decided to computerise all rainfall data but not the climate data before 1957 for cost reasons. The CLIMARC project digitised the pre-1957 climate data for 50 locations. Prior to CLIMARC there were only 5 locations in Australia where the entire observational record had been digitised. A number of other locations had already been partly digitised either by the Bureau of Meteorology or other organisations. The pre-1957 data for the 50 CLIMARC locations, together with the small amount of data which were already available, made it possible to construct gridded surfaces for the pre-1957 period. The methodology is described in the CLIMARC report (PDF, 2.5M, last updated 09:13AM, 24 June 2010)*.


 

* Requires Adobe Reader

Last updated 24 October 2016

SILO climate data