Long Paddock

 

Please let us know what you think of the Long Paddock website. Take 2 Minute Survey!

Programmer notes

Technical specifications for the standard data format are included below.

 

SILO Data Drill and Patched Point Data files consist of two blocks of information:

Descriptive block.

All lines start with a double quote ". There are two sub-sections:

Data block

Note: It is strongly recommended that the Metadata always be kept in the file. If you are writing a program to import 'Standard format' into a program (such as C, Fortran, perl, awk, Basic,) then the Metadata and headings can easily be skipped by testing the first character in the file. If the 1st character is a 1 or 2 then it is a data line, otherwise it is Metadata or headings.

This description is intended to be a generic standard, and more data columns may be added in the future. So it is essential that programs written to use this standard sense the position of the columns that they require using the column titles.

Fields currently included in the Standard format

Below is a list of fields currently included in the Standard format. Other formats may contain other data fields, but these programming notes remain relevant.

Column title Data description

Date

Date as an 8 digit integer ie yyyymmdd

Day

Day of year as a 3 digit integer, range 1-366

Date2

Date in dd/mm/yyyy format intended for use in spreadsheets

T.Max

Maximum temperature

T.Min

Minimum temperature

Rain

Rain (including other precipitation)

Evap

Evaporation

Radn

Short wave Solar Radiation for a horizontal surface

VP

Atmospheric Water Vapour Pressure

RHmaxT

Relative Humidity at temperature T.Max, derived from T.Max and VP

RHminT

Relative Humidity at temperature T.Min, derived from T.Min and VP

FAO-56

Reference potential Evapotranspiration, calculated using the FAO56 formula

 

Source code fields

Data field Corresponding Source code field

T.Max

Smx

T.Min

Smn

Rain

Srn

Evap

Sev

Radn

Ssl

VP

Svp

Each source code field indicates the source of the data in the corresponding data field.

 

Data Drill source code values and meanings

Value Meaning

25

interpolated from daily observations for that date

26

Synthetic Class A pan evaporation, calculated from interpolated temperatures, radiation and vapour pressure

35

interpolated from daily observations using anomaly interpolation method for CLIMARC data

75

interpolated from the long term averages of daily observations for that day of year

These values and meanings will not be changed. However new codes may be added in the future. Data Drill source codes are a subset of the ones used for the Patched Point Dataset (PPD).

 

Patched Point Dataset (PPD) source code values and meanings

Value Meaning

0

Official observation as supplied by Bureau of Meteorology (may come from a volunteer)

13

Official observation that was from a period > 1 day, redistributed to daily data using daily observations from a comparable station

15

As for 13, but redistributed using interpolated data (see code 25)

23

Comparable station, official observation supplied by the Bureau of Meteorology

25

interpolated from daily observations for that date

26

Synthetic Class A pan evaporation, calculated from interpolated temperatures, radiation and vapour pressure

35

interpolated from daily observations using anomaly interpolation method for CLIMARC data

75

interpolated from the long term averages of daily observations for that day of year

 

Notes on CLIMARC interpolation method (code 35)

The CLIMARC interpolations (code 35) provide better data than was previously available, however the quality of the data returned by the interpolation method is still lower than the normal interpolations (code 25) and these are also of lower quality than actual data (code 0) but better than long term averages (code 75).

The CLIMARC anomaly interpolations:

  1. are only in the period 1889-1956
  2. are based on far fewer observations (about 60 compared to the code 25 which is based on around 600 for temperature)
  3. show less variation than post 1957 data
  4. are derived from very old data that in some cases contain uncorrected instrument biases (small biases)
  5. are NOT suitable for certain studies because of 2,3,4. e.g. Climate Change studies, number of frosts, extreme events
  6. in some areas and years, are no different to the previously supplied long term averages (code 75), e.g. In Western Australia there are very few observations before 1907, and Ceduna didn't start until about1940. Please see Appendix E of the Technical Report to get a better idea of the effects of this.
  7. are a significant improvement on the long term averages (code 75) previously available for this period
  8. preserve to a higher degree the relationships between different elements, eg that minimum temperatures will be higher if it is cloudy.

Australian synthetic daily Class A pan evaporation (PDF, 1.8M, last updated 09:13AM, 24 June 2010)*

 

Notes on redistribution (codes 13,15)

Much of the data in Australia have been collected by volunteer observers at Post Offices, Police stations, etc.  Some of these workplaces only operate on weekdays. In this case, for example, the rainfall measured on Monday will be for the period since 9am Friday. SILO redistributes this rainfall back to the days when it probably fell according to the amount and days that rain fell at nearby stations.

 

Notes on comparable stations (codes 13, 23)

Sometimes when a station ceases recording and there is another one recording nearby, the data for the two stations will be compared to see if they can be treated as a single 'composite' station.  This might happen, for example, with a Post Office and Hospital in the same town. In this case, data for the PPD station that was requested will be indicated by a source code of 0, and data coming from the other 'comparable' station will be indicated with a source code of 23.  Sometimes the two stations will have recorded for an overlapping period and the comparable station used to redistribute data for the requested station. When this happens a source code of 13 is used. 

 

Brief history of CLIMARC

CLIMARC project was led by Nick Clarkson (Queensland Department of Primary Industries) with support from several state and federal departments. CLIMARC increased the amount of climate data, i.e. non-rainfall data such as temperature, for the period before 1957. When the Bureau of Meteorology started to store all observations electronically, they decided to computerise all rainfall data but not to computerise other climate data before 1957 for cost reasons. The CLIMARC project (>$300,000) was a project to computerise and make available the pre-1957 climate data for 50 locations. Prior to CLIMARC there were only 5 locations in Australia where the climate data had been electronically back to the start of their records. A number of other locations had been partly computerised in either by the Bureau of Meteorology or by other organisations. The pre-1957 data for the 50 CLIMARC locations together with the small amount of other early data that was available made it possible to interpolate these data, but using a methodology that is different to the post 1957 interpolations used in SILO. The methodology is described in the CLIMARC report and is also described elsewhere on SILO.

* Requires Adobe Reader

Last updated 28 October 2014

SILO climate data