Grid-SAFE

PBS Parser

Parser for PBS (Portable Batch System) records. These records can have a different format depending on their record type which is specified in argument two of the record. This parser is aware of all the standard PBS attributes that can be present an will parse them into PropertyTags with the same name as the PBS attribute. The tags will reside in a property registry called pbs. Additionally, extension attributes may be specified using configuration properties. Strictly speaking, the specification only allows extensions of the form Resource_List.resource_name and resources_used.resource_name, however this parser will allow extensions with any name.

Parsing different PBS record types

An instance of this parser will only parse certain record types. For all other record types, the parse will throw a SkipRecord exception. Which types are parsed and which are skipped is determined by the mode of this parser. The mode takes the form of a comma separated list of record types the parser can parse. This is set as a configuration property. During initialisation, the PBS Parser looks for the following property:

  pbs.recordTypes.table_name = comma,separated,type,list

for example

pbs.recordTypes.PBSReservation = B 

All record types in the list will be parsed. All other record types will be skipped. If the property isn't found, record type E is assumed. See one of the PBS specifications for a comprehensive list of all record types.

Standard PBS Attributes

All PBS attributes found in a PBS record are stored in AccountingProperties with the same name as the PBS attribute. The registry the AccountingProperties belong to is called pbs. Attributes are parsed regardless of whether or not they are allowed to be in a PBS record of the type being parsed.

The following is a list of all standard PBS attributes recognised, along with the parsers used to extract the value from the attribute. All parsers come from the package uk.ac.ed.epcc.safe.accounting.parsers.value.

Attribute	Parser Used to Extract Values	PBS record types it should be present in*
accounting_id	StringParser	S
account	StringParser	B, E
alt_id	StringParser	E
authorized_groups	StringParser	B
authorized_hosts	StringParser	B
authorized_users	StringParser	B
ctime	TimestampParser	B, E, S
date_time	PbsDateParser	All record types**
duration	IntegerParser	B
etime	TimestampParser	E, S
exec_host	StringParser	B, S
Exit_status	IntegerParser	E
group	StringParser	E, S
id_string	StringParser	All record types**
jobname	StringParser	E, S
owner	StringParser	B
qtime	TimestampParser	B, E
qtime	TimestampParser	E, S
queue	StringParser	B, E, Q, S
record_type	StringParser	All record types**
resvID	StringParser	B, E
resvname	StringParser	B, E
session	IntegerParser	S
start	TimestampParser	B, E, S
user	StringParser	E, S

* This parser will parse attribues even if they are not expected to be in a PBS record of the type being parsed. This column is included purely for convenience.

** If these attributes are missing or malformed, the record will not be parsed. An error will be reported on a per-record basis to this effect.

Declaring New PBS Attributes (Extension Attributes)

The PBS specification allows extra extensions to be used in it's record types. Most of these are of the form Resource_List.resource_name and resources_used.resource_name. To create new ones, the property format is:

pbs.table_name.attribute_type.attribute_name = type

Where table_name is the name of the database table in which the PBS records are being stored, attribute_name is the name of the attribute to parse (the name that will appear in the PBS record) and type is the type of value the the attribute is. The type can be any of the default or user defined types managed by the ValueParserService ValueParserService.

All extension attributes appear in the registry pbsExtension. Declaring extension attributes with the same name as standard attributes changes the type of the old attribute. The new declaration will, however be seen as an extension, meaning the AccountingProperty will be part of the pbsExtension registry.

Note Some PBS attribute extensions allow dots (.) in their name (for example, Resource_List.resource_name). This parser will accept attribute names with dots in them but change them to underscores for the accounting property names. This is because AccountingProperties may not contain dots in their names. For example, the configuration property pbs.PBSTable.attribute_type.resources_used.mem will parse an attribute called resources_used.mem from a PBS record, but will store the value in an accounting property called resources_used_mem. If a different name is preferred, make use of DerivedProperty to make a new accounting property with a more appropriate name.

Here is an example of some extension records:

pbs.Hector.attribute_type.Resource_List.place = string
pbs.Hector.attribute_type.Resource_List.walltime = duration
pbs.Hector.attribute_type.resources_used.ncpus = integer
pbs.Blue.attribute_type.resources_used.cpupercent = integer
pbs.Blue.attribute_type.vhost = string
pbs.Blue.attribute_type.startTime = timestamp

In the above example, when a PBS parser is being used to parse values into Hector, it will parse the following attributes:

attribute parsed	type it's value is parsed into	name of AccountingProperty associated with the value
Resource_List.place	string	Resource_List_place
Resource_List.walltime	duration	Resource_List_walltime
resources_used.ncpus	integer	resources_used_ncpus

When the PBS parser is used to parse values into Blue, it will parse parse the following attributes:

attribute parsed	type it's value is parsed into	name of AccountingProperty associated with the value
resources_used.cpupercent	integer	resources_used_cpupercent
vhost	string	vhost
starttime	timestamp	startTime

The database table names must be the same as the AccountingProperty names generated, although this can be changed by setting up a DerivedProperty to assign the value to a more appropriate name.