uk.ac.ed.epcc.safe.accounting.parsers

Class PbsParser

java.lang.Object

uk.ac.ed.epcc.safe.accounting.update.AbstractPropertyContainerUpdater

uk.ac.ed.epcc.safe.accounting.update.AbstractPropertyContainerParser

uk.ac.ed.epcc.safe.accounting.update.BaseParser

uk.ac.ed.epcc.safe.accounting.update.BatchParser

uk.ac.ed.epcc.safe.accounting.parsers.AbstractPbsParser

uk.ac.ed.epcc.safe.accounting.parsers.PbsParser

All Implemented Interfaces:

PropertyContainerParser, PropertyContainerUpdater, uk.ac.ed.epcc.webapp.Contexed

public class PbsParser
extends AbstractPbsParser
implements uk.ac.ed.epcc.webapp.Contexed, PropertyContainerParser

Parser for PBS (Portable Batch System) records. These records can have a different format depending on their record type which is specified in argument two of the record. The different attributes record types can have are hard coded into the PbsRecordType object this parser uses to parse records. Additionally, extension attributes may be specified using properties set in the ConfigService stored in the AppContext used to construct a PbsParser. Strictly speaking, the specification only allows extensions of the form Resource_List.resource_name and resources_used.resource_name, however this parser will allow extensions with any name.

Declaring New PBS Attributes (Extension Attributes)

The PBS specification allows extra extensions to be used in it's record types. Most of these are of the form Resource_List.resource_name and resources_used.resource_name. To create new ones, the property format is:

pbs.table_name.attribute_type.attribute_name = parser

Where table_name is the name of the database table in which the PBS records are being stored, attribute_name is the name of the attribute to parse (the name that will appear in the PBS record) and parser is the parser to use to parse the value. There are several parsers already defined that can be used to parse values. It is also possible to write custom parsers and use them too. See ValueParserService for more information on what parsers are provided by default, as well as how to add custom parsers.

Note: Some pbs attribute extensions allow dots in their name (Resource_List.resource_name). This parser will accept attribute names with dots in them but change them to underscores in the accounting properties. For example, the configuration property pbs.PBSTable.attribute_type.resources_used.mem will parse an attribute called resources_used.mem from a PBS record, but will store the value in an accounting property called resources_used_mem. If a different name is preferred, make use of DerivedPropertyPolicy to make a new accounting property with a more appropriate name.

Record Types Property

An instance of this parser will only parse certain record types. For all other record types, the parse will throw a SkipRecord exception. Which types are parsed and which are skipped is determined by the mode of this parser. The mode takes the form of a comma separated list of record types the parser can parse. This is set as a property in the ConfigService stored in the AppContext used to construct the PbsParser. During initialisation, a PbsParser looks for the following property:

pbs.recordTypes.table_name = comma,separated,type,list

for example

pbs.recordTypes.PBSReservation = B

All record types in the list will be parsed. All other record types will be skipped. If the property isn't found, record type E is assumed.

Author:

jgreen4, adrianj

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PbsParser.PBSNodesCPUEntryMaker Nodes cpu EntryMaker Some varients of PBS use a modes=:ppn= syntax for the nodes

Field Summary

Fields

Modifier and Type	Field and Description
static PropertyRegistry	ADDITIONAL_REGISTRY
static PropertyTag<java.lang.String>	PBS_NODES_PROP
static PropertyTag<java.lang.Integer>	PBS_NUM_CPUS_PROP
static PropertyTag<java.lang.Integer>	PBS_NUM_NODES_PROP
static PropertyTag<java.lang.Integer>	PBS_PROC_PER_NODE_PROP
static PropertyTag<java.lang.String>	PBS_USER_HOST_PROP

Fields inherited from class uk.ac.ed.epcc.safe.accounting.parsers.AbstractPbsParser

PBS_ACCOUNT_PROP, PBS_ACCOUNTING_ID_PROP, PBS_ALTERNATE_ID_PROP, PBS_AUTHORIZED_GROUPS_PROP, PBS_AUTHORIZED_HOSTS_PROP, PBS_AUTHORIZED_USERS_PROP, PBS_ELIGIBLE_TIME_PROP, PBS_ENDED_PROP, PBS_ENTERED_QUEUE_PROP, PBS_EXEC_HOST_PROP, PBS_EXIT_PROP, PBS_GROUP_PROP, PBS_ID_STRING_PROP, PBS_JOB_NAME_PROP, PBS_JOB_TIMESTAMP_PROP, PBS_OWNER_PROP, PBS_QUEUE_PROP, PBS_RECORD_TYPE_PROP, PBS_RESERVATION_DURATION_PROP, PBS_RESERVATION_ID_PROP, PBS_RESERVATION_NAME_PROP, PBS_SESSION_PROP, PBS_STARTED_PROP, PBS_SUBMITTED_PROP, PBS_TIME_USED_PROP, PBS_USER_PROP

Fields inherited from class uk.ac.ed.epcc.safe.accounting.update.BatchParser

ACCOUNT_PROP, batch, BATCH_SCRIPT_PROP, CPU_WAIT_PROP, EXPECTED_FINISH, JOB_ID_PROP, JOB_NAME_PROP, NODE_COUNT_PROP, PARTITION_PROP, PROC_COUNT_PROP, QUEUE_PROP, REQUEST_ACCURACY_PROP, REQUESTED_CPUS_PROP, REQUESTED_WALLCLOCK_PROP, RESIDENCY_PROP, SERIAL_PROP, SLOWDOWN_PROP, SUBJOB_PROP, SUBMITTED_PROP, SUBMITTED_TIMESTAMP, SUCCESS_PROP, TIME_PROP, WAIT_PROP, WALLCLOCK_PROP

Fields inherited from class uk.ac.ed.epcc.safe.accounting.update.AbstractPropertyContainerParser

DUPLICATE_KEY

Constructor Summary

Constructors

Constructor and Description
PbsParser(uk.ac.ed.epcc.webapp.AppContext context) Constructs a new PbsParser.

Method Summary

Methods

Modifier and Type	Method and Description
ContainerEntryMaker	addExtension(java.lang.String attributeName, ContainerEntryMaker maker) Adds a new extension to the parser.
PropExpressionMap	getDerivedProperties(PropExpressionMap previous) return a set of derived property definitions.
protected ContainerEntryMaker	getEntryMaker(java.lang.String attr)
MakerMap	getExtensions() Returns the non-standard attributes that this parser uses.
PropertyFinder	initFinder(uk.ac.ed.epcc.webapp.AppContext ctx, PropertyFinder prev, java.lang.String mode) Generate a PropertyFinder that will find any PropertyTag generated by this class.

Methods inherited from class uk.ac.ed.epcc.safe.accounting.parsers.AbstractPbsParser

endParse, getContext, parse, startParse

Methods inherited from class uk.ac.ed.epcc.safe.accounting.update.BaseParser

readDouble, readInteger, readLong, readTime

Methods inherited from class uk.ac.ed.epcc.safe.accounting.update.AbstractPropertyContainerParser

getDefaultUniqueProperties, modifyDefaultTableSpecification, splitRecords, unique

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface uk.ac.ed.epcc.webapp.Contexed

getContext

Methods inherited from interface uk.ac.ed.epcc.safe.accounting.update.PropertyContainerParser

getDefaultUniqueProperties, parse, splitRecords

Methods inherited from interface uk.ac.ed.epcc.safe.accounting.update.PropertyContainerUpdater

endParse, modifyDefaultTableSpecification, startParse

Field Detail

ADDITIONAL_REGISTRY

public static final PropertyRegistry ADDITIONAL_REGISTRY

PBS_USER_HOST_PROP

@AutoTable(length=64)
public static final PropertyTag<java.lang.String> PBS_USER_HOST_PROP

PBS_NODES_PROP

@AutoTable
public static final PropertyTag<java.lang.String> PBS_NODES_PROP

PBS_NUM_NODES_PROP

@AutoTable
public static final PropertyTag<java.lang.Integer> PBS_NUM_NODES_PROP

PBS_NUM_CPUS_PROP

@AutoTable
public static final PropertyTag<java.lang.Integer> PBS_NUM_CPUS_PROP

PBS_PROC_PER_NODE_PROP

public static final PropertyTag<java.lang.Integer> PBS_PROC_PER_NODE_PROP

Constructor Detail

PbsParser

public PbsParser(uk.ac.ed.epcc.webapp.AppContext context)

Constructs a new PbsParser. The parser should not be used until it's initFinder(AppContext, PropertyFinder, String) method has been called.

Parameters:

context - The context in which this parser is operating

Method Detail

addExtension

public ContainerEntryMaker addExtension(java.lang.String attributeName,
                               ContainerEntryMaker maker)

Adds a new extension to the parser. When the parser next parses a file, an attribute with the same name as the one specified here will use the maker to parse the value of the attribute and populate a PropertyContainer accordingly.

Parameters:

attributeName - The name of the attribute this maker should be used for

maker - The object that will parse the attribute's value and generate entries in a PropertyContainer

Returns:

The previous ContainerEntryMaker that as being used for this attribute, or null if there was no ContainerEntryMaker associated with attributes of this name.

getExtensions

public MakerMap getExtensions()

Returns the non-standard attributes that this parser uses. These are attributes not explicitly and completely defined in the PBS specification but may be used by PBS records. These can be attributes of the form Resorce_List.RESOURCE_NAME or resources_used.RESOURCE_NAME as allowed by the PBS specification, however they may be other types of attributes.

Returns:

The extension attributes used by this PBS parser in a name-to attribute maker map.

initFinder

public PropertyFinder initFinder(uk.ac.ed.epcc.webapp.AppContext ctx,
                        PropertyFinder prev,
                        java.lang.String mode)

Description copied from interface: PropertyContainerUpdater

Generate a PropertyFinder that will find any PropertyTag generated by this class. This call should be treated as a secondary constructor and should be called before any other method from this interface.

All properties defined at run-time should have their PropertyTag objects constructed by the time the first call to this method returns. As some parsers properties will generate different things depending on the other properties already in scope this method is passed a PropertyFinder for any properties that may already be in scope. This will include any default parameters specified when the data was uploaded and any properties generated from the database meta-data. Though it is perfectly legal to return a PropertyFinder that returns a superset of the generated properties the only requirement is that the returned finder can find any property that this classes parse method may actually generate values for. So Normally the contents of the super-finder are not included in the result unless these values are actually generated by the parse. If values are generated then a MultiFinder can be populated with the PropertyRegistrys from the generated properties

Specified by:

initFinder in interface PropertyContainerUpdater

Overrides:

initFinder in class AbstractPbsParser

prev - PropertyFinder for the higher levels in the call chain.

mode - String name of the destination table in case per-table customisation is needed

Returns:

PropertyFinder or null

getEntryMaker

protected ContainerEntryMaker getEntryMaker(java.lang.String attr)

Overrides:

getEntryMaker in class AbstractPbsParser

getDerivedProperties

public PropExpressionMap getDerivedProperties(PropExpressionMap previous)

Description copied from interface: PropertyContainerUpdater

return a set of derived property definitions. Each derived property is defined in terms of a simple expression over other properties (that may themselves be derived properties).

Definitions are allowed for properties that are also generated directly in the parse phase. In this case the parse value should always be used by preference and the derived definition only provides a fall-back implementation if the parse value is unavailable. For example if not persisted in the database layer. This allows one parser/policy to specify a default implementation as a derived property definition which is then overridden by having the same property generated in a parse method of a different policy/parser

Specified by:

getDerivedProperties in interface PropertyContainerUpdater

Overrides:

getDerivedProperties in class AbstractPbsParser

Parameters:

previous - Previous definitions

Returns:

Map modified map