uk.ac.ed.epcc.safe.accounting.parsers

Class AbstractPbsParser

java.lang.Object

uk.ac.ed.epcc.safe.accounting.update.AbstractPropertyContainerUpdater

uk.ac.ed.epcc.safe.accounting.update.AbstractPropertyContainerParser

uk.ac.ed.epcc.safe.accounting.update.BaseParser

uk.ac.ed.epcc.safe.accounting.update.BatchParser

uk.ac.ed.epcc.safe.accounting.parsers.AbstractPbsParser

All Implemented Interfaces:

PropertyContainerParser, PropertyContainerUpdater, uk.ac.ed.epcc.webapp.Contexed

Direct Known Subclasses:

PbsParser

public abstract class AbstractPbsParser
extends BatchParser
implements uk.ac.ed.epcc.webapp.Contexed, PropertyContainerParser

Abstract Parser for variants of the PBS (Portable Batch System) records. PBS uses a meta-format which is extensible using name-value pairs of attributes. these attributes can in turn have complex structure that may generate multiple properties so each attribute is processed by a ContainerEntryMaker. This class defines the subset of common properties that are common to most PBS versions but will need to be extended to parse a usable set of properties.

Record Types Property

An instance of this parser will only parse certain record types. For all other record types, the parse will throw a SkipRecord exception. Which types are parsed and which are skipped is determined by the mode of this parser. The mode takes the form of a comma separated list of record types the parser can parse. This is set as a property in the ConfigService stored in the AppContext used to construct the PbsParser. During initialisation, a PbsParser looks for the following property:

pbs.recordTypes.table_name = comma,separated,type,list

for example

pbs.recordTypes.PBSReservation = B

All record types in the list will be parsed. All other record types will be skipped. If the property isn't found, record type E is assumed.

Author:

jgreen4, adrianj, spb

Field Summary

Fields

Modifier and Type	Field and Description
static PropertyTag<java.lang.String>	PBS_ACCOUNT_PROP
static PropertyTag<java.lang.String>	PBS_ACCOUNTING_ID_PROP
static PropertyTag<java.lang.String>	PBS_ALTERNATE_ID_PROP
static PropertyTag<java.lang.String>	PBS_AUTHORIZED_GROUPS_PROP
static PropertyTag<java.lang.String>	PBS_AUTHORIZED_HOSTS_PROP
static PropertyTag<java.lang.String>	PBS_AUTHORIZED_USERS_PROP
static PropertyTag<java.util.Date>	PBS_ELIGIBLE_TIME_PROP
static PropertyTag<java.util.Date>	PBS_ENDED_PROP
static PropertyTag<java.util.Date>	PBS_ENTERED_QUEUE_PROP
static PropertyTag<java.lang.String>	PBS_EXEC_HOST_PROP
static PropertyTag<java.lang.Integer>	PBS_EXIT_PROP
static PropertyTag<java.lang.String>	PBS_GROUP_PROP
static PropertyTag<java.lang.String>	PBS_ID_STRING_PROP
static PropertyTag<java.lang.String>	PBS_JOB_NAME_PROP
static PropertyTag<java.util.Date>	PBS_JOB_TIMESTAMP_PROP
static PropertyTag<java.lang.String>	PBS_OWNER_PROP
static PropertyTag<java.lang.String>	PBS_QUEUE_PROP
static PropertyTag<java.lang.String>	PBS_RECORD_TYPE_PROP
static PropertyTag<java.lang.Number>	PBS_RESERVATION_DURATION_PROP
static PropertyTag<java.lang.String>	PBS_RESERVATION_ID_PROP
static PropertyTag<java.lang.String>	PBS_RESERVATION_NAME_PROP
static PropertyTag<java.lang.Number>	PBS_SESSION_PROP
static PropertyTag<java.util.Date>	PBS_STARTED_PROP
static PropertyTag<java.util.Date>	PBS_SUBMITTED_PROP
static PropertyTag<java.lang.Number>	PBS_TIME_USED_PROP
static PropertyTag<java.lang.String>	PBS_USER_PROP

Fields inherited from class uk.ac.ed.epcc.safe.accounting.update.BatchParser

ACCOUNT_PROP, batch, BATCH_SCRIPT_PROP, CPU_WAIT_PROP, EXPECTED_FINISH, JOB_ID_PROP, JOB_NAME_PROP, NODE_COUNT_PROP, PARTITION_PROP, PROC_COUNT_PROP, QUEUE_PROP, REQUEST_ACCURACY_PROP, REQUESTED_CPUS_PROP, REQUESTED_WALLCLOCK_PROP, RESIDENCY_PROP, SERIAL_PROP, SLOWDOWN_PROP, SUBJOB_PROP, SUBMITTED_PROP, SUBMITTED_TIMESTAMP, SUCCESS_PROP, TIME_PROP, WAIT_PROP, WALLCLOCK_PROP

Fields inherited from class uk.ac.ed.epcc.safe.accounting.update.AbstractPropertyContainerParser

DUPLICATE_KEY

Constructor Summary

Constructors

Constructor and Description
AbstractPbsParser(uk.ac.ed.epcc.webapp.AppContext context) Constructs a new PbsParser.

Method Summary

Methods

Modifier and Type	Method and Description
java.lang.String	endParse() End a batch parse this de-allocates any temporary storage and and perform any final operations.
uk.ac.ed.epcc.webapp.AppContext	getContext()
PropExpressionMap	getDerivedProperties(PropExpressionMap previous) return a set of derived property definitions.
protected ContainerEntryMaker	getEntryMaker(java.lang.String attr)
PropertyFinder	initFinder(uk.ac.ed.epcc.webapp.AppContext ctx, PropertyFinder prev, java.lang.String mode) Generate a PropertyFinder that will find any PropertyTag generated by this class.
boolean	parse(PropertyMap pbsMap, java.lang.String record) Parse a String representation of the UsageRecord.
void	startParse(PropertyContainer defaults) Start a batch parse.

Methods inherited from class uk.ac.ed.epcc.safe.accounting.update.BaseParser

readDouble, readInteger, readLong, readTime

Methods inherited from class uk.ac.ed.epcc.safe.accounting.update.AbstractPropertyContainerParser

getDefaultUniqueProperties, modifyDefaultTableSpecification, splitRecords, unique

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface uk.ac.ed.epcc.safe.accounting.update.PropertyContainerParser

getDefaultUniqueProperties, splitRecords

Methods inherited from interface uk.ac.ed.epcc.safe.accounting.update.PropertyContainerUpdater

modifyDefaultTableSpecification

Field Detail

PBS_ENDED_PROP

@AutoTable(unique=true)
public static final PropertyTag<java.util.Date> PBS_ENDED_PROP

PBS_EXIT_PROP

@AutoTable
public static final PropertyTag<java.lang.Integer> PBS_EXIT_PROP

PBS_GROUP_PROP

@AutoTable(length=128)
public static final PropertyTag<java.lang.String> PBS_GROUP_PROP

PBS_STARTED_PROP

@AutoTable
public static final PropertyTag<java.util.Date> PBS_STARTED_PROP

PBS_USER_PROP

@AutoTable
public static final PropertyTag<java.lang.String> PBS_USER_PROP

PBS_ACCOUNT_PROP

@AutoTable(length=128)
public static final PropertyTag<java.lang.String> PBS_ACCOUNT_PROP

PBS_ID_STRING_PROP

@AutoTable(unique=true,
           length=128)
public static final PropertyTag<java.lang.String> PBS_ID_STRING_PROP

PBS_JOB_NAME_PROP

@AutoTable(length=128)
public static final PropertyTag<java.lang.String> PBS_JOB_NAME_PROP

PBS_QUEUE_PROP

@AutoTable
public static final PropertyTag<java.lang.String> PBS_QUEUE_PROP

PBS_SUBMITTED_PROP

@AutoTable
public static final PropertyTag<java.util.Date> PBS_SUBMITTED_PROP

PBS_ACCOUNTING_ID_PROP

@AutoTable
public static final PropertyTag<java.lang.String> PBS_ACCOUNTING_ID_PROP

PBS_ALTERNATE_ID_PROP

@OptionalTable
public static final PropertyTag<java.lang.String> PBS_ALTERNATE_ID_PROP

PBS_AUTHORIZED_USERS_PROP

@OptionalTable(length=512)
public static final PropertyTag<java.lang.String> PBS_AUTHORIZED_USERS_PROP

PBS_AUTHORIZED_GROUPS_PROP

@OptionalTable(length=512)
public static final PropertyTag<java.lang.String> PBS_AUTHORIZED_GROUPS_PROP

PBS_AUTHORIZED_HOSTS_PROP

@OptionalTable(length=1024)
public static final PropertyTag<java.lang.String> PBS_AUTHORIZED_HOSTS_PROP

PBS_ELIGIBLE_TIME_PROP

@OptionalTable
public static final PropertyTag<java.util.Date> PBS_ELIGIBLE_TIME_PROP

PBS_ENTERED_QUEUE_PROP

@OptionalTable
public static final PropertyTag<java.util.Date> PBS_ENTERED_QUEUE_PROP

PBS_EXEC_HOST_PROP

@OptionalTable(length=1024)
public static final PropertyTag<java.lang.String> PBS_EXEC_HOST_PROP

PBS_JOB_TIMESTAMP_PROP

@OptionalTable
public static final PropertyTag<java.util.Date> PBS_JOB_TIMESTAMP_PROP

PBS_OWNER_PROP

@OptionalTable(length=128)
public static final PropertyTag<java.lang.String> PBS_OWNER_PROP

PBS_RECORD_TYPE_PROP

@OptionalTable(length=1)
public static final PropertyTag<java.lang.String> PBS_RECORD_TYPE_PROP

PBS_RESERVATION_DURATION_PROP

@OptionalTable(target=java.lang.Long.class)
public static final PropertyTag<java.lang.Number> PBS_RESERVATION_DURATION_PROP

PBS_RESERVATION_NAME_PROP

@OptionalTable
public static final PropertyTag<java.lang.String> PBS_RESERVATION_NAME_PROP

PBS_RESERVATION_ID_PROP

public static final PropertyTag<java.lang.String> PBS_RESERVATION_ID_PROP

PBS_SESSION_PROP

@OptionalTable
public static final PropertyTag<java.lang.Number> PBS_SESSION_PROP

PBS_TIME_USED_PROP

@AutoTable
public static final PropertyTag<java.lang.Number> PBS_TIME_USED_PROP

Constructor Detail

AbstractPbsParser

public AbstractPbsParser(uk.ac.ed.epcc.webapp.AppContext context)

Constructs a new PbsParser. The parser should not be used until it's initFinder(AppContext, PropertyFinder, String) method has been called.

Parameters:

context - The context in which this parser is operating

Method Detail

getContext

public uk.ac.ed.epcc.webapp.AppContext getContext()

Specified by:

getContext in interface uk.ac.ed.epcc.webapp.Contexed

endParse

public java.lang.String endParse()

Description copied from interface: PropertyContainerUpdater

End a batch parse this de-allocates any temporary storage and and perform any final operations.

Specified by:

endParse in interface PropertyContainerUpdater

Overrides:

endParse in class AbstractPropertyContainerUpdater

Returns:

Error string if any

getDerivedProperties

public PropExpressionMap getDerivedProperties(PropExpressionMap previous)

Description copied from interface: PropertyContainerUpdater

return a set of derived property definitions. Each derived property is defined in terms of a simple expression over other properties (that may themselves be derived properties).

Definitions are allowed for properties that are also generated directly in the parse phase. In this case the parse value should always be used by preference and the derived definition only provides a fall-back implementation if the parse value is unavailable. For example if not persisted in the database layer. This allows one parser/policy to specify a default implementation as a derived property definition which is then overridden by having the same property generated in a parse method of a different policy/parser

Specified by:

getDerivedProperties in interface PropertyContainerUpdater

Overrides:

getDerivedProperties in class BatchParser

Parameters:

previous - Previous definitions

Returns:

Map modified map

initFinder

public PropertyFinder initFinder(uk.ac.ed.epcc.webapp.AppContext ctx,
                        PropertyFinder prev,
                        java.lang.String mode)

Description copied from interface: PropertyContainerUpdater

Generate a PropertyFinder that will find any PropertyTag generated by this class. This call should be treated as a secondary constructor and should be called before any other method from this interface.

All properties defined at run-time should have their PropertyTag objects constructed by the time the first call to this method returns. As some parsers properties will generate different things depending on the other properties already in scope this method is passed a PropertyFinder for any properties that may already be in scope. This will include any default parameters specified when the data was uploaded and any properties generated from the database meta-data. Though it is perfectly legal to return a PropertyFinder that returns a superset of the generated properties the only requirement is that the returned finder can find any property that this classes parse method may actually generate values for. So Normally the contents of the super-finder are not included in the result unless these values are actually generated by the parse. If values are generated then a MultiFinder can be populated with the PropertyRegistrys from the generated properties

Specified by:

initFinder in interface PropertyContainerUpdater

Overrides:

initFinder in class BaseParser

prev - PropertyFinder for the higher levels in the call chain.

mode - String name of the destination table in case per-table customisation is needed

Returns:

PropertyFinder or null

startParse

public void startParse(PropertyContainer defaults)

Description copied from interface: PropertyContainerUpdater

Start a batch parse. This allocates any temporary state appropriate to the policy. Some properties may be set globally for all records in the parse rather than provided record by record. The startParse method is passed all known properties of this type as they may be useful in setting up the parse. There is no requirement to process these in any way.

Specified by:

startParse in interface PropertyContainerUpdater

Overrides:

startParse in class AbstractPropertyContainerUpdater

Parameters:

defaults - properties constant for this accounting run.

getEntryMaker

protected ContainerEntryMaker getEntryMaker(java.lang.String attr)

parse

public boolean parse(PropertyMap pbsMap,
            java.lang.String record)
              throws AccountingParseException

Description copied from interface: PropertyContainerParser

Parse a String representation of the UsageRecord. If the parse needs to be aborted throw and AccountingParseException. If the parse cannot succeed but no error needs to be reported (e.g. an empty or comment line) this method can return false

Specified by:

parse in interface PropertyContainerParser

Throws:

AccountingParseException

Parameters:

pbsMap - PropertyMap to add quantities

Returns:

true if parse ok