A parser for parsing OGF usage records. This type of usage record is defined in Open Grid Forum Specification GFD.98. The current version of the specification can be found at http://www.ogf.org/documents/GFD.98.pdf.

Deviations from the Specification

There are points where the specification contradicts itself. For example, section 12.8.1 states that the TimeInstant element may have a description attribute, however the provided schema (appendix D) does not specify such an attribute for TimeInstant. In accordance with section 7.3.1 of the specification, whenever there is a contradiction this parser follows the structure defined in the schema in favour of the structure defined elsewhere in the specification.

This parser is unable to distinguish between elements with the same qualified name and attributes with identical values. This is not an issue for differentiated properties as the specification states that such elements can only appear with different attribute values (e.g. <memory> and <memory metric="average"> may appear in the same usage record, but <memory metric="average"> cannot appear twice). The only element that may cause a problem is <ProcessId>. This element is allowed to appear more than once with no differentiating attribute. This parser will not be able to distinguish between the occurrences of the element and will overwrite it's value for the element every time a new element is encountered. This problem could potentially occur for extension elements too, however if multiple occurrences of an extension element can be differentiated by one of it's attributes, this will not be a problem.

Default Accounting Properties

This parser will parse values it encounters into a set of predefined accounting properties. Each default accounting property will only be present if it's associated element is present in a usage record. A list of all default accounting properties can be found at the end of this document. Other custom accounting properties can be defined. This done by setting the appropriate configuration properties. Custom accounting properties will take priority over default accounting properties with the same name. However, the default accounting properties can be referenced by prefixing the accounting property name with the appropriate property registry name in the usual manner.

Terminology

UR elements

A UR element is an XML element that may appear in an OGF usage record. All UR elements that are defined in the OGF Usage Record specification have been predefined and do not need to be explicitly declared. These definitions may be overwritten and new custom UR elements may be defined using configuration properties.

UR attributes

A UR attribute is an XML attribute that may appear in an OGF usage record. All UR attributes that are defined in the OGF Usage Record specification have been predefined and do not need to be explicitly declared. These definitions may be overwritten and new custom UR attributes may be defined using configuration properties specified below. There is no need to define new UR attributes unless they are to be used in new UR elements (or added to existing UR elements).

UR Properties

A UR Property is a property that may be extracted from an OGF usage record. The UR property has a name which it uses to create an AccountingProperty which is stored in this manager's property registry and is used as a key to values the UR property extracts. UR properties extract values from UR elements or UR attributes. Each one must be explicitly defined using the configuration properties specified below. A UR property will extract a value from the UR element it is associated with. The value comes either from the contents of the UR element, or the value of one of it's attributes.

A UR element may appear more than once in an OGF usage record. To avoid ambiguity, UR properties may place constraints on the UR element it extracts a value from. For example, it may specify that a UR element instance must contain a particular attribute. Furthermore, it may define what that attribute's value must be. UR properties are not required to place these constraints, however if the same UR element occurs more than once without constraints being used to inform a UR property which configuration of the UR element to use to extract values from, it may result in the parsed value of that UR property being overwritten by the second occurrence of the UR element. The order of elements in an OGF usage record are allowed to vary which means this could produce unpredictable results.

Configuration Properties

The OGF usage record parser already maps all attribute and elements that can be found in an OGF usage record onto UR elements and UR attributes. However, the specification allows for extension elements to be added. For this reason, new UR elements and UR attributes may be defined. Existing UR elements and UR attributes may also be reconfigured. All UR properties must be defined as well.

In the definition of these properties, namespace prefixes are used to denote XML namespaces at various points. Properties defining these namespaces are also processed by this object. The configuration properties that may be specified are described in the following subsections.

All configuration properties use the name of the table that they are parsing records into. Configuration properties that feature the table name will take precedence over ones that don't. All configuration property keys start with the string ogf.ur. properties starting with the string ogf.ur.mode will take precedence over properties that do not. So for example, consider the two properties:

ogf.ur.OGFRecord.namespace.ex = http://example.com
ogf.ur.namespace.ex = http://www.example.com

When an OGF usage record parser is parsing records into the table called OGFRecord, the ex namespace prefix will map to http://example.com. However, when a OGF usage record parser is parsing records into another table, ex namespace prefix will map to http://www.example.com. For this reason, tables an OGF usage record parser parses records into can never take the name attribute, attributes, element, elements, namespace or property.

General Properties

This parser has one general property that configures how it behaves when it encounters a value in an element that it does not understand. The event will be logged, however the parser may continue parsing or reject the entire record. This is set using the property

ogf.ur.fail-on-bad-value = boolean-value

If no present, this property defaults to false.The acceptable values for the property are true, false or a number - 0 representing false and a non-zero value representing true. If set to true, Only records that have had all their values successfully parsed will be stored in the database. If set to false, records that have uninterpreted values will be stored in the database with null values used where the appropriate ones couldn't be parsed.

Namespace Properties

Many of the properties need to define namespaces at some point. Namespaces can contain many different types of characters which makes parsing complicated expressions containing namespaces difficult. As a result, properties requiring namespaces must use namespace prefixes. The mapping of namespace prefix to actual namespace is done using the property

ogf.ur.namespace.prefix-name = namespace 

for example:

ogf.ur.namespace.urf = http://schema.ogf.org/urf/2003/09/urf 

The above example demonstrates mapping the prefix urf to the namespace http://schema.ogf.org/urf/2003/09/urf. This particular mapping is actually not required because most properties allow a namespace prefix to be omitted, and if it is omitted they assume the namespace http://schema.ogf.org/urf/2003/09/urf. However, it is better practice to define all namespace prefixes and always use them where appropriate.

It is possible to map more than one prefix to the same namespace. However, when reading custom UR element configuration properties, prefixes are not dereferenced which may lead to custom UR elements not being defined properly. For this reason it is recommended that only one prefix is used per namespace declaration.

UR Property Configuration Properties

UR properties are essentially containers for accounting properties!. They associate an accounting property with a UR element. They may also place extra constraints on the presence and value of attributes a UR element can have before it is used to extract a value from. The accounting property will reside in the ogfur_ext PropertyRegistry and have the same name as the UR property. The UR property is defined using the configuration property:

org.ur.property.tag-name = namespace:ur-element-name[constraint][constraint2]#attribute-name 

The UR property and the AccountingProperty it will use to mark values takes the name tag-name. The UR element associated with it is specified with a declaration that takes the following format:

namespace

(optional) specifies the namespace (denoted by prefix) of the element. The prefix must have already been defined using a namespace declaration property. If it is absent, the colon (:) separating namespace form UR element name must also be omitted and the OGF usage record namespace is assumed (http://schema.ogf.org/urf/2003/09/urf).

UR-element-name

(required) the name of the element.

constraint

(optional) zero or more constraints contained within square brackets ('[' and ']').

attribute-name

(optional) indicates the UR property should take the value of the attribute whose name is specified by. attribute-name. If this is not present, the fragment separator (#) must be omitted form the declaration.

If the tag name of the new UR property is the same as one of the predefined accounting properties, the predefined UR property is removed from the set of possible UR properties that can appear in a record.

UR Element Constraints

Constraint may be optionally specified. They come directly after the element name and before the attribute name (if present). Each constraint should be contained within square brackets ('[' and ']'). [attr=value] asserts that the attribute called attr must have value value. If the attribute's name is present without an equals sign (e.g. [attr]), the attribute has to be present but it's value doesn't matter. To assert that an attribute must not be present, assert the attribute has no value, e.g. [attr=].

Warning: The attributes of some UR elements have default values. For example, the default value of the metric attribute of Disk is total. This value will be inserted if one wasn't specifically provided. For example, consider this configuration property:

ogf.ur.property.mem = Memory[metric=]

Now consider what would happen when this record is parsed:

<urf:UsageRecord xmlns='http://schema.ogf.org/urf/2003/09/urf' xmlns:urf='http://schema.ogf.org/urf/2003/09/urf'>
  <urf:RecordIdentity urf:createTime="2003-08-15T14:25:56Z" urf:recordId="example:record:123"/>
  <urf:Status>Complete</Status>
  <urf:Memory urf:storageUnit='MB'>10</urf:Memory>
</urf:UsageRecord>

The mem accounting property will not contain a value because the default value of the metric attribtute of memory is total. To extract the value of the UR elementurf:Memory into accounting property mem, one needs to define mem in the following way:

ogf.ur.property.mem = Memory[metric=total]

or, simply

ogf.ur.property.mem = Memory

Examples

ogf.ur.namespace.urf = http://schema.ogf.org/urf/2003/09/urf
ogf.ur.property.MaxMem = urf:Memory[metric=max]
ogf.ur.property.MaxMemUnit = urf:Memory[metric=max]#storageUnit

In the above example, the urf: part could be omitted as the default namespace is assumed to be http://schema.ogf.org/urf/2003/09/urf. However, it is recommenced that it is included for clarity. The namespace prefix mapping for urf is not assumed so this must be explicitly expressed. The above example creates a UR property called MaxMem will extract a value from the OGF usage element Memory when Memory contains the attribute metric and metric has a value of max. A UR property called MaxMemUnit will extract a value from the attribute storageUnit contained in the OGF usage element Memory when Memory contains the attribute metric and metric has a value of max.

Custom UR Element Configuration Properties

All OGF Usage Record elements defined in the specification have already been predefined and can be used by name to assign values to UR properties. However, users may wish to define new custom elements that can be parsed. This can be one with UR element configuration properties.

Note: elements defined already cannot be redefined. Attempting to do so will result in an error message being logged. The parse will continue but will use the original definition.

To define new UR elements, use the configuration property:

ogf.ur.elements = comma-separated-element-qualified-name-list 

For example

ogf.ur.namespace.saf = http://epcc.ed.ac.uk/gridsafe
ogf.ur.elements = saf:aCustomElement,saf:anotherElement, aThirdElement

The list is comma separated. Leading and trailing whitespace is removed from each element name. Names can be qualified in the usual way. If they are, the namespace prefix must be defined too. If the prefix is omitted (as in aThirdElement for example) the namespace http://schema.ogf.org/urf/2003/09/urf is assumed (OGF usage record namespace). If a predefined element is named here, in error message being logged. The parse will continue but will use the original definition. The elements are assumed to have string types and have no attributes. Attributes may be present in the records that are processed but their values cannot be assigned to UR Properties and cannot be used in constraints. The declared elements can be customised further with the following properties:

ogf.ur.element.qualified-element-name.type = type
ogf.ur.element.qualified-element-name.attributes = comma-separated-attribute-list

for example

ogf.ur.element.saf:aCustomElement.type = integer
ogf.ur.element.saf:aCustomElement.attributes = type,metric:average

The name used to denote the UR element being configured must be exactly the same as the name used in the ogf.ur.elements property. If a namespace prefix was used, exactly the same prefix must be used here to. Another namespace prefix cannot be used even if it maps to an identical namespace. Likewise, if a namespace prefix was omitted in the ogf.ur.elements property, it cannot be used in a ogf.ur.element property.

The type can be any type defined by the ValueParserService (either a default type or one that has been added or re-defined). The attribute list is comma separated with leading and trailing whitespace removed form each element. The attribute must either be one of the predefined UR attributes, or one added using one of the UR Attribute configuration properties. If the colon character (:) is present in the attribute name, the colon is removed and the string following it is used as the default value of the attribute should one not be set in an OGF usage record. For example, if a saf:aCustomElement element in a record does not contain a metric attribute, it is added anyway with the value average.

Issues With Java Properties

Be careful when defining UR element configuration properties. A UR element definition can (and should) contain the namespace prefix, however the colon character has a special meaning in Java properties files. Therefore, if using Java properties files, the colon must be escaped (e.g. ogf.ur.element.saf\:aCustomElement.attributes). If the property is being defined in another way (through a web interface or direct insertion into the database for example) the backslash escape character is not needed.

Custom UR Attribute Configuration Properties

As with UR elements, all OGF Usage Record attributes defined in the specification have already been predefined and can be used by name to assign values to UR properties. However, users may wish to define new custom UR attributes that can be parsed. This can be one with UR attribute configuration properties. Unlike UR elements, the namespace of UR attributes cannot be defined. The namespace of an attribute is always ignored.

Note: attributes defined already cannot be redefined. Attempting to do so will result in an error message being logged. The parse will continue but will use the original definition.

To define new UR attributes, use the configuration property:

ogf.ur.attributes = comma-separated-attribute-name-list 

For example

ogf.ur.attributes = newAttr,attr2

The list is comm separated. Leading and trailing whitespace is removed from each name. The attributes are assumed to hold string values. The declared attributes can be customised further with the following properties:

ogf.ur.attribute.attribute-name.type = type

The type can be any type defined by the ValueParserService (either a default type or one that has been added or re-defined).

Predefined Accounting Properties

All of these accounting properties are part of the ogfur property registry. The accounting property will only be present if it's associated element or attribute is present in the accounting record. If the accounting property is associated with an element, the contents of the element is taken as it's value (regardless of whether the element contains simple or complex content).

Default Record Identity Properties

Default Job Identity Properties

Default User Identity Properties

Default Base Properties

Default Derived Properties

Default Extension Properties

These are defined in the OGF-UR specification. as examples of extension properties.

Predefined UR Attributes

All parsers listed come from the package uk.ac.ed.epcc.safe.accounting.parsers.value.

Predefined UR Elements

All parsers listed come from the package uk.ac.ed.epcc.safe.accounting.parsers.value. Some attributes have default values. This means that if value for the attribute is not present in the usage record, the default values is used. The default values used here are the same as the ones declared in the GFD.98 OGF Usage Record specification.