Concurrent Processing Framework

Plug-in Overview

A CPF Plug-in module is a Maven project that contains one or more business application plug-ins. A business application plug-in is a service that takes one or more request parameters, performs an action using the request parameters and generates one or more result attributes. The CPF allows a user to create a job that contains one or more requests to be executed against a single business application. For example a job may contain 1 million addresses to geocode.

For an example a power service would have the request parameters base and exponent and the result attribute result, which is the calculation of base to the power of exponent.

For input parameters a plug-in can accept either one or more structured request attributes or a per request input data with an InputStream containing an opaque blob of data.

• Structured request attributes can be thought of as a key value pair map of values. The users of the plug-in can specify a single file (e.g. CSV, JSON) with a record containing the values for each request to be executed by the plug-in. For example each address to geocode would have one record in the input file and the new instance plug-in will be executed for each record. In addition to request level parameters there are also job parameters that apply to all requests in the job. The CPF will convert the input data from the file format specified by the user to Java objects for use by the plug-in.
• Per request input data is used where the data is more complex than key value pairs, or is a binary blob of data. Each request is specified using a separate file stream or URL to the binary blob of data. For example a face detection plug-in would take a JPEG file as an input parameter for each face to recognize. Per request input data plug-ins cannot also have request attributes, although they may have job attributes. If structured request attributes and binary blobs are required for the same plug-in the plug-in can accept a URL to the input data and process that URL within the plug-in.

For result fields the plug-in can either return a single record of structured result data, a list of structured result data or a single binary blob of data.

• Like structured request attributes structured result attributes are a key value pairs. The CPF will create a single file in the requested output file for the user to download with one record for each result returned from the plug-in.
• The list of structured results also creates a single output file for the user to download. There will be one record for each entry in the list of results, with a relative result number showing the order of the result in the list of results.
• Like per request input data, per request result data returns one binary blob of data for each request. There will be one file for the user to download for each request. For example a WMS service would return a JPEG, or PNG for each map to render.

API Documentation and Download

The most recent version of the CPF is ${project.version}. For development purposes the most recent snapshot version of the CPF libraries and the trunk sub-tree in the subversion repository should be used. The snapshot and the trunk are only updated under controlled circumstances when new functionality or bug fixes are to be delivered to the plug-in developers. When delivering a final version of the application to the Ministry the plug-in must use the same version as is deployed to the Ministry’s server.

The CPF and plug-ins depend heavily on the Spring Framework and the XML configuration mechanism. Developers should have a least a basic understanding of defining spring beans and dependency injection.

Use the links in the following table to view the Plug-in API documentation or download the API libraries.

API	Documentation	Download
Plug-in Java (6+)	API Docs	Download