Spectra Logic

1-800-833-1132 : 1-303-449-6400

Create VERIFY Job

Description

Create a job to verify objects. A VERIFY job reads data from the permanent data store and verifies that the CRC of the data read matches the expected CRC. VERIFY jobs ALWAYS read from the data store - even if the data currently resides in cache.

The BlackPearl system will process a maximum of 1,000 concurrent active jobs.

Requests

Syntax

PUT http[s]://{datapathDNSname}/_rest_/bucket/{bucket UUID or name}?operation=START_BULK_VERIFY[&aggregating=TRUE|FALSE]​[&name={string}][&priority=URGENT|HIGH|NORMAL|LOW]

Request Parameters

Parameter	Description	Required
operation	The operation to perform. For this request, the operation is to start a bulk VERIFY. Value: START_BULK_VERIFY	yes
aggregating	Whether or not to aggregate jobs. When selected, if additional VERIFY jobs using the same bucket and chunk_client_processing_order_guarantee are created within 30 minutes of the initial job creation, they are appended to the initial job to create a larger job. Values: TRUE, FALSE (default) Notes:   § Do not use aggregating for jobs larger than 50 GB. § The client must be able to handle the changing job structure to use this feature. § Data is not written to tape storage domains until the 30 minute aggregation time is complete.	no
name	A name to assign to the job for tracking.	no
priority	The priority for processing this job. The job priority determines the assigned resources and processing order. Jobs with Priority URGENT can use up all of the resources and prevent other jobs from making progress. Use this priority sparingly. Verify jobs can be interrupted every 30 minutes if a job with a higher priority is received. Values: URGENT, HIGH, NORMAL, LOW Default: Configured in the data policy for the bucket.	no

Request Elements

An XML payload must be sent to describe the PUT job to create, formatted as follows:

<Objects
   <Object Name="{string}" Length="{64‑bit integer}"/>
   <Object Name="{string}" Length="{64‑bit integer}"/>
</Objects>

where the parameters are defined as follows:

Parameter	Description	Required
Objects	A container for the list of objects.	yes
Object	The container for information about one object.	yes
Name	The name of an object to verify. All objects in the list must be in the same bucket.	yes
Length	The length in bytes of the object.	yes

Responses

Response Elements

<MasterObjectList
   Aggregating="TRUE|FALSE"
   BucketName="{string}"
   CachedSizeInBytes="{64‑bit integer}"
   ChunkClientProcessingOrderGuarantee="IN_ORDER|NONE"
   CompletedSizeInBytes="{64‑bit integer}"
   EntirelyInCache="TRUE|FALSE"
   JobId="{string}"
   Naked="TRUE|FALSE"
   Name="{string}"
   OriginalSizeInBytes="{64‑bit integer}"
   Priority="CRITICAL|URGENT|HIGH|NORMAL|LOW|BACKGROUND"
   RequestType="VERIFY"
   StartDate="YYYY-MM-DDThh:mm:ss.xxxZ"
   Status="IN_PROGRESS|COMPLETED|CANCELED"
   UserId="{string}"
   UserName="{string}">
   <Nodes>
      <Node EndPoint="{string}" Id="{string}"/>
   </Nodes>
   <Objects
      ChunkId="{string}"
      ChunkNumber="{32‑bit integer}">
      <Object Id="{string}" InCache="TRUE|FALSE"
         Latest="TRUE|FALSE" Length="{64‑bit integer}"
         Name="{string}" Offset="{64‑bit integer}"
         Version="{64‑bit integer}"/>
      ...
   </Objects>
   ...
</MasterObjectList>

where the response elements are defined as follows:

Parameter	Description
MasterObjectList	The container for the response.
Aggregating	Whether the job can have additional PUTs or GETs appended to it. Jobs aggregate if created with the aggregating request parameter set to TRUE, or if created by the BlackPearl system in response to Amazon S3 PUT and GET requests. Values: TRUE, FALSE
BucketName	The name of the bucket that is acted on by the job request.
CachedSizeInBytes	The amount of data that has been transferred to the cache for this job. For RequestType=VERIFY, this is the amount of data loaded into cache from the permanent data store.
ChunkClient ProcessingOrder Guarantee	Specifies whether the job chunks will be processed in order. Values: IN_ORDER, NONE
CompletedSizeIn Bytes	The amount of data that is completely processed for this job. For RequestType=VERIFY, this indicates the amount of data with CRCs verified.
EntirelyInCache	Whether all objects in the job are in the BlackPearl cache. Values: TRUE, FALSE
JobId	The UUID for the job.
Naked	Whether the job was created as the result of a native S3 command (TRUE) or a Create Bulk GET/PUT/VERIFY job command (FALSE). Values: TRUE, FALSE
Name	The name assigned to the job for tracking.
OriginalSizeInBytes	The full size of the job.
Priority	The priority for processing this job. The job priority determines the assigned resources and processing order. Values: CRITICAL, URGENT, HIGH, NORMAL, LOW, BACKGROUND
RequestType	Type of job request. Values: VERIFY
StartDate	The date and time the job was started in the format YYYY-MM-DDThh:mm:ss.xxxZ.
Status	The current status of the job. Values: § IN_PROGRESS — The job is currently running. § COMPLETED — The job completed. § CANCELED — The job was canceled by the user or automatically due to internal timeouts.
UserId	The UUID for the user who initiated the job.
UserName	The username of the user who initiated the job.
Nodes	A container for information about all BlackPearl nodes.
Node	A container for information about a single BlackPearl node.
EndPoint	The IP address or DNS name of the BlackPearl node.
Id	The UUID for the node.
Objects	Container for information about the objects in one chunk.
ChunkId	The UUID for the job chunk.
ChunkNumber	The position of the chunk within the job.
Object	The container for information about a single object.
Id	The UUID for the object.
InCache	Indicates if the object is currently in cache on the BlackPearl Converged Storage System. Values: TRUE, FALSE
Latest	Whether this version of the object is the latest. Values: TRUE, FALSE
Length	The length in bytes of the object.
Name	The name of the object.
Offset	The offset in bytes from the start of the object.
Version	The version number of the object.

Example

Sample Request

This request creates a job to verify two objects in the bucket named “bucket1”.

PUT http://blackpearl-hostname/_rest_/bucket/bucket1​ ?operation=START_BULK_VERIFY&name="VerifyJob" HTTP/1.1

<Objects
   WriteOptimization="CAPACITY">
   <Object Name="test.aaf" Length="107426611200"/>
   <Object Name="T950.tif" Length="9572972"/>
</Objects>

Sample Response

HTTP/1.1 200 OK
<MasterObjectList
   Aggregating="FALSE"
   BucketName="bucket1"
   CachedSizeInBytes="0"
   ChunkClientProcessingOrderGuarantee="NONE"
   CompletedSizeInBytes="0"
   EntirelyInCache="FALSE"
   JobId="28f15f6d-3a14-4f97-b81d-8c2908030e7b"
   Naked="FALSE"
   Name="VerifyJob"
   OriginalSizeInBytes="0"
   Priority="LOW"
   RequestType="VERIFY"
   StartDate="2015-10-07T22:33:22.000Z"
   Status="IN_PROGRESS"
   UserId="9f7b5821-0f66-4bd5-a0e2-d5b944b49a82"
   UserName="user_name">
   <Nodes>
      <Node EndPoint="blackpearl-hostname"
         Id="f304e3d8-b167-4fd7-bdb2-7f91afab4549"/>
   </Nodes>
   <Objects ChunkId="4d09ce4c-8dc3-43e9-806e-25ac79a3d9b7"
      ChunkNumber="1">
      <Object Id="98e754ac-2015-4c90-bdb3-fc7414e0fde1"
         InCache="FALSE" Latest="TRUE" Length="10" Name="o1"
         Offset="10" Version=1"/>
   </Objects>
</MasterObjectList>