1-800-833-1132 : 1-303-449-6400


Spectra BlackPearl Converged Storage System : Spectra S3 Bucket, Object, and Job Operations : Job Operations : Create Bulk PUT

Create Bulk PUT
Description
Create a job to stream PUT object requests. Clients should use this before putting objects to physical data stores. See Processing a Bulk PUT Job for an overview of the process.
When this request is issued, an idle timer starts and the reservation expires after 24 hours elapse. The timer is reset by performing an IO operation (such as a `PUT Object` operation) or by issuing a Modify Job request (see Modify Job), even if you do not modify anything.
*
Requests
Syntax
PUT http[s]://{datapathDNSname}/_rest_/bucket/{bucket UUID or name}?operation=START_BULK_PUT[&aggregating=TRUE|FALSE]​[&force][&ignore_naming_conflicts]​[&implicit_job_id_resolution=TRUE|FALSE]​[&max_upload_size={64‑bit integer}][&minimize_spanning_across_media=TRUE|FALSE][&name={string}]​[&priority=URGENT|HIGH|NORMAL|LOW]
Request Parameters
Whether or not to aggregate jobs. When selected, if additional PUT jobs using the same bucket and chunk_client_processing_order_guarantee are created within 30 minutes of the initial job creation, they are appended to the initial job to create a larger job.
Values: TRUE, FALSE (default)
Notes:  
Values: TRUE, FALSE (default)
Values: TRUE, FALSE (default)
The maximum size for any object part transferred. The maximum and default is 100 GB. Only specify a smaller max_upload_size if the network connection cannot reliably handle large uploads.
Notes:  
Values: TRUE, FALSE (default)
The priority for processing this job. The job priority determines the assigned resources and processing order. Jobs with Priority URGENT can use up all of the resources and prevent other jobs from making progress. Use this priority sparingly.
Values: URGENT, HIGH, NORMAL, LOW
Request Elements
An XML payload must be sent to describe the PUT job to create, formatted as follows:
<Objects
   WriteOptimization="CAPACITY|PERFORMANCE">
   <Object Name="{
string}" Length="{64‑bit integer}"/>
   <Object Name="{string}" Length="{64‑bit integer}"/>
</Objects>
where the parameters are defined as follows:
Specifies whether job chunks are written as quickly as possible or across as few pieces of media as possible. When CAPACITY mode is used for writing to tape, job chunks are written concurrently to multiple tapes using multiple drives if the tapes being written to are already assigned to the bucket, or there is so much data to write that the system knows multiple tapes are needed to write all of the data. When in PERFORMANCE mode, job chunks are written as quickly as possible, even if that means that more tapes are allocated to the bucket than are necessary to write the data.
It is better to use CAPACITY mode if the tapes will be ejected after the job completes or if the bucket is written to very rarely and capacity in the library is of concern. PERFORMANCE mode is recommended in all other cases.
Pools should generally have a CAPACITY write optimization since pools are very fast and under less contention. It is rare for a pool storage domain to benefit from a PERFORMANCE write optimization.
Values: CAPACITY, PERFORMANCE
Responses
Response Elements
<MasterObjectList
   Aggregating="TRUE|FALSE"
   BucketName="{
string}"
   CachedSizeInBytes="{64‑bit integer}"
   ChunkClientProcessingOrderGuarantee="IN_ORDER|NONE"
   CompletedSizeInBytes="{64‑bit integer}"
   EntirelyInCache="TRUE|FALSE"
   JobId="{string}"
   Naked="TRUE|FALSE"
   Name="{string}"
   OriginalSizeInBytes="{64‑bit integer}"
   Priority="CRITICAL|URGENT|HIGH|NORMAL|LOW|BACKGROUND"
   RequestType="PUT"
   StartDate="YYYY-MM-DDThh:mm:ss.xxxZ"
   Status="IN_PROGRESS|COMPLETED|CANCELED"
   UserId="{string}"
   UserName="{string}">
   <Nodes>
      <Node EndPoint="{string}" Id="{string}"/>
   </Nodes>
   <Objects
      ChunkId="{string}"
      ChunkNumber="{32‑bit integer}">
      <Object Id="{string}" InCache="TRUE|FALSE"
         Latest="TRUE|FALSE" Length="{64‑bit integer}"
         Name="{string}" Offset="{64‑bit integer}"
         VersionId="{32‑bit integer}"/>
      ...
   </Objects>
   ...
</MasterObjectList>
where the response elements are defined as follows:
Whether the job can have additional PUTs or GETs appended to it. Jobs aggregate if created with the aggregating request parameter set to TRUE, or if created by the BlackPearl system in response to Amazon S3 PUT and GET requests. Values: TRUE, FALSE
The amount of data that has been transferred to the cache for this job. For RequestType=PUT, this is the amount of data successfully transferred to the BlackPearl Converged Storage System from the client.
The amount of data that is completely processed for this job. For RequestType=PUT, this indicates the amount of data written to all permanent data stores.
Whether the job was created as the result of a native S3 command (TRUE) or a Create Bulk GET/PUT/VERIFY job command (FALSE). Values: TRUE, FALSE
§
IN_PROGRESS — The job is currently running.
§
COMPLETED — The job completed.
§
CANCELED — The job was canceled by the user or automatically due to internal timeouts.
Example
Sample Request
This request creates a job to PUT two objects into the bucket named “bucket1”.
PUT http://blackpearl-hostname/_rest_/bucket/bucket1?operation=start_bulk_put​​&name="PutJob" HTTP/1.1
<Objects
   WriteOptimization="CAPACITY">
   <Object Name="test.aaf" Length="107426611200"/>
   <Object Name="T950.tif" Length="9572972"/>
</Objects>
Sample Response
HTTP/1.1 200 OK
<MasterObjectList
   Aggregating="FALSE"
   BucketName="bucket1" CachedSizeInBytes="0"
   ChunkClientProcessingOrderGuarantee="NONE"
   CompletedSizeInBytes="0"
   EntirelyInCache="FALSE"
   JobId="823f7f21-8af4-44fd-b0f0-70c5a0ab52aa"
   Naked="FALSE"
   Name="PutJob"
   OriginalSizeInBytes="0" Priority="LOW" RequestType="PUT"
   StartDate="2015-10-07T22:33:22.000Z" Status="IN_PROGRESS"
   UserId="b332af88-0130-4e4d-bbb5-fac6f3ec1970" UserName="user1">
   <Nodes>
      <Node EndPoint="blackpearl-hostname"
      Id="a3f5c59e-6bdd-457d-9dde-0ffcf7236c7a"/>
   </Nodes>
   <Objects ChunkId="2e0353e1-96d7-42a5-a507-d60f61c28d6e"
      ChunkNumber="1">
      <Object Id="2ee4f397-c169-4a59-96cc-a07280412f43"
         InCache="FALSE" Latest="TRUE" Length="10"
         Name="o2" Offset="0" Version=1"/>
   </Objects>
   <Objects ChunkId="e8d5e8b1-2e15-45a5-8b2d-e48a59114a40"
      ChunkNumber="2">
      <Object Id="ce6f4fef-2aa0-49fd-b9a3-1f0bf51e7274"
         InCache="FALSE" Latest="TRUE" Length="10"
         Name="o1" Offset="0" Version=1"/>
   </Objects>
</MasterObjectList>