BlackPearl acts as a caching gateway in front of Spectra Logic’s tape libraries. Typical client applications will both send groups of files to (PUT) and request groups of files from (GET) BlackPearl. BlackPearl uses “Jobs” to contain and keep track of these individual input/output operations.
BlackPearl is capable of managing many jobs at once. Jobs have a selectable “Priority” for processing the job so that client applications can have some control over the resources assigned to each job. The job priority settings are only relevant when the cache or tape drive resources are constrained, in which case the BlackPearl is said to be “throttled”. If there are no resource constraints, then all jobs will be processed equally.
Files moved by a job are broken into one or more “chunks” for processing. Cache space is required to store each chunk. The job priority determines how many chunks a job can use at any one time. The more chunk cache spaces that are available for a job, the more chunks that can be uploaded or downloaded by the job at any one time. Job priority can also affect how many tape drives a job can utilize at any one time. It takes at least two chunks to efficiently and continuously feed a tape drive.
Job Priority Values & Chunk Allocation
Jobs can be set by client applications to have one of the following priority. For chunk allocation, these priorities are only applicable if the BlackPearl is throttled.
- Low – Low priority jobs get a maximum of four chunk cache spaces at any one time.
- Normal -- (default for PUT) Normal priority jobs get a maximum chunk cache space at any one time of either: (a) eight or (b) two times the number of tape drives, whichever is greater.
- High -- (default for GET) High priority jobs get a maximum chunk cache space at any one time of either: (a) sixteen or (b) three times the number of tape drives, whichever is greater.
- Urgent – Urgent priority jobs get special prioritization. An Urgent job is exempt from any maximum limitations. It can use all available chunk cache space that it requests.
When jobs are requesting cache space for their chunks, and there are only a limited number of spaces available, the job that asks for the spaces first will get them, but no more than the maximums described above.
BlackPearl can also create its own system jobs with Critical and Background priority. These priority values are not available for jobs created by client applications, but you may see them on the Jobs Status Screen on the BlackPearl web interface.
- Critical – The job must be executed immediately and cannot wait. This is typically used for tape drive cleaning operations.
- Background – The job can be done when resources are available. This is typically used for tape inspection and reclamation.
Prioritization for Tape Read/Write Operations
The prioritization for tape drive read and write operations for chunks works in this order of preference:
- Is the priority of the chunk “Urgent”? If yes, goes before all others (this prioritization preference introduced BlackPearl 3.3)
- Can the chunk use a tape that’s already in a drive? If yes, it goes before a chunk that can’t.
- Is the chunk’s priority higher (excluding Urgent priority)? If yes, it goes before lower priority chunks.
- Can the chunk proceed without allocating another tape to itself? If yes, it goes before chunks requiring another tape.
- Has the chunk been waiting longer in queue? If yes, it goes before newer chunks.
Note that no job, not even an Urgent priority job, can stop or kill other active tape read or write operations.
Other Ways to Improve Job Performance
If the goal is to improve performance on a PUT operation, then the job’s Write Optimization can be set to “Performance” instead of the default “Capacity”. Using performance means that the job data may be written to more tapes simultaneously, which will allow for faster write performance.
The BlackPearl cache can also be set to its “Performance” configuration rather than its default “Capacity” configuration. This will significantly improve the performance for all jobs. See the “Configure the Cache” section of the BlackPearl User Guide.