Spectra RioBroker Best Practices

Spectra Logic’s RioBroker is one method for integrating to BlackPearl. While the REST API of RioBroker is relatively straightforward, developers must thoughtfully build their RioBroker integration to ensure that the integration is complete, reliable, and fast. The following RioBroker design best practices should be followed to ensure that the integration is successful and can pass Spectra’s Certification Program.

Below we will refer to the software that integrates with RioBroker as the “client”.

Maximizing Transfer Rate

While RioBroker is responsible for moving the files to be archived or restored to BlackPearl, the client also plays an important part in ensuring fast file transfers. There are two areas where the client can contribute to maximizing transfer performance:

  • Include as many files into one command as possible. When issuing an individual archive or restore command, the client specifies one or more files to be transferred. Every effort should be made to include as many files as possible into each individual archive or restore command. This will allow RioBroker to group the files when interacting with BlackPearl, which works faster and more efficiently with groups of files. Please note though that RioBroker currently has a limit of 1,000 files per job, so do not specify more than 1,000 files in each archive or restore command.
  • Send transfer requests in parallel. RioBroker can process multiple archives and restore commands in parallel. Clients should send the archive and restore requests as soon as possible and in parallel to achieve maximum RioBroker transfer performance.

Jobs Management

In RioBroker, an archive or restore operation is called a “Job”. The status of an actively-running job can be determined with the following command: GET /api/jobs/{jobId}.  The job should be monitored by the client until the status of the job changes to “completed”.

Many clients will have a need to cancel actively running RioBroker jobs. This can be done using the following command: PUT /api/jobs/{jobId}. Be sure to set the “cancel” parameter to true. When canceling a RioBroker job, it is up to the client to determine which files have already been transferred in the archive or restore operation and then clean them up accordingly. When RioBroker receives a cancel command, it will cease further transfers, but it will not remove any files that were already transferred.

Session Management

In order to send API commands to RioBroker, the client must include a session authentication token. The authentication token can be acquired using the following command: POST /api/tokens. The client will also need to pass in the username and password parameters (spectra/spectra). The authentication token will expire in one hour, so the client will need to track the age of the token and request a new one every hour.

Use .NET SDK

Spectra Logic has provided a RioBroker .NET SDK for clients that are written in .NET. This SDK makes client integration easier and faster. We recommend using this SDK if at all possible. See our Documentation page for help on using this SDK.

Managing Unexpected Conditions

Job Failure Error

The status of a RioBroker transfer job (archive and restore) can be found using the following API command: GET /api/jobs/{jobId}. The job status returned will be one of the following: active, completed, paused, canceled, error. If the job status is “error”, the job has failed for some reason. The client needs to be able to handle an error status appropriately. Here are some of the reasons that an error could occur:

  • Files that the client is attempting to restore are only available on ejected tape (requires operator to insert tapes back into tape library)
  • BlackPearl is unavailable
  • BlackPearl storage target (tape, disk, cloud, replicated BlackPearl) is unavailable
  • The source storage system is unavailable
  • BlackPearl cache is full
  • Restore is taking too long
  • RioBroker does not have permissions to files on BlackPearl (bad S3 credentials)
  • RioBroker doesn’t have permissions to files on source storage (bad FTP or file system credentials)
  • The file does not exist and therefore cannot be moved
  • File already exists and therefore cannot be overwritten

Clients should be designed to handle the error status and display the status message to the user. In many cases, the message will include important information for the user. For example, if the files are only available on ejected tapes, the message will include the tape barcodes so that the user can find the tapes and re-insert them into the tape library.

RioBroker Unavailable

It is possible that your client may not be able to communicate with RioBroker due to the network or the RioBroker server being down. In this case, the client must handle this condition and must provide a useful message to the user so that they can attempt to resolve the issue.

Partial File Restore

RioBroker includes a feature called Time-Based Partial File Restore (PFR). Using this feature a client can restore part of a video file by specifying the time range.

In order to use PFR, the client must set the parameter “indexMedia” to “true” when the file is archived. This indexing will allow the PFR feature to be used on the file.

When the file is restored, the client must specify the “timeCodeRange” parameter with the time range of the file that is desired.

There are two possible errors that could occur when using PFR:

  • When attempting to archive the file, it is possible that the file cannot be indexed.
  • When attempting to restore the partial file, it is possible that the partial file cannot be created.

In both of these cases, the job status will be “error”. The client should be able to handle these errors.

Maintaining State

It is possible that the client, independent of RioBroker, could crash, or the server on which the client is running could reboot. In this case, the client will need to be able to return to its previous state and know the status of all of the active RioBroker jobs. The RioBroker command GET /api/jobs can be used to get the status of all RioBroker jobs.

Including File Size on Archive

When issuing an archive command (POST /api/brokers/{brokerName}/archive), one of the input parameters is the size of each file to be transferred in bytes. This file size is a required parameter when the source storage has an FTP interface. For source storage with a file path interface (e.g. \\server1\shareA), the file size parameter is optional -- RioBroker can determine the file size on its own. However, if at all possible, the client should provide the file size to maximize RioBroker performance.

Importing Foreign LTFS Tapes

BlackPearl has the ability to import foreign LTFS tapes and make the files on those tapes visible to RioBroker. In some cases, your customers may want to import these foreign LTFS tapes and make them accessible in your client. New files can be identified in Rio Broker by using the command PUT /api/brokers/{brokerName}/agents/{agentName} and specifying the “index” and “re-index” parameters both as “true”. The client can then choose to add these new files to its catalog/database and possibly restore the files if appropriate.

Multiple RioBroker Systems for Failover

BlackPearl supports the ability to replicate data to another BlackPearl, typically at another physical site. Replication is set at the bucket level in BlackPearl (a bucket in BlackPearl corresponds to a broker in RioBroker), and this replication is typically set up to be bidirectional.

A RioBroker server can be placed in front of each of these BlackPearls so that the customer has a fully redundant archive storage system at each site. In the scenario, the client will need to be aware of each RioBroker server endpoint and be able to fail over to the second RioBroker if the first RioBroker is not available. In the event that the failover occurs, the client may need to tell the secondary RioBroker to update its file index with the command PUT /api/brokers/{brokerName}/agents/{agentName} and specifying the “index” and “re-index” parameters both as “true”.

Utilizing Online Documentation -- Swagger Docs

Once you have installed RioBroker on a computer, the online API help documents can be accessed at https://localhost:5050/api/viewer/index.html. This swagger HTML version of the documentation is easier to use than the PDF version of the API documentation that is available on this site.

Installation and Operation Guide for Your Users

Make sure to provide your customers with written instructions for configuring and operating their RioBroker client in your application. Customers should be able to easily figure out how to set up and use the client. Remember that RioBroker requires that a cluster, broker, and agent be set up before it can be used. If your client doesn’t set up these objects itself, then you will need to explain in your written instructions how the customer does this themselves. Remember that the client is the customer’s primary interface not only to your application but also to RioBroker and BlackPearl.