Upcoming BlackPearl 5.0 Release Requires Client Recertification

Written by

BlackPearl’s next release, 5.0, is expected in June, and will introduce several important and exciting new features for our customers, including the following:

  • Intelligent Object Management – This features allows objects to self-heal if there is a problem detected with one copy and there are multiple copies available; allows tapes to be re-compacted; allows data to be migrated from one tape type to another; and allows objects to be staged from tape to disk for faster read access.
  • Versioning – BlackPearl will allow multiple versions of the same objects to be uploaded and saved

In order to implement these new features, changes to the BlackPearl API are required that could affect existing BlackPearl clients. With some major BlackPearl releases (3.0, 5.0, etc.), changes to the BlackPearl API are introduced to support new features and to remove features that have been deprecated. We do not take these API changes lightly, as we know that any change can affect the functionality of BlackPearl clients. We do everything we can to limit these changes while still allowing for the addition of new features.

The changes in 5.0 will require all BlackPearl clients to be updated, recertified, and retested before they can be used with BlackPearl 5.0. At a high level, here are some of the changes being made to the API that could affect existing clients:

Syntactic changes:

  • Removed fields in payload responses -- can cause errors when these entities are queried.
    • Tapes and pools no longer have storage domain ids -- they have storage domain member ids
    • Objects no longer have the old “version” field because the id is now considered the version
    • Data Policy no longer has “always replicate deletes” or “LTFS naming allowed” -- this functionality has been moved
  • Removed request parameters -- can cause errors if a customer tries to use them
    • The “import conflict resolution mode” query parameter for performing tape and pool imports has been removed
    • The following bucket attributes have been removed:
      •     DEFAULT_WRITE_OPTIMIZATION
      •     DEFAULT_GET_JOB_PRIORITY
      •     DEFAULT_PUT_JOB_PRIORITY
      •     OBJECT_SPANNING
      •     DEFAULT_CHECKSUM
      •     MIN_CLIENT_CHECKSUM
      •     MAX_CLIENT_CHECKSUM
    • Items in the previous section can no longer be filtered by their removed attributes. For example, “get tapes” cannot be filtered by “storage domain id”
  • XML payload attributes that are no longer valid:
    •   priority (now specified as a query parameter)
    •   chunk_client_processing_order_guarantee (now specified as a query parameter)
    •   write optimization (deprecated for xml payloads)
  • Changes to enums -- this includes things like new error message types that can cause errors when received by older clients
    • “Keep multiple versions” is a new versioning type.
    • “Auto compaction in progress” is a new tape type

Semantic Changes:

  • Requests that used to return a 503 “object not uploaded yet” now return a 404.
  • Objects not fully uploaded are no longer listed in an AWS-style “get bucket” command
  • Objects are now marked “latest” once they fully arrive in cache -- not when the job is created.
  • CacheAvailableRetryAfterInSeconds decreased from 300 seconds to 60 seconds -- this is how long Black Pearl recommends the client wait between checking for work to be done on a job
  • The “import conflict resolution mode” query parameter for performing tape and pool imports is still available as an optional parameter, but is ignored if sent.

You can view the full API difference comparison between BlackPearl 4.1 and 5.0. Note that the BlackPearl 5.0 API is still subject to change.

At the very least, clients will need to update to a new release of our SDKs, which will be available on our Downloads page. In some cases, code changes may be required to client integrations.

We have released the 5.0 beta SDKs for Java and .NET now. The Python, C, and Golang SDKs for 5.0 should be available shortly. We also have a BlackPearl running the 5.0 beta release that is available over the Internet. Please Contact the Developer Program Team when you wish to access it.

Testing and recertification will be required to ensure that your client works with BlackPearl 5.0. We will require you to redo the file transfer tests (tests 5 through 8) specified in our Certification Program Test Plan. These tests will need to be done in our lab or at a facility with a BlackPearl and tape library, and the tests results need to be approved by Spectra Logic in order to receive certification for BlackPearl 5.

Please Contact the Developer Program Team if you require assistance.