In Part 1 of my BlackPearl 3.0 blog post, I discussed the new storage mediums and data policies that can be used with BlackPearl. Today I focus on another new feature of BlackPearl 3.0, Access Control Lists (ACLs). As the name implies, with ACLs you will be able to control what type of access users and applications have to the data in BlackPearl.

The concept of ACLs is not new. Amazon has been using ACLs with its public cloud storage for some time. BlackPearl’s ACL features are very similar, but not identical, to Amazon’s ACLs. These differences are due to the nature of a private cloud (BlackPearl) versus a public cloud system.

ACLs and Buckets

In BlackPearl, ACLs are primarily used to control permissions on objects in buckets, which is a top-level container in BlackPearl. Bucket permissions can be granted to a user or a group.

Groups are a new feature in BlackPearl 3.0, and can consist of users and other groups. Groups can be managed on the Users page in the web management interface. BlackPearl 3.0 ships with two default groups, “Everyone” and “Administrators”, and more groups can be created.

The following permission(s) can be granted to a user or group on a bucket:

• List – List all objects in a bucket
• Read – Download (GET) objects from a bucket
• Write – Upload (PUT) objects to a bucket
• Delete – Delete objects from a bucket
• Job – Modify or cancel jobs associated with the bucket, even if the job wasn’t created by that user or group
• Owner – Full control of bucket. Includes all permissions above. By default the user that creates the bucket is given Owner permissions on that bucket

When a bucket is created or edited in the web management interface, ACLs can be set on the bucket as shown in the first image below. A typical use case might be that you want to give full access to a bucket to certain users and only read access to other users. In this case you could create a “Full Access” group and set an ACL giving that group “Owner” permissions on the bucket. You could then create another group called “Read Only Access” and set an ACL giving that group “List” and “Read” permissions.

Global Permissions

Users and groups can be granted “Global” ACLs on all buckets. So for example, a user or group could be given “Read” permission on all objects in all buckets in BlackPearl. These settings are controlled in the “Global Bucket Access Control List” in the User and Group settings as shown in the two images below.

ACLs and Data Policies

As mentioned Part 1 of this blog post, data policies can be set on buckets that control how many copies of the objects are kept on each storage medium and for how long. In some cases, you may only want certain users to have access to use certain data policies when creating a bucket. BlackPearl allows you to set ACLs on data policies that allow you to control this access. BlackPearl 3.0 includes a built-in “Everyone” group, and members of this group by default have global permissions to use all data policies as shown by the checked box in the image above. This means that by default all users can access all data policies. However, administrators can remove this setting and only let certain users access certain data policies, as shown below.

ACLs in the API and SDKs

All the ACL settings that are available in the web management interface, as shown in the screens above, can also be controlled using the BlackPearl API and SDKs. So for example, not only can a bucket be programmatically created, but the ACLs on the bucket can also be programmatically set.

Coming Up Next

In my next blog post, I’ll talk about how developers will be able to use the new BlackPearl 3.0 features via the API and SDKs. I’ll also explain how developers that have are already built or in the process of building clients can prepare to migrate to BlackPearl 3.0.

On October 15, 2015, Spectra Logic announced ArcticBlue, new nearline disk solution that sits behind the BlackPearl Deep Storage Gateway. BlackPearl now provides an S3 object private cloud interface to the following storage products:

• Spectra Logic tape libraries – BlackPearl has supported archive to tape libraries since its original release
• ArcticBlue – ArcticBlue is a new nearline storage target for BlackPearl. Read more about ArcticBlue
• Spectra Online Disk – Spectra Online Disk with Enterprise SAS drives are also a new storage target for BlackPearl

As part of the ArcticBlue release in December, we will also be releasing the next major software version of BlackPearl, Version 3.0 (we are skipping 2.0 to get BlackPearl and Verde on a common code base release). This new version will not only include support for ArcticBlue and Spectra Online Disk, but also includes two other new major features:

• Advanced Bucket Management – Allows data policies to be set on buckets to control how many copies and for how long objects are stored on each storage product listed above. Advanced Bucket Management is covered below.
• Access Control Lists – Provides sophisticated permission control on objects and buckets. Access Control Lists will be covered in Part 2 of this blog post.

Advanced Bucket Management

Advanced Bucket Management (ABM) is an extremely powerful new feature provided at no additional cost in BlackPearl Version 3.0. Policies are set on buckets that determine which storage type each object in the bucket will be stored on and for how long each object will be stored on each storage type. You can see an example scenario in the diagram below. Though this is probably not a realistic scenario, it does show all the different policy options.

In the diagram above, Bucket 3 has a 4-copy data policy. When objects are moved to Bucket 3, a copy of the object is immediately placed in each of the four storage domains:

• A copy will be placed on online disk for 30 days for very fast object retrieval.
• A copy will be placed on ArcticBlue nearline disk for 2 years for fairly fast object retrieval (ArcticBlue is “power down” disk so it takes a bit longer to respond than online disk).
• A copy will be placed in a Spectra T950 tape library with TS1150 tape drives. This copy has no expiration.
• A copy will be placed in a Spectra T200 tape library with LTO-7 tape drives. The tapes on which this object is stored will be ejected from the library for offsite storage.

When an object stored in Bucket 3 is requested by an application, the BlackPearl knows to retrieve it from the fastest available storage domain. So if the object is being requested within the first 30 days, it will be retrieved from online staging disk. Between 31 days and 2 years, the object will be retrieved from ArcticBlue nearline disk. And after two years the object will be retrieved from tape.

When a bucket is created, it must now be assigned a data policy. In the web management interface you will be forced to choose a data policy (see below). If you create a bucket via the API/SDK, you can also assign a data policy. But if you don’t assign a data policy, the user’s default data policy will be assigned to the bucket.

NOTE: If you are upgrading from 1.x to 3.0, you will need to assign a default data policy for each user.

BlackPearl will ship with a number of common data policies, as shown on the screen image above. These policies are automatically created based on the hardware attached. If only tape is attached then two tape policies will be auto generated and will work for most users. However, users can create their own data policies as well. Developers will be able to manipulate nearly all aspects of data policy management via the API and SDKs. We will be providing documentation on how to do this as we get closer to the release date of BlackPearl 3.0 in December.

To support Advanced Bucket Management at the most basic level in a BlackPearl client, the client should support the ability to use multiple buckets. Having multiple buckets, as shown above, will allow for the user to choose different data policies. One policy for frequently accessed data could have one copy in Spectra Online Disk for 120 days and one copy in ArcticBlue Nearline Disk forever. A second policy could be the “One Copy Tape, One Copy Nearline Disk” which is two permanent copies, great for warm data that needs parallel access while providing genetic diversity with extremely high level of durability. This would provide users two different types of storage profiles within one platform.

You can learn more about the new features of BlackPearl by viewing the recording of our inaugural Developer Summit.

In Part 2 of this blog post, I will describe the new Access Control Lists (ACLs) feature.