The Project Build Library (PBL) is a centralized storage area for each project within CUBiT for storing data that do not require version control. The PBL is ideal for storing daily builds or custom packages that require distributing amongst developers but are not significant enough to be maintained in Subversion, i.e. binaries that could be deleted at a later time.
Basic Concepts
- Each project has its own Project Build Library
- Centralized storage area for data
- Data is not version control
- Data and access is protected by CollabNet role based permissions
- Accessed either though the CUBiT Web interface or through the HTTPS protocol
- Within each project's Build Library, files can be designated either as Public or Private. This designation governs who can download files from the project's Build Library.
- Public files are available to all registered users in all projects on the site (regardless of whether or not those projects are CUBiT projects) who have the 'Project Document - View' permission in at least one project.
- Private files are available only to users who have the 'Project Document - View' permission in that project.
- Uploading is done via a special cross-platform, commandline, open-source client. A special client is needed to interface with the CUBiT server and provide all the needed metadata to the PBL. We also publish the specs of the API, if users want to develop their own clients.
- Uploading files, and other "write" operations which change files (e.g., file move or remove) or file metadata (e.g., change the description associated with a file) can be done by any user with 'CUBiT - Root Access' privileges or better in the project.
In this paper the following topics will be covered.
- Overview of the PBL through the CUBiT web interface
- How to upload and deleting data in the PBL using the CUBiT Web API
- How to access the PBL using the HTTPS protocol
- How to access the PBL through Profiles
Overview of the PBL through the CUBiT web interface
Each project has its own dedicated PBL and after login into CUBiT it can be access via the following tabs
Projects -> {Project name} -> Build Library
The following example screenshot is the PBL page of the “cubit-demo” project.
The initial PBL page shows statistics on activities within the PBL, “Most Popular Downloads”, “Most Recent Downloads” and “Most Recent Uploads”.
The PBL is viewable through the “View Build Library” tab and after clicking on the link you will be asked to re-authenticate. An example of this is below; a hierarchal file system structure will be presented to you where you can transverse different directory structures and files. Clicking on any file will allow you to download that particular file.
The PBL is searchable through the “Search” tab. An example of the “Search” page for the “cubit-demo” project is shown below.
Through this page it is possible to search “All Projects” or any specific project for a file name or description. A combination of patterns, substrings or userids can be used in the search.
CUBiT provides auditing for the PBL and will show actions performed by users such as downloads, uploads, moves and deletes. The audit messages can be access via the “Audit Log” tab. A sample is shown below for the “cubit-demo” project.
Each entry has the following:
- Event Id - a unique integer that identifies the event
- File Name - the file name or directory that the action was performed against
- Event - the event that occurred against the file name
- User - the user performing the action
- Action - the action that occurred, i.e. download, upload, move or delete
- Time - timestamp of the action
How to upload and deleting data in the PBL using the CUBiT Web API
CollabNet provides CUBiT Web API’s to manage data in the PBL and to make matters easier CollabNet also provided is a Python client that utilize these API’s. The API’s methods that are specifically used for the PBL are:
- PblChangeDesc - Used to change the description of files and directories in the PBL
- PblDelete - Used to permanently remove files and directories in the PBL
- PblUpload - Used to create files and directories in the PBL
- PblMove - Used to move files and directories in the PBL
Explanations of these methods are outside the scope of this document and you should refer to “Build it - Web API's and example source code”. This remaining section will show you how to use the PBL through the Python client.
* It is assumed before proceeding that you have generated yourself a CUBiT Web API key.This can be done through “My Pages” after login into CUBiT web interface.
CollabNet provides a PBL Python client called “pbl.py” and is installed on every CUBiT node regardless of operating system type. Even though out of the scope of this document it is entirely possible to install the “pbl.py” on a system not managed by CUBiT to interact with the PBL.
A screenshot of the “pbl.py” client on a Red Hat node
The following functions can be performed from the client, upload, delete, move and changedesc.
Below is example of an upload in to the “cubit-demo” project.
/usr/bin/pbl.py upload --api-user=username --api-key=713cdf90-2549-1350-80c3-2d0bcf9a1697 --api-url http://sc74.sjc.collab.net/cubit_api/1 --project=myproject -t pub -r /foo/bar/baz -d "This is the description." /home/Release.zip
The following parameters are set in this command
- This is an upload
- The API user is “username”, this is same ID that is used to log you into CUBiT web interface
- The API key is generated from “My Pages”
- The API-URL is the URL to the CUBiT managers Web API, in our example this will be the CUBiT manager “sc74.sjc.collab.net”
- The project is “myproject”
- This is a Public file
- The directory that it will be uploaded to is “/foo/bar/baz”. If the directory does not exist it will be created.
- The description is "This is the description"
- The file “/home/Release.zip” will be uploaded
An example of Changing the Description of a File
Using the pbl.py changedesc command, you can change the description associated with a file or directory without changing the file itself or the md5 checksum of the file.
In the below command, we demonstrate changing the description of the file that we uploaded in the Uploading a File example above. This example assumes that you have already authenticated and saved your username and key credentials in your home directory. If you have not, you will also need to specify the --api-user and --api-key arguments.
pbl.py changedesc -l http://sc74.sjc.collab.net/cubit_api/1 --project=myproject -t pub -r /foo/bar/baz/Release.zip -d "This is the new description"
An example of Moving a File
With the pbl.py move command, you can move files or directories within a project, or even between projects. The syntax for this command is a bit different than the rest of the commands, because the other commands only operate on one project/file/directory at a time, and that is not the case the the move operation.
--srcproj projname
The name of the project the source file is located in.
--destproj projname
The name of the project to move the file to. If left blank, defaults to value of --srcproj.
--srcpath path
The path to the file or directory to move.
--destpath path
The destination path for the file or directory specified in --srcpath. Two important things to note about this option:
- If you specify a path which does not exist, that path will be automatically created for you as part of the move.
- If the --destpath parameter ends with a slash ("/"), the destination will be assumed to be a directory. If it does not end with a slash, the destination will be assumed to be a file. An example of this behavior is below. This is approximately how the UNIX "mv" command behaves.
--srctype {pub|priv}
The visibility type of the source file, either pub or priv.
--desttype {pub|priv}
The visibility type of the destination file, either pub or priv.
--force
If the destination file exists, the --force option must be used to replace it.
In the simplest case, we move a file, or a directory and all its contents, from one name to another. Note that because destpath does not end with a slash -- /foo/bar/baz/Release_old.zip -- the last component of the path is interpreted as "a file named 'Release_old.zip'".
pbl.py move -l http://sc74.sjc.collab.net/cubit_api/1 --srcprj=myproject --srctype=pub --srcpath=/foo/bar/baz/Release.zip --destpath=/foo/bar/baz/Release_old.zip
In the next example we move a file from one project to another, and also change it from public to private. Note that because destpath ends with a slash -- " /foo/bar/baz/archive/" -- the last component of the path is interpreted as "a directory named 'archive'".
pbl.py move -l http://sc74.sjc.collab.net/cubit_api/1 --srcprj=myproject --destproj=myproject_archive --srctype=pub --desttype=priv --srcpath=/foo/bar/baz/Release.zip --destpath=/foo/bar/baz/archive/
An example of Deleting a File
First off let us state something very important: Files and directories deleted from the Project Build Library are gone forever! Unlike a version control system, there is no way to "roll back" to a previous version, or to recover a file which has been deleted.
The pbl.py client supports both single-file and recursive deletion.
pbl.py delete -l http://sc74.sjc.collab.net/cubit_api/1 --project=myproject -t pub -r /foo/bar/baz/Release.zip -c "Deleting this file because it is no longer needed"
When deletion of a non-empty directory is requested, the --force option must also be specified. This is to help insure that you do not accidentally remove entire directory trees. In the below example, we delete the entire subdirectory that contains the file we have just been working with, and we also use the -c option to leave a comment in the audit log to explain why the deletion was performed:
pbl.py delete -l http://sc74.sjc.collab.net/cubit_api/1 --project=myproject -t pub -r /foo/bar -c "Deleting this subdirectory because the baz release has finished" --force
Finally, the --dry-run option can also be used with the delete option. If the --dry-run option is specified, no files will actually be deleted.
How to access the PBL using the HTTPS protocol
Access to the PBL is done over the HTTPS protocol and therefore can be supported by any standard web browser or command line utility that supports the HTTPS protocol.
Here is an example through Firefox to download the binary “cruisecontrol_cubitdemo.tar.z”.
The following URL will be used and typically you will be asked to authenticate as in the screenshot. Again you will use you CUBiT logon and password.
https://sc74.sjc.collab.net/pbl/cubit-demo/pub/CruiseControl/cruisecontrol_cubitdemo.tar.z
Once you have successfully authenticated the download will commence.
An example of a command line HTTPS is “wget”, as shown in the screenshot
How to access the PBL through Profiles
* Please note for comprehensive document for Profile creation then refer to “Build it - Profile creation”. This section will only discuss PBL integration into Profiles.
CUBiT Profiles can access PBL files using the URI pbl://
This allows the reuse and automated installation of PBL stored binaries, builds and packages directly into a provisioned system via a profile.
Example 1 - Access through the HTTPS protocol
<script>
### Added to install and start CruiseControl
cd /
wget --no-check-certificate https://sc74.sjc.collab.net/pbl/cubit-demo/pub/CruiseControl/cruisecontrol_cubitdemo.tar.z
tar zxvpf cruisecontrol_cubitdemo.tar.z
</script>
Within the <script> section of the XML defined profile a “wget” can be added to download the binary. After it has been downloaded then actions can be performed on that binary.
Example 2 - Access through the URI pbl://
<rpms action="install" path="pbl://sc74.sjc.collab.net/pbl/cubit-demo/pub/java/">
<rpm>jdk-6u3-linux-i586.rpm</rpm>
</rpms>
The action defined in the XML profile will download and install the RPM“jdk-6u3-linux-i586.rpm” from the “cubit-demo” project PBL from the path “/pub/java”.
Conclusion and Summary
This concludes the article on the CUBiT Project Build Library and to summarize what we have learnt today.
- A Project Build Library is a centralized storage area allowing the reuse and management of binaries that typically would remain unmanaged on servers and workstation.
- Binaries within the PBL become reusable assets promoting collaboration and efficiency.
- The PBL is protect by CollabNet role based access control and can be access by the standard HTTPS protocol.
- CollabNet provides Web API’s to integrate your own tools with the PBL
- CollabNet provides integration between CUBiT profile and the PBL
