Improve Performance with The JetBackup Priority System

As a general recommendation, we typically suggest scheduling your Backup Jobs during your server’s off-peak hours to get the most out of your backup times. Considering that larger accounts can often take much longer for their backups to complete, we have seen cases where backup times significantly vary depending on the time backups for certain accounts get processed.

Prioritizing the backups for your accounts can be a helpful tool in optimizing your backup jobs and ensuring that backups for larger accounts take place during your server’s off-peak hours. With the priority system introduced in JetBackup 5, you now have the ability to control and organize the order in which backups for each account get processed in the queue.

This guide will walk you through the general process of designating queue priorities for an individual account(s) as well as automating this task for all accounts on a server based on their inode usage!

Specifying priority for individual accounts

If you notice that a specific account(s) is taking most of the time to back up, consider setting up these accounts to a higher priority and see if it helps improve your total backup time. To do so, please do the following:

1. Select the accounts tab.

2. Select the desired account and then select “Manage Account”.

3. Select the “Account Queue Priority Group Override” drop-down and select the desired location within the queue priority.

Assigning priority for all accounts on a server via script.

For clients with multiple servers and thousands of accounts, it can be impractical to set priorities for each account individually. In such a case, you can utilize our JetBackup 5 API to automate this process and set your account’s priorities according to specified parameters. In the example below, we have a sample script that automatically sets the priority for each account depending on their Inode usage:

Useful API commands

API commands allow you to access information and make changes to accounts via the command line. This in turn allows you the capability to automate and script tedious tasks for large numbers of accounts at once. Below are some commands that are useful for assigning queue priorities in this manner. Additional API commands and more detailed information can be found in our documentation.

• listAccounts
  • Lists useful information for all accounts on a server such as the account ID which can be used to make changes to accounts.
  • Important return information:
    • _id
  • Example:
    • jetbackup5api -F listAccounts
• getAccount
  • Lists more detailed information for a specific account. For example, this command will specify which priority group has been assigned to the account if one has already been set.
  • Important return information:
    • quota
      • inode_usage
      • disk_usage
  • Example:
    • jetbackup5api -F getAccount '_id={account_id}'
• listQueuePriorities
  • Lists queue priority groups and outputs useful information such as their IDs which are used in conjunction with the manageAccount command to assign accounts to priority groups.
  • Important return information:
    • _id
      • ID used to identify a queue priority and assign priorities to accounts. This ID is not the same as an account ID.
  • Example:
    • jetbackup5api -F listQueuePriorities
• manageAccount
  • Primary command for manipulating accounts and making changes to them. Used to assign or
    change an account priority in this example.
  • Parameters:
    • _id
      • Specify account being managed.
    • queue_priority
      • Assign queue priority. This must be done using the queue priority ID from the getQueuePriorities command.
  • Example:
    • jetbackup5api -F  manageAccount -D "_id={account_id}&queue_priority={queue_priority_id}"

Example Script

This script automatically assigns all accounts on the server to queue priority groups “High”, “Normal” and “Low” based on their inode usage. The inode usage requirements for each group are specified by variables within the script and can be customized to suit individual servers or use cases. This script will also skip any accounts already assigned to a Queue Priority Group. Additionally, a script like this can be written to prioritize accounts by other parameters of the user’s choosing.

#!/bin/bash

accountids=($(jetbackup5api -F listAccounts | grep -w '_id' | awk '{print $2}'))


#logic for finding queue priority id to be used for assignment
queuepriohigh=($(jetbackup5api -F listQueuePriorities | grep -w 'name: High' -B1 | grep -w '_id:' | awk '{print $2}'))
queueprionorm=($(jetbackup5api -F listQueuePriorities | grep -w 'name: Normal' -B1 | grep -w '_id:' | awk '{print $2}'))
queuepriolow=($(jetbackup5api -F listQueuePriorities | grep -w 'name: Low' -B1 | grep -w '_id:' | awk '{print $2}'))


#inode limit variables. Can be changed to suit individual server needs. 
lowinode=3500 #accounts below this limit will be assigned to "low" priority
norminode=30000 #accounts between low and normal limit will be assigned to "normal". Above normal limit will be assigned to "high"

for id in ${accountids[@]}
do
    checktag=$(jetbackup5api -F getAccount -D _id=${id} | grep 'queue_priority:' | awk '{print $2}') #logic for checking if already tagged

        if [ -z "$checktag" ] #if no tag exists assign tag based on inode usage.
            then
                findsize=$(jetbackup5api -F getAccount -D _id=${id} | grep 'inode_usage:' | awk '{print $2}')

                    if [[ $findsize -gt 0 ]] && [[ $findsize -le $lowinode ]];
                        then
                            jetbackup5api -F  manageAccount -D "_id=${id}&queue_priority=${queuepriolow}"
                            echo "Tagging id: ${id} under 3500 inode"
                    elif [[ $findsize -gt $lowinode ]] && [[ $findsize -le $norminode ]];
                        then
                            jetbackup5api -F  manageAccount -D "_id=${id}&queue_priority=${queueprionorm}"
                            echo "Tagging id: ${id} between 3500 and 30000 inode"
                    elif [[ $findsize -gt $norminode ]];
                        then
                            jetbackup5api -F  manageAccount -D "_id=${id}&queue_priority=${queuepriohigh}"
                            echo "Tagging id: ${id} above 30000 inode"
            fi
        else
        echo "${id} already proritized. Skipping..."
fi
done
echo "Done."

Additional information on Queue Priorities

For more information regarding JetBackup5’s Priority Systems, please visit https://docs.jetbackup.com/v5.2/adminpanel/faq.html#how-does-jetbackup-prioritize-which-tasks-to-run-first