BigIPReport (REST)

Introduction

This is the new version that removes the dependency on the iControl powershell plugin. This allows the script to be executed at any platform that supports Powershell (including linux).

This is a script which will generate a report of the BigIP LTM configuration on all your load balancers making it easy to find information and get a comprehensive overview of virtual servers and pools connected to them (including those specified in iRules).

Feature requests / Help

Feel free to post any feature requests on the code share page on Devcentral. They are very much appreciated.

You can also try our Discord.

Requirements

  • A server to run the report script on
  • PowerShell version 6.0 or newer
  • A web server of your choice

Only here for the code?

All releases are linked here:
https://devcentral.f5.com/codeshare/bigip-report

Fast track installation instructions

First you might want to try the docker installation which is currently under development. It’s the fastest way to get up and running on a linux distribution.

https://loadbalancing.se2021/01/05/running-bigipreport-on-docker/

If docker is not your thing you can also try the Kubernetes deployment. If none of these sounds interesting and you haven’t set this up before I would recommend following the more detailed path below. If you’re in a rush or simply don’t want to read that much, here’s the quick and dirty alternative.

  1. Setup a auditor user with the same password on all load balancers
  2. Find yourself a server with Powershell 6.0 or above
  3. Make sure that the server/client running the script has HTTPS access to all load balancer management interfaces
  4. Configure at least the minimal config in bigipreportconfig.xml (Credentials, Device groups, Report Root and Default document).
  5. Cross your fingers for luck
  6. Run the script and look for any errors.
  7. Browse your report in a web browser (only Chrome and Firefox tested, gave up on IE)

Tips from Tim Riker

  • The script now copies the underlay files to the report root when starting.
  • Use slashes instead of back slashes in the Report root in order to favor a more OS agnostic configuration file.
    Example: <ReportRoot>../../../inetpub/wwwroot/</ReportRoot> can be used in both Windows and Linux

Installation instructions

Install a web server

The Bigip report script needs a web server to host the results of the script. Since there’s no server side execution it does not matter which Web server you choose, just that you have one (I’ve been hosting Bigipreport on both IIS and Apache).

I won’t bother writing a guide on how to install a web server since there’s so many online, but here’s a few direct links:

Windows 8/10
http://www.howtogeek.com/112455/how-to-install-iis-8-on-windows-8/

Windows server 2012
http://www.iis.net/learn/install/installing-iis-85/installing-iis-85-on-windows-server-2012-r2

Apache on Ubuntu
https://help.ubuntu.com/lts/serverguide/httpd.html

Download the BigipReport package

  1. Choose a version on the code repository page
    Unpack the zip file to a directory of your choosing.
  2. The script and the bigipreportconfig.xml file should be copied to where you store scripts (up to you).

Create a read-only auditor user on the F5 Load balancers

The script will need a user on the F5 load balancers. While it’s possible to use your own user it’s highly recommended out of security reasons to create an auditor user for the script.

Just make sure that the user has read access to all partitions and that the same credentials is used on all load balancers.

Configure the script

So you’ve prepared the web server, downloaded the Bigip Report package and configured user accounts for the script? Well done, almost there!

The configuration file has explanations in it so I won’t go through more than the bare minimum in this guide. If you feel something is missing, please let me know!

  1. Open the “bigipreportconfig.xml
  2. First we need to configure the credentials the script uses:
    <Credentials>
    <Username>icuser</Username>
    <Password>icontrolisfun</Password>
    </Credentials>
  3. Then we need to tell the script which load balancers to connect to (read the section about Status VIPs below for more information about that):
    <DeviceGroups>
    <DeviceGroup>
    <Name>Stockholm QA</Name>
    <Device>192.168.10.23</Device>
    <Device>192.168.10.23</Device>
    <StatusVip>https://lb1statusvip.domain.local</StatusVip>
    </DeviceGroup>
    <DeviceGroup>
    <Name>London Cluster</Name>
    <Device>192.168.11.55</Device>
    <Device>192.168.11.56</Device>
    <StatusVip></StatusVip>
    </DeviceGroup>
    </DeviceGroups>
  4. Please note that the ReportRoot has to be configured according to your system. If using linux you can use full paths, ie. /var/www/html and if using windows ie. C:\InetPub\wwwroot\
  5. And where to put the result of the script:
    <ReportRoot>
    \\bigipreport.mydomain.local\bigipreport\
    </ReportRoot>
  6. And finally the default document:
    <DefaultDocument>
    default.html
    </DefaultDocument>

That’s it for the mandatory script configuration, look below for more documentation about the other options.

Testing the script

  1. Open a powershell window as a non-admin
  2. Navigate to the script directory
  3. Execute the script by entering .\[script name] and enter
  4. Watch for any exceptions or errors

Schedule the script on Windows

If you’re happy with the result you might want to schedule the script to keep the content updated.

  1. Click on the start menu and then enter “task scheduler
  2. In the task scheduler, click on create task (located on the right hand side:
    schedule
  3. Under the general tab, choose a name for the task and choose to run the task whether the user is logged in or not
    For security reasons you might want to change the user the script is running as as well
  4. Then click on the tab called “Trigger” and click “New
    Enter the settings you desire and click OK.
  5. Move on to the tab called “Action” and click “New
    Action: Start a program
    Program/Script: Browse to the script
  6. Click OK and then “OK
  7. Enter the credentials of the chosen account and then click “OK

Schedule the script on Linux

Like most things in Linux this is a matter of taste, but I’d put the powershell in a dedicated folder and then execute it with a set interval using crontab.

Optional – Usage of multiple configuration files

Example on how to launch the script using an alternate configuration file:

./bigipreport.ps1 -ConfigurationFile bigipreportconfig2.xml

The script defaults to "$PSScriptRoot/bigipreportconfig.xml" so there is no need to change anything if you don’t plan to run multiple versions of BigipReport.

Additional recommended script configurations

Adjust the paralell processing

The build script runs one parent thread and one thread for each listed Device. The MaxJobs setting controls how many parallel PowerShell jobs are launched. Increasing it will work well if your build host has the memory and cpu power. Decreasing it may be required if running on a build machine with fewer resources.

Adjust RestPageSize

Some devices might have issues generating data for large configurations. To prevent timeouts due to heavy API calls we have introduced RestPageSize. The option will determine the maximum objects per call. Example, setting it to 50 (default) will make the report script retrieve 50 virtual servers per call. This means that if you have ie. 170 Virtual servers 4 calls will be made when retrieving that particular information.

Setting a higher value could result in slightly faster report generation but could also result in timeouts. Setting it lower will result in slower report generation but could also mitigate timeouts if you experience them.

Configure logging

I would highly recommend to also configure logging. It will help you troubleshoot in case the report is failing later on.

Edit the bigipreportconfig.xml file and modify the section called LogSettings. Examples are available in the file.

Configure error reporting

Error reporting will have the script send an email when the script runs into trouble. Configure the ErrorReporting section in the bigipreportconfig.xml file. Examples are available in the file.

Pool Member State Polling

Polling uses the F5 iRules engine to determine the state of pool members which enables the report to provide the users with live updates of member states without them having access to the actual F5 interface. This is done by letting the users browser interface with the configured Status VIPs using Javascript XHR.

However, there’s one caveat here. The iRule engine is a bit limited compared to the iControl interface, which means that the results is not as detailed as if using the reports original icons. Instead, the result would indicate whether the member is capable of passing traffic or not. Also, the virtual server statuses will not be changed. This might be added later on if I can figure out a reliable way to do so.

Essentially this means that the icons will not be as detailed in the main view of the report, however, the pool details would still contain the original state AND the current one.

Here’s a translation table with most of them:

Original iconMember AvailabilityMember StateTranslated Real-time stateTranslated icon
No monitor assignedEnabledEnabled
Monitor downDisabledDown
N/AForced downDown

If you encounter another, please let me know.

Performance considerations

The iRule impact on the system has been tested by sending a constant barrage of requests to the status VIP. While doing this I was barely able to make a dent in the CPU usage of my lab instance. That said, you probably want to keep track on the statistics to be sure.

In order to keep the resource usage in control there’s three settings in the xml config file:

  • MaxPools – How many pools that can be queued up in total
  • MaxQueue – How many that can be polled simultaneously
  • RefreshRate – How often the state is refreshed

A tip is to start with the default settings and see if that’s good enough.

How to enable polling

Now that you understand the limitations, you can follow these steps to enable the real-time status in the report.

Setting up a member Status VIP

  1. Open up the bigipreport_pool_status.tcl iRule located in the iRules folder in the zip file.
  2. Create an iRule named bigipreport_pool_status with the content of the iRule bigipreport_pool_status.tcl found in the iRules folder of the bigipreport zip file.
  3. Create a virtual server with an HTTP profile and assign the iRule to the virtual server. Please note that if your BigIPReport is using SSL, so must your status VIP.
  4. Make sure that all of your report users has access via the firewall to the virtual server.

Prerequisites

If your bigipreport instance use HTTPS

Since the polling uses XHR the following conditions must be true:

  1. Your status VIPs must use HTTPS.
  2. Your status VIP must have a DNS pointed to it.
  3. Your status VIP must have an SSL certificate that validates the DNS.
  4. The SSL certificate must be trusted by the clients.

Configuring the report to use it

In the device group section of the configuration, define the URL to the member status VIP. Here’s the example form the configuration file itself.

 <DeviceGroups>
     <DeviceGroup>
          <Name>F5YP Devices</Name>
          <StatusVip>https://poolstatus.j.local</StatusVip>
          <Devices>
              <Device>
                   <IP>192.168.10.23</IP>
              </Device>
              <Device>
                   <IP>192.168.10.24</IP>
              </Device>
           </Devices>
      </DeviceGroup>
 </DeviceGroups>

If you have any problems with the member endpoints you can check out these sections from the FAQ below:

Brotli support

If you do not know what Brotli is you can look at it as a much more efficient way to compress css, json and javascript files. It’s a bit slower to compress but on the upshot it’s much smaller and using it will speed up the BigIPReport application delivery significantly. The last week we’ve been focusing on making it easier for people to use Brotli with their BigIPReport installations by creating server templates for different web server vendors.

For those with larger BigIPReport installations I’d really really recommend checking this out. It pretty easy and the gain is high.

Before you start though, please note that the report must run over HTTPS for Brotli to be supported!

Docker/Kubernetes

If you’re running bigipreport/frontend:v5.6.8 or latest this is already supported out of the box as long as you have an HTTPS proxy of some sort in front of it.

F5 iRules

For those serving BigIPReport via an F5 you can grab an iRule which will do the necessary rewrites to use Brotli. The iRule can be found here:
https://github.com/net-utilities/BigIPReport/blob/master/other/ServeBrotliViaF5/serve-brotli.tcl

IIS

Still stuck with an old Windows installation? First I’d recommend moving to a Linux based installation instead. If this is not possible Tim has been so kind as to share his IIS web.config here:
https://github.com/net-utilities/BigIPReport/blob/master/other/iis/web.config

Apache

For those that uses Apache our superstar Tim has yet again delivered. You can find the Apache config here:
https://github.com/net-utilities/BigIPReport/blob/master/other/apache/brotli.conf

Nginx

If you prefer to run your own Nginx server you can check out the file used in the frontend container:
https://github.com/net-utilities/BigIPReport-Docker/blob/master/frontend/default.conf

Double checking that it works

Using curl this is easy enough. Just run the command below and look for “content-encoding: br”:

 curl -I -H "Accept-Encoding: br" https://bigipreport.xip.se/json/pools.json
HTTP/2 200 
server: istio-envoy
date: Wed, 11 May 2022 20:59:36 GMT
content-type: application/json
content-length: 709
last-modified: Wed, 11 May 2022 20:58:19 GMT
etag: "627c236b-2c5"
content-encoding: br
vary: Accept-Encoding
accept-ranges: bytes
x-envoy-upstream-service-time: 0

  You can also double check this by opening up the developer tools of Chrome, head over to the Network tab and refresh your BigIPReport page. If Brotli is used as it should you should see “content-encoding: br” in the response headers. See the screenshot below:

Patrik_Jonsson_0-1652303110730.png

Protect the report content using APM

A nice guy named Shannon offered details on how to protect the report behind an APM. This provides a much cleaner interface than using integrated authentication in IIS and is recommended if your company uses APM.

For more information about that, please refer to this post:

https://loadbalancing.se2018/04/08/protecting-bigip-report-behind-an-apm-by-shannon-poole/

Upgrading the script

If the new version is much higher than the old one there might be a configuration file syntax change which would require you to re-do the configuration again. There is a column called “Need Config update?” in the version history of bigipreport.ps1 that indicates when the configuration file needs to be changed between different versions. Sometimes there might also be new sections which controls new features, like the real time member status (look above).

  1. Replace the bigipreport.ps1 script with the new version.
  2. Check the version history to know if you need to also update the bigipreportconfig.xml file.
  3. Run the script
  4. When the report has finished, test it by accessing it and hitting CTRL + F5 to ignore the cache.

F.A.Q

Unable to get a token from the device

I get authentication errors for the report user

Of course, verify the credentials by logging into the Web interface with the report user. Please note that in order for the script to work the user has to have at least the role auditor on all partitions.

If using LDAP authentication on v14 you might also want to check out this article:
https://cdn.f5.com/product/bugtracker/ID881085.html

If you have verified that the user is an auditor you can also clear the REST API and restart the agent:
https://support.f5.com/csp/article/K30288514

Authentication failed see inner exception

If the script is throwing the error Authentication failed see inner exception you’ll want to look at the ciphers configured on your management interface. Run the following command to see the configured ciphers:

tmsh sys httpd ssl-ciphersuite

You should then see a string that contains your cipher configuration. Taking this list and running the tmm -clientciphers command generates a list of the ciphers that your management interface supports:

tmm --clientciphers '<cipher string>'       
           ID  SUITE                    BITS PROT   CIPHER  MAC     KEYX
 0: 49199  ECDHE-RSA-AES128-GCM-SHA256  128  TLS1.2 AES-GCM SHA256  ECDHE_RSA
 1: 49171  ECDHE-RSA-AES128-CBC-SHA     128  TLS1   AES     SHA     ECDHE_RSA
... abbreviated...

Look here for an article on how to modify the ciphers.

No certificate information is shown

Verify that the report user has the role Auditor. The old report required guest, but certificate information is not available as Guest using the REST API.

I get a status code of 403 when browsing to the script directory

This is probably due to the web server not having the chosen report file name in its default document list.

Possible solutions

The script finished, but no file was written

Run the script manually and check the logs.

Possible ReasonSolution
The script failed to connect to one or more of the configured load balancersRemove the failing load balancer from the configuration
Check the credentials of the script user
The user running the script does not have permissions to write to the folderGrant permissions to the user running the script
Open a windows share by configuring a share in the bigipreportconfig.xml file (shares section)
Script and web server is running on separate servers and the firewall is not open between the server running the script and the web server.Open the firewall

The script has failed but no report has been sent

Always try to run the script manually in a powershell window to get exact details by using the following syntax:

./bigipreport.ps1 bigipreport.xml loadbalancer.example.com

Please note though, that no report will be generated when running this. It’s only for troubleshooting purposes.

Possible reasonSolution
The firewall is not opened for smtp traffic from the script server to the smtp serverOpen the firewall
The mail server does not allow the script server to send emails through itConfigure the mail server to allow relay from the script server

I have scheduled the report but it does not write any files

There could be multiple reasons for this, and the best way to troubleshoot is to start a powershell window as the user running the report. Then execute the script manually.

Possible reasonSolution
The user does not have write access to the configured log folderCheck the log folder permissions
The user running the script does not have Run as batchjob user permissionsEdit the group policy to allow the user access
Some pre-execution tests failedRun the script as the intended user and fix the problem.
The script user does not have write permissions in the report directoryAdd permissions to the report directory

The report is looking strange

Possible reasonSolution
The front-end files does not match the script versionFollow the upgrade guide above
A java script has failed to loadCopy the java script exception from the browser developer console and report it on the Bigip Report Devcentral page.

The script reports missing JSON files

If running an older version of IIS (7.x or older) you might have to configure MIME types for JSON files.

For IIS7.x or older

Open “IIS Manager”, click on “Mime types” and then “Add”. Use the following settings:

Extension: .json
MIME Type: application/json

If this does not solve the issue, try creating a file called web.config in the wwwroot folder with the following content:

  <system.webServer>
    <staticContent>
      <mimeMap fileExtension=".json" mimeType="application/json" />
    </staticContent>
  </system.webServer>

Other web servers

Script has had issues when creating the files due to lack of permissions or network issues. Double check your script execution logs, web folder content and try again?

The member status polling says it’s disabled

This is due to that the javascript testing the polling endpoints has determined that there is no functioning endpoint available. See the section below on how to troubleshoot member status endpoints.

One or more status endpoints has been marked as failed

Before initializing the polling feature the javascript will test each endpoint to determine if they are valid and working. Most of the time just looking at your browsers Javascript console should give you a good idea of where the issue lies but but here’s a few possible reasons:

ReasonSolution
The configured endpoint is using HTTP while your BigIPReport is using HTTPS. This would trigger security features in modern browsers and prevent any AJAX request.Configure the endpoint to use HTTPs, or less optimal, configure the report to use HTTP.
The endpoint is not using a certificate trusted by your browser.Sign a certificate trusted by your browser.
The DNS record is not resolvable by your clients.Create a record that is resolvable by your clients.
You have create the member status endpoint on one load balancer but forgot to synchronize the configuration.Synchronize the configuration