The following steps are required to completely configure our on-premise SWOOP Miner for Microsoft Teams.
In order to provide you with the credentials we will require a mobile (or SMS friendly) number to send you an encryption key. Please provide this when requesting the script described below.
1. Azure Application Creation
The application for the miner will need to be created under Azure Active Directory, with suitable permissions. In addition the application ID needs to be submitted to Microsoft for white-listing in order to do data mining in Microsoft Teams. The miner runs continuously in the background, without a user login. These steps explain how to create the app:
- Go to the Azure Portal.
- Select "Azure Active Directory" menu item from the left hand side menu.
- Select "App registrations" from the Azure Active Directory sub-menu.
- Press the button "New registration" at the top.
- Give the app a name, such as "SWOOP Miner for Microsoft Teams" and as a standard SWOOP uses the the redirect URL https://<your server address>/ which ends up redirecting again back to your REDIRECT value (see table below).
- Click the "Register" button at the bottom.
- After the registration is done, click the "View API Permissions".
- Click "Add a Permission".
- Click into the permission section called "Microsoft Graph".
- Under the "Delegated Permissions" you should see User.Read already ticked so leave that.
- Under the "Application Permissions" tick the following four permissions only:
- Click "Add Permissions" at the bottom.
- Once the permissions are saved, click the "Grant admin consent" button. The result should look like this:
- Click into "Overview" at top of sub-menu.
- Take a note of the UUID under "Directory (tenant) ID"
- Take a note of the UUID under"Application (client) ID"
- Click "Certificates & Secrets" in the sub-menu.
- Use the "New client secret" button and follow the steps to get a secret. Keep a note of the secret because it only will show one time. NOTE: if you set it to expire then in future you will need to go back and do this step again each time the secret reaches expiry date. Do NOT send this secret to SWOOP support, only install it yourself.
- Send the "Directory (tenant) ID" to SWOOP support so that we can create an Analytics Instance for you and send you the miner script.
2. Submit Microsoft Request Form
The on-premise data miner requires access to what Microsoft refers to as 'protected APIs'. To request access to these protected APIs, Microsoft requires that a request form is submitted for their approval. Please see the link below:
"Microsoft Teams APIs in Microsoft Graph that access sensitive data are considered protected APIs. These APIs require that you have additional validation, beyond permissions and consent, before you can use them."
Please fill in the form and include firstname.lastname@example.org and your own email address in question. Please email email@example.com with a copy of the form so we can follow up with Microsoft.
For question 6 we suggest the following answer:
The application is the backend part of https://appsource.microsoft.com/en-us/product/office/WA200000877 (also see question 7)
For question 7 we suggest the following answer:
The application is a local implementation of this: https://appsource.microsoft.com/en-us/product/office/WA200000877. We are not using the multi-tenant app for privacy reasons (we are hosting the application instead of SWOOP). Contact firstname.lastname@example.org for more information if required.
We are happy to help you fill in the form if required.
3. General Miner Installation
You must install your own Virtual Machine, typically inside Microsoft Azure, although other host platforms also work.
- Minimum of 10 GB Server Disk
- Minimum 32 GB Data Disk (script required /dev/sdc).
- RHEL Linux 7 or CentOS 7 or RHEL Linux 8 or CentOS 8 (use latest update of major version).
- Minimum 1.75 GBytes Memory
- Minimum CPU Requirements: Azure "Standard A1" class single CPU, not throttled, or equivalent on other hosting platforms.
- Host name for mining server (e.g. miner-examplecorp.swoopanalytics.com)
- SSL/TLS Configured on server (certificates must be supplied by end-user customer).
- Firewall (in Azure this is called Network Security Group) needs to open inbound HTTPS (port 443), and usually also SSH (port 22) but these should be restricted to trusted networks.
- Firewall allows outbound access to Microsoft Graph API Servers (can be via proxy server)
We will provide you with a script to configure the instance. The script should be run as user 'root' after the operating system is installed. If you are using Azure, the "Run command" option from the "Operations" menu can run the script once the VM is running (paste the script into the Azure portal). Alternatively it can be run by connecting via SSH and using the sudo operation (copy the script into your VM). Take note of the output from the script operation, it may be useful in future.
This script has a number of parameters at the top of the file which must be filled in. You will need to provide values as per below:
URL for your allocated SWOOP Analytics domain name, including https prefix
Azure Application (client) ID (see above).
Matching secret for ClientID above. Only used for miner, keep this secure.
Azure Directory (tenant) ID (see above).
The URL for the miner server VM, including https prefix
Blank for none or host and address for HTTPS proxy in format 'host:port'.
|OS||Either 'CENTOS' or 'RHEL' (for Redhat Enterprise) or 'OTHER' for generic install on a system that supports yum.|
|PASSPHRASE||A passphrase provided separately to decode the content contained in the script. Please Note: The code provided contains the credentials to update the SWOOP Analytics server. Keep it secure.|
When you run the script it will perform the following action (see comments in script):
- Install package groups to support application, mainly the Apache web server with PHP support.
- Configure PHP
- Install files necessary to run the application
- Install log rotator for miner
- Install miner configuration file
- Install miner scheduler (runs hourly)
- Install miner code downloader in web server.
These options can be changed by editing the /var/www/html/config.php at a later stage.
You may like to allocate a separate mount point (on separate disk) for the miner data (so you can snapshot and restore separately). All the data is stored by default in '/data' for AWSLINUX and '/var/www/storage' for RHEL.
Please ensure internal linux firewall settings (for network and machine) allow access to the web server on your server (e.g. for CENTOS firewall-cmd --zone=public --permanent --add-service=http;firewall-cmd --reload ).
Miner SSL/TLS configuration
SSL/TLS must be configured by the end-user, including providing certificates. These are available from various sources, including self-signed if that is sufficient, or Let's Encrypt, or other commercial server certificate providers. Full details are beyond the scope of this document.
Remember that DNS name of the mining server should match the certificates, as per normal webserver setup (see REDIRECT configuration setting above).
Once you have installed the script you will need to connect to the miner from your web browser. To do this start up the root page by going to the 'REDIRECT' Url from above.
- Download the latest version of the yammer miner from the SWOOP Analytics engine (this is transparent). If your proxy is not configured properly you will find out here.
- Provide a login button.
- Test the Microsoft Graph API connection and display a message that updates can proceed.
After a login, you should see that it detects the application setup and consent process worked correctly, by showing your user name in the top, right corner and also shows correct status like so.
Updates will then be scheduled on an hourly basis and no other action is required. At the bottom of the miner screen, the activity log can be loaded by clicking the "Refresh" button in the corner of that box. This shows a sample of some miner logs.
Enabling and Configuring Sentiment Analysis
If your SWOOP subscription includes sentiment analysis, then add the following configuration for the miner by editing the '/var/www/html/config.php' file:
$config->SentimentAPI = 'https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/sentiment';
$config->SentimentAPISubscription = 'xxxxxxxxxxxxxx';
$config->SentimentEnabled = true;
Replace x's with your own Azure Cognitive Services Subscription Key via the Microsoft Azure Portal. Click on "Show Access Keys" and then look at the hex value in "Key1".
Upgrading the Miner
The miner can be upgraded by logging to the miner and clicking the 'Update Software'.
If you are on the server console the miner can be updated with the following sequence of commands:
The two 'grep's will show you the version number changing.
Making the URL to the Miner accessible in SWOOP
It can be very helpful for other Yammer or SWOOP admins to be able to access the miner, and it is possible to store the URL to the on-premise data miner within the SWOOP end-user interface. To do this follow these steps:
- Log on to SWOOP
- Click 'Admin' in the footer
- Click on the 'Miner' tab
- Update the URL (by default it will simply have "miner-" prefix in front of instance name).
Note - only Azure admins will be able to access the SWOOP data miner.