Server status:
  USA
  Europe
  Germany
  Canada
  Australia

Articles in this section

On-Premise Data Miner - Microsoft Teams

Avatar
Telford Tendys
  • Updated
Follow

The following steps are required to completely configure our on-premise SWOOP Miner for Microsoft Teams.

Prerequisites

In order to provide you with the credentials we will require a mobile (or SMS friendly) number to send you an encryption key. Please provide this when requesting the script described below.

1.  Azure Application Creation

The application for the miner will need to be created under Azure Active Directory, with suitable permissions. In addition the application ID needs to be submitted to Microsoft for white-listing in order to do data mining in Microsoft Teams. The miner runs continuously in the background, without a user login. These steps explain how to create the app:

  1. Go to the Azure Portal.
  2. Select "Azure Active Directory" menu item from the left hand side menu.
  3. Select "App registrations" from the Azure Active Directory sub-menu.
  4. Press the button "New registration" at the top.
  5. Give the app a name, such as "SWOOP Miner for Microsoft Teams" and as a standard SWOOP uses the the redirect URL  https://<your server address>/ which ends up redirecting again back to your REDIRECT value (see table below).

    app_reg_1.png
  6. Click the "Register" button at the bottom.
  7. After the registration is done, click the "View API Permissions".
  8. Click "Add a Permission".
  9. Click into the permission section called "Microsoft Graph".
  10. Under the "Delegated Permissions" you should see User.Read already ticked so leave that.
  11. Under the "Application Permissions" tick the following four permissions only:
    • ChannelMessage.Read.All
    • Directory.Read.All
    • Group.Read.All
    • User.Read.All
    • Reports.Read.All    (report statistics for Calls / Chats / Meetings)
  12. Click "Add Permissions" at the bottom.
  13. Once the permissions are saved, click the "Grant admin consent" button. The result should look like this:

    miner_permissions.png

  14. Click into "Overview" at top of sub-menu.
  15. Take a note of the UUID under "Directory (tenant) ID" 
  16. Take a note of the UUID under "Application (client) ID"
  17. Click "Certificates & Secrets" in the sub-menu.
  18. Use the "New client secret" button and follow the steps to get a secret. Keep a note of the secret because it only will show one time. NOTE: if you set it to expire then in future you will need to go back and do this step again each time the secret reaches expiry date. Do NOT send this secret to SWOOP support, only install it yourself.
  19. Send the "Directory (tenant) ID"  to SWOOP support so that we can create an Analytics Instance for you and send you the miner script.

2.  Submit Microsoft Request Form

The on-premise data miner requires access to what Microsoft refers to as 'protected APIs'. To request access to these protected APIs, Microsoft requires that a request form is submitted for their approval. Please see the link below:

"Microsoft Teams APIs in Microsoft Graph that access sensitive data are considered protected APIs. These APIs require that you have additional validation, beyond permissions and consent, before you can use them."

Please fill in the form and include support@swoopanalytics.com and your own email address in question. Please send a copy of the form at https://support.swoopanalytics.com/hc/en-us/requests/new so we can follow up with Microsoft.  

For question 6 we suggest the following answer:

The application is the backend part of https://appsource.microsoft.com/en-us/product/office/WA200000877 (also see question 7)

For question 7 we suggest the following answer:

The application is a local implementation of this: https://appsource.microsoft.com/en-us/product/office/WA200000877. We are not using the multi-tenant app for privacy reasons (we are hosting the application instead of SWOOP). Contact support at https://support.swoopanalytics.com/hc/en-us/requests/new for more information if required.

We are happy to help you fill in the form if required.

3.  General Miner Installation

You must install your own Virtual Machine, typically inside Microsoft Azure, although other host platforms also work.

System Requirements:

  • Minimum of 10 GB Server Disk
  • Minimum 32 GB Data Disk (script required /dev/sdc).
  • RHEL Linux 7 or CentOS 7 or RHEL Linux 8 or CentOS 8 (use latest update of major version).
  • Minimum 1.75 GB Memory
  • Minimum CPU Requirements: Azure "Standard A1" class single CPU, not throttled, or equivalent on other hosting platforms.
  • Host name for mining server (e.g. miner-examplecorp.swoopanalytics.com)
  • SSL/TLS Configured on server (certificates must be supplied by end-user customer).
  • Firewall (in Azure this is called Network Security Group) needs to open inbound HTTPS (port 443), and usually also SSH (port 22) but these should be restricted to trusted networks.
  • Firewall allows outbound access to Microsoft Graph API Servers (can be via proxy server)

We will provide you with a script to configure the instance. The script should be run as user 'root' after the operating system is installed. If you are using Azure, the "Run command" option from the "Operations" menu can run the script once the VM is running (paste the script into the Azure portal). Alternatively it can be run by connecting via SSH and using the sudo operation (copy the script into your VM). Take note of the output from the script operation, it may be useful in future.

This script has a number of parameters at the top of the file which must be filled in. You will need to provide values as per below:

Configuration Parameter

Value

NAME

URL for your allocated SWOOP Analytics domain name, including https prefix
e.g. https://examplecorp.swoopanalytics.com/

CLIENT_ID

Azure Application (client) ID (see above).

CLIENT_SECRET

Matching secret for ClientID above. Only used for miner, keep this secure.

NETWORK

Azure Directory (tenant) ID (see above).

REDIRECT

The URL for the miner server VM, including https prefix
i.e. https://<your server address>/ 

PROXY

Blank for none or host and address for HTTPS proxy in format 'host:port'.
OS Either 'CENTOS' or 'RHEL' (for Redhat Enterprise) or 'OTHER' for generic install on a system that supports yum.
PASSPHRASE A passphrase provided separately to decode the content contained in the script. Please Note: The code provided contains the credentials to update the SWOOP Analytics server. Keep it secure.

When you run the script it will perform the following action (see comments in script):

  1. Install package groups to support application, mainly the Apache web server with PHP support.
  2. Configure PHP
  3. Install files necessary to run the application
  4. Install log rotator for miner
  5. Install miner configuration file
  6. Install miner scheduler (runs hourly)
  7. Install miner code downloader in web server.

These options can be changed by editing the /var/www/html/config.php at a later stage.

You may like to allocate a separate mount point (on separate disk) for the miner data (so you can snapshot and restore separately).   All the data is stored by default in '/data' for AWSLINUX and '/var/www/storage' for RHEL.

Please ensure internal linux firewall settings (for network and machine) allow access to the web server on your server (e.g. for CENTOS   firewall-cmd --zone=public --permanent --add-service=http;firewall-cmd --reload ).

Miner SSL/TLS configuration

SSL/TLS must be configured by the end-user, including providing certificates. These are available from various sources, including self-signed if that is sufficient, or Let's Encrypt, or other commercial server certificate providers. Full details are beyond the scope of this document.

Remember that DNS name of the mining server should match the certificates, as per normal web server setup (see REDIRECT configuration setting above). 

Miner Operation

Once you have installed the script you will need to connect to the miner from your web browser. To do this, start up the root page by going to the 'REDIRECT' Url from above.

This will:

  1. Download the latest version of the Teams miner from the SWOOP Analytics engine (this is transparent).  If your proxy is not configured properly you will find out here.
  2. Provide a login button.
  3. Test the Microsoft Graph API connection and display a message that updates can proceed.

After a login, you should see that it detects the application setup and consent process worked correctly, by showing your user name in the top, right corner and also shows correct status like so.

teams_permissions_granted.png

Updates will then be scheduled on an hourly basis and no other action is required. At the bottom of the miner screen, the activity log can be loaded by clicking the "Refresh" button in the corner of that box. This shows a sample of some miner logs.

teams_miner_log.png

Enabling and Configuring Sentiment Analysis

If your SWOOP subscription includes sentiment analysis, then add the following configuration for the miner by editing the '/var/www/html/config.php' file:

$config->SentimentAPI = 'https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/sentiment';
$config->SentimentAPISubscription = 'xxxxxxxxxxxxxx';
$config->SentimentEnabled = true;

Replace x's with your own Azure Cognitive Services Subscription Key via the Microsoft Azure Portal. Click on "Show Access Keys" and then look at the hex value in "Key1".

Updating Miner Configuration

The configuration for the miner by editing the '/var/www/html/config.php'.   This can be used to update the parameters set in '3.  General Miner Installation'.

Upgrading the Miner

Please select one of the options below depending on convenience:

Option 1 - Via the web browser

The miner can be upgraded by logging to the miner and clicking the 'Update Software':

MicrosoftTeams-image.png

The new version of the miner will be shown when the screen refreshes.

Option 2 - Via the server console or ssh

If you are on the server console the miner can be updated with the following sequence of commands:

sudo su -s /bin/bash apache
cd /var/www/html
grep 'define.*MINER_VERSION' index.php
php update_sig.php
grep 'define.*MINER_VERSION' index.php

The two 'grep's will show you the version number changing.

 

Making the URL to the Miner accessible in SWOOP

It can be very helpful for other Yammer or SWOOP admins to be able to access the miner, and it is possible to store the URL to the on-premise data miner within the SWOOP end-user interface. To do this follow these steps:

  1. Log on to SWOOP
  2. Click 'Admin' in the bottom left fly-out menu of SWOOP
  3. Click on the 'Miner' dashboard
  4. Update the URL (by default it will simply have "miner-" prefix in front of instance name).

Note - only Azure admins will be able to access the SWOOP data miner.

Related articles

Comments

0 comments

Please sign in to leave a comment.