Voice Troubleshooting

This document provides troubleshooting guidance for the voice feature of MindLink web client, first introduced with version 18.7.

It is broken down into three sections: a section on local client setup problems such as access to microphone devices. A section on how to get voice diagnostic error information from the MindLink client and server can help identify the nature of the problem. A section on troubleshooting networking issues between different client endpoints and supporting servers. Finally, a section on compatibility with other clients (such as the Microsoft SfB desktop client).

Voice calling in MindLink occurs over two separate sessions: a session with the SfB server, and a direct connection between the client endpoints where the audio media is streamed between the clients. Under certain network conditions a direct connection between the client endpoints is not possible and additional server components are required to support these cases, see the voice feature technical overview document for more information.

Diagnosing common local issues#

This section details various issues with client setup, as well as known issues and limitations.

Microphone/recording device access#

The MindLink web client will attempt to use the default audio recording and reproduction device available on the machine it is running on. There are several issues to take into account in these cases:

  1. Missing audio recording device (microphones): the client will fail to capture user audio and will abort the call. The client should display a warning after logging on if it cannot find a microphone device. In modern browsers, it is necessary to grant audio recording permissions from the browser tab in order for the voice feature to work. Enterprise policies on granting such permissions may prevent the feature from working.
  2. Occupied audio recording device: a common pitfall is attempting to establish a call between two client endpoints from the same machine (different browser tabs, different browsers etc), microphone access is often exclusive, so if there is a browser or other application currently capturing microphone input, then attempting to access the device from another application will result in an error or incorrect behaviour. This can also occur when two remote desktop sessions are being used for testing, each running from the same physical machine.

It is therefore recommended that basic feature testing and setup is performed using two separate physical machines, each with their own recording and reproduction devices, without making use of remote desktop.

Cannot select recording/reproduction device#

There is currently no support for manually selecting which reproduction/recording device to use. The MindLink client will use the system default devices.

Internet Explorer cannot perform calls#

The MindLink client uses the webrtc API in modern browsers to perform calls, voice calling is supported in Internet Explorer 10 and 11 which do not implement this modern API by using Microsoft's "Skype Meetings App" (also known as "Skype for Business Web App Plugin") browser plugin (see the voice feature technical overview document for more information). This plugin must be separately installed on the same machine running Internet Explorer for voice calling to work. A warning should be shown after logging on in case the client cannot detect the plugin. In case the plugin is installed while the client is open, the client must be restarted by refreshing the page or logging out after installing the plugin.

Outdated and unsupported versions of this plugin are distributed with the SfB 2015 server installation but will not work with MindLink web client, the version required by MindLink web client can be found at: https://swx.cdn.skype.com/s4b-plugin/

Windows OS edition N, KN and similar#

Editions N (and related) of Windows require the appropriate media pack for the OS version in order for Firefox and Internet Explorer to be able to capture audio. For more information see: https://support.microsoft.com/en-gb/help/3145500/media-feature-pack-list-for-windows-n-editions.

Firefox version 63#

Version 63 of Firefox introduced a bug with the webrtc API, this affects MindLink web client when receiving calls from Internet Explorer. This does not occur on Firefox62 and lower, and was fixed for Firefox64 and higher, see here for more details: https://bugzilla.mozilla.org/show_bug.cgi?id=1495569.

Edge browser does not use STUN/TURN server#

STUN servers are not supported by the Edge browser. TURN servers are supported, but only under the following conditions:

There are two issues with how the Edge browser handles TURN server addresses. First of all Edge only supports TURN over UDP, and cannot negotiate with a TURN server over TCP. Secondly, the TURN server URL configured in the MindLink server management centre must conform to the following syntax in order for Edge to parse it (this is not required for any other browser): tun:<server location>?transport=udp

Finally, the TURN server must be configured to reply with a REALM attribute other than the wildcard "*" (for example, an internal domain name). Configuration of this value depends on the TURN server implementation used.

Internet Explorer only uses SfB Edge server for the first call#

This is a known issue with version 18.7 of the MindLink client and should be fixed in version 18.8.

Unable to configure per-user STUN/TURN server credentials#

There is currently no support to configure per-user credentials. The credentials configured in the management centre will be used by all users connecting to that MindLink server when attempting to establish calls.

Multiparty voice conferencing support#

Multiparty voice is supported only for Internet Explorer (via the Skype plugin mentioned above) and Google Chrome.

Note that Chrome will only successfully join the conference if it can directly connect to the server hosting the SfB MCU. This is due to a bug where it ignores configured ICE servers and will be fixed in a future release.

Note that presenter-muting of participants on Chrome does not work correctly in 18.8 but will be fixed for the next release 19.1. Presenter-muting of a participant on IE should work.

This is because the SfB MCU only supports SDES transport security for the peer2peer connection, but most browsers only support DTLS in accordance with webrtc standards. Chrome only has legacy support for SDES.

Video call support#

There is currently no support for video calls, only voice calls are supported. This applies to both 1-1 and multiparty.

Putting calls on hold#

There is currently no support for putting calls on hold, or handling being put on hold by the remote peer.

Looking for the MindLink logs#

This section explains what to look for in the client and server logs to help diagnose voice issues.

Local client error#

Local client logs can help diagnose issues with the establishment of the peer to peer connection between client endpoints.

To enable client logging, add "?enableLogging=true" to the end of the browser tab URL (e.g. "https://mindlink.mycompany.com/?enableLogging=true"). Once enabled, use the browser developer tools (which can be accessed by pressing the F12 key in most browsers) and navigate to the "console" section of the developer toolsconsole-logs-edge.

Simply filtering the console messages via the developer tools (for Chrome and Firefox CTRL + F or cmd + F) looking for "voice" or "audio" should show the relevant log messages. Error messages will contain an error stack showing the original API error. Depending on the browser, this error will be generated by the webrtc API or by the Skype Internet Explorer plugin (see the voice technical overview document for more information), since the first is implemented in slightly different ways in every browser, errors will vary, but can contain useful information.

Remote client error#

If the error that prevented the call from establishing is generated on the peer client, an error will appear at the top of the client explaining the issue. Similarly, if the call is attempted and accepted on the peer end, but after a while fails then the client logs will contain something similar to the following: "Handling removal of current call 'some ID' by transitioning to ended". This line is an indication that the error occurred on the other client, and its logs should be inspected for errors or other messages. This could also indicate a failure of the peers to reach each other, indicating the need for a STUN/TURN server.

In certain situations the error will be not between the peer connection, but with the connection to the SfB server. In these cases the server logs may contain useful information.

The MindLink service logs are normally kept inside the MindLink server installation directory. It may be useful to ensure that the service is configured to log at "Verbose" (maximum) level. This can be changed from the management centre and requires a service restart.

Like with the client logs, filtering the log file for lines containing "voice" or "audio" should show the relevant log messages, again, errors will be logged with an error stack.

Diagnosing network issues#

Since the connection over which the audio is streamed between client endpoints is direct, there are several issues that can occur connecting two clients due to network architecture. This section explains how to troubleshoot peer connection and STUN/TURN server connections (see the voice technical overview for more information).

This section is divided between diagnosing direct or STUN-ip client connection and troubleshooting STUN/TURN server connection. The process essentially involves finding the IP address of the remote machine and filtering the network packets capture by this IP address.

The IP addresses will either be found in the call SDP documents (logged by the client and server), or will be the IP of the Edge or STUN/TURN server. Additionally, it is possible to filter for specific UDP or TCP ports which can help narrow down the range of filtered logs further, protocol, IP and port are all described in the SDP.

Peer connection troubleshooting using WireShark#

Wireshark is a free tool to analyse network traffic to/from the local machine. It is a very powerful and versatile tool.

In this section we will show examples of how to use Wireshark to diagnose network issues. The basic process to use in every case involves a few steps:

  • Identify the network device on the machine that will be actually used to attempt the connection, select this device as the Wireshark live capture target.

In general, this is the network interface on the local machine that is used to connect to the network, a WiFi device or ethernet controller.

  • Find the IP addresses of the remote call peer (the machine of the endpoint which is being called, or has started the call). These can be found from the MindLink client logs from the remote SDP document.

alt text

  • Filter Wireshark live capture with these IP addresses (assuming there is no other network traffic between the local machine and the IP this should show only the packets relevant to performing the call connection).

The screenshot shows a "sunny day" scenario where two local IP addresses establish a connection directly as they are both directly reachable.

alt text

If no traffic at all appears between the machine and the target IP address this means neither of the two is actually attempting the connection. Alternatively, requests sent from the local machine appear but no response can be seen from the peer's IP addresses. This can point to a firewall (on either host machine, or the NAT router) blocking the connection and/or that the target IP address belongs to a different private network from the local machine (in this latter case, a STUN/TURN server is required in order for calls to be routable between the endpoints).

alt text

If traffic appears but if only TCP RST (connection rejected) messages or ICMP "port unreachable" messages are shown this means the IP address was reached but actively rejected the connection. This means that either (1) the IP address on the network the machine belongs to maps to a different machine (i.e. the two machines are on different local networks, and it just happens to be that the target IP address corresponds to some other machine locally), in which case direct connection will fail and STUN/TURN is necessary, AND/OR that the target machine is rejecting the connection from local firewall software or similar.

Other ICMP error codes than "port unreachable" (the code and corresponding description should be shown in Wireshark) can indicate a different problem from the above, the code description should provide clues as to what the issue is.

Also note that it is normal to see TCP RST logged when both UDP and TCP were attempted and the UDP connection succeeds, as UDP is preferred to TCP by browsers and by the native SfB client and after the UDP connection succeeded the other connection attempts are aborted.

Standard STUN/TURN server setup troubleshooting#

Analysing STUN/TURN server setup via Wireshark involves more or less the same exact steps as analysing direct peer connection, except the IP address of the server will be used to filter the capture log.

alt text

The previous problematic cases apply here as well.

For each STUN/TURN server configuration added from the server management centre, the clients will attempt to establish a connection to the server every time a call is started or answered. This can lead to very long wait times as each server is contacted one after the other. It is recommended that no more than two servers are configured to keep call initialization time minimal.

SfB Edge server setup troubleshooting#

This section details how to troubleshoot configuring the Internet Explorer plugin to use the SfB Edge server. Running MindLink on Internet Explorer will attempt to use the SfB Edge server automatically. It will also attempt to use any STUN/TURN servers configured in the management too, but will fail to utilize them properly in case they do not support the MS-TURN protocol.

A token identifying the user endpoint with the SfB Edge server is retrieved from the MindLink server via the SfB frontend, this is then forwarded to the client and normally contains two separate addresses for the Edge server, one reachable from the internet and another reachable from the intranet. The MindLink web client attempts to use both and uses the first one that responds.

The process of client authentication against the SfB Edge server follows Microsoft's extension of the STUN protocol spec MS-TURN. This extension is incompatible with the standard protocol implemented by STUN/TURN servers and clients (such as webrtc clients implemented by Chrome and Firefox and TURN server implementations like coturn).

This means the SfB Edge server cannot be used by MindLink web client (webrtc) on modern browsers, but can be used by the Internet Explorer plugin as this was developed by Microsoft primarily for compatibility with SfB. For this reason, the plugin can only use the Edge server and cannot use standard STUN/TURN servers as it only supports the extended protocol. The MindLink web client cannot use the Edge server to receive the SfB client media because the SfB Edge server requires authentication via MS-TURN, however the SfB client can use a standard STUN/TURN server to receive the browser media when the TURN server does not require authentication from the receiving party (coturn by default, for example, only requires the sending party to authenticate).

Commercial STUN/TURN server implementations that support both MS-TURN as well as standard STUN/TURN and shim between can help alleviate these issues.

Again, the cases mentioned before apply to the SfB Edge server. In addition, here are shown cases displaying what happens when a webrtc client attempts to use the Edge server, and what happens when the Internet Explorer plugin attempts to use a standard TURN server instead of the Edge server.

alt text

The following shows that attempting to use a TURN server that does not support the MS-TURN extensions from the IE plugin results in rejection responses.

alt text

Client Compatibility#

This section details cross client compatibility between MindLink web client and other clients leveraging an on premise SfB deployment.

Skype for Business Windows Desktop Client#

The MindLink web client is supported against the Skype for Business 2016 desktop client versions 16.0.4639.1000 and higher. The client version can be found under "Help" -> "About Skype for Business" on the client itself (should also be visible under the installed programs view on Windows). Older versions of the Skype for Business client may or may not work.

Older clients will only work with the MindLink web client when this runs on Internet Explorer. The reason is that the only security mechanism supported by older client versions is not supported by the webrtc API in modern browsers (see the voice technical overview for more details).

Attempting to use an old Skype for Business client will result in the webrtc client rejecting the SDP due to this missing a fingerprint line or (equivalently) DTLS security details. This is because originally the client had no support for this security mechanism, but this was added in more recent versions. DTLS is the only mechanism modern browsers are meant to support.

Additionally, it is possible outdated clients will reject the SDP originated from the webrtc client with no explanation. This will likely show as a SIP 488 "Not acceptable here" error in the MindLink server logs.

Skype for Business MacOS Desktop Client#

The macOS version of the Skype for Business client is currently unsupported. Support may be investigated in the future.

Skype for Business Mobile Clients#

The Skype for Business Android and iOS mobile apps are only supported for inbound calls on modern browsers, this is due to the MS-TURN extensions to the STUN protocol (see above): the mobile client does not strictly follow MS-STUN when receiving a call, but strictly follow the protocol (which the native browser API does not support) when making a call outbound.

This results in the Skype for Business mobile client not responding to packets received from the web client on modern browsers, and can be observed via Wireshark.

MindLink mobile clients support#

There is currently no voice support for the MindLink mobile clients.

  1. The Edge and Internet Explorer browsers are very slow at displaying logs, and due to the volume of messages logged, it can take a couple of minutes after starting a call before the log messages and errors relating to the call appear in the logs. Furthermore, the Internet Explorer browser offers no logs filtering feature, however in Internet Explorer 11 it is possible to right click the console log and select "Copy all" to copy all log messages to a text editor or log monitor. Additionally, Internet explorer both tends to truncate long log messages, this means long SDP documents may be truncated and not all candidates will be visible in the client. These should however be fully logged by the server too. It is possible to cross-reference the call ID from the client logs with the server logs to find the full SDP documents for a given call.