Immersitech Engage™Freeswitch-Adapter
Immersitech Engage™ Adapter for Freeswitch is a plugin that enables Immersitech Engage Core sound processing inside your Freeswitch instance. With just a few simple steps, you will be able to hear clear spatial audio in your own Freeswitch project.
Prerequisites
Installation of Linux
Immersitech Engage™ Adapter for Freeswitch V1.0 is developed for Freeswitch V1.10, which runs well in Ubuntu 18.04.3. Debian 10 is also recommended for use with Freeswitch V1.10.
Installation of Freeswitch
Freeswitch V1.10.X is supported. Please find documentation of how to install Freeswitch in document below:
https://freeswitch.org/confluence/display/FREESWITCH/Installation
Install Immersitech Engage™ Adapter for Freeswitch
sudo mkdir /usr/lib/immersitech
-
immersitech_license_key.dat- This is the license file for the Immersitech Engage functionality. Note that if you do not have a license or your license expires, the program will still run except that features of the library will be permanently bypassed, effectively functioning like normal freeswitch without the Immersitech libraries. -
room_config.json- This is the default room configuration file in JSON format. You can find more details about this file and how to customize your own room configurations later in the documentation. -
imm_websocket_config.json- This is the default websocket configuration file. You can find more details about this file and how to custom it to work with your own server in the event manager documentation. -
libimmersitech-sound-manager.so- The Immersitech Sound Manager library file -
libimmersitech-room-manager.so- The Immersitech Room Manager library file -
libimmersitech-event-manager.so- The Immersitech Event Manager library file -
libimmersitech-engage-core.so- The Immersitech Engage Core library file
Note that the default search path for all Immersitech core libraries is set to be /usr/lib/immersitech, though you can manually change the runtime link path of a shared object.
-
mod_conference.so- This is a modified mod_conference module that has the necessary modifications to connect Freeswitch to the Engage Core -
mod_imm_adapter.so- This is the FreeSwitch Adapter library
imm_adapter.conf.xml- This is the configuration file for the Immersitech libraries. Learn more about it in the Immersitech Configuration File section of this documentation.
By default, only a few freeswitch modules are enabled. You can enable/disable these modules, including the Immersitech modules, in a configuration file. Edit /usr/local/freeswitch/conf/autoload_configs/modules.conf.xml and change as per your requirements. Please make sure the following Immersitech modules are loaded automatically by adding these calls to the file:
<load module="mod_conference"/>
<load module="mod_imm_adapter"/>
Make sure that your freeswitch dial-plan uses conference profiles in conf/autoload_configs/conference.conf.xml that are set up to utilize 2 channels output and match the settings in your imm_adapter.conf.xml file.
The following are the settings you must guarantee will be used per your freeswitch dial-plan:
<param name="rate" value="48000"/> <!-- Must match "imm_spatial_output_audio_sample_rate" -->
<param name="channels" value="2"/> <!-- Must be two channel output to hear 3D audio -->
<param name="interval" value="20"/> <!-- Must match "imm_spatial_output_audio_frames_per_buffer" -->
Run Freeswitch with Engage Freeswitch Adapter
- Start Freeswitch, load the Adapter module if not loaded automatically:
cd /usr/local/freeswitch/bin
./freeswitch
load mod_imm_adapter
The following log will be shown if Freeswitch started and the Adapter module is loaded successfully:
Trying to load immersitech conference at: ../mod/libimmersitech.so
The Immersitech Conference library version is v0.08.012
2020-04-08 15:17:09.731354 [INFO] mod_imm_adapter.c:191 Subscribed to conference custom events!
2020-04-08 15:17:09.731354 [CONSOLE] switch_loadable_module.c:1804 Successfully Loaded [mod_imm_adapter]
2020-04-08 15:17:09.731354 [NOTICE] switch_loadable_module.c:412 Adding API Function 'imm_adapter'
+OK Reloading XML
+OK
- Optionally, you can view Version information about your product by typing following command into a Freeswitch Command Line Interface:
imm_adapter get_version
The output from this command which look something like the following:
Engage Adapter Version: v0.0.0.3
Engage Core Version: v0.0.0.2
Sound Manager Version: v0.08.012
- To configure SIP softphones (Linphone or Zoip) to register to Freeswitch and let softphones join to the same spatial conference, dial a number in the range of “3500 - 3599” (which is the default stereo conference number in the Freeswitch dial plan). Spatial audio should be heard once they have joined the conference successfully.
Default Behavior
By default, a new participant will enter a round table room with 8 fixed seats. You can view the Default room layout (labeled as “default”) by using following command:
imm_adapter show_room_configs
The spatial position of any participant can be viewed by using the show_rooms commands. All new participants will be assigned to specific seat one by one starting from index 1:
imm_adapter show_rooms
Any participant can be moved to any seat, resulting in a change in their spatial sound field. You can move a participant to a new seat with the move_seat command. The following is an example of a participant with id 1 that will be moved to the seat with index 2 in conference 3500:
imm_adapter move_seat 3500 1 2
By default, a new participant joins with all audio effects enabled and their device set to headphones. You can change or view the state of any participant in real time. There are several state changes you can make detailed further in the API section. For example, let us use the set_participant_state function to turn on Automatic Gain Control for participant 2 in conference 3500:
imm_adapter set_participant_state 3500 2 IMM_CONTROL_AGC_ENABLE 1
For detailed API commands explanation, please refer to the following API commands chapter.
Immersitech Configuration File
You will notice above you moved an xml configuration file imm_adapter.conf.xml for freeswitch to setup the immersitech libraries with. Below we will explain what each configuration option can do for you, although it is recommended to leave both the immersitech and freeswitch configuration as our default recommendations.
License_file_path
This parameter allows you to tell the immersitech library where to find your license file. Note if you use a relative path, it will be relative to the location of mod_imm_adapater.so.
room_config_file_path
This parameter allows you to tell the immersitech library where to find your room config file. Note if you use a relative path, it will be relative to the location of mod_imm_adapater.so.
websocket_config_file_path
This parameter allows you to specific where your websocket config file is located. Note if you use a relative path, it will be relative to the location of mod_imm_adapter.so.
websocket_enabled
This parameter specifies whether or not the websocket event management system will be active or not. If you want to use the websocket server to send and receive events, then you should set this parameter to 1. If you do not want to have the websocket server active then you should set this parameter to 0.
imm_spatial_output_audio_sample_rate
This parameter describes the sampling rate of the audio coming out of the immersitech libraries. Currently Immersitech only supports 48 kHz output sampling rates.
imm_spatial_impulse_length
This parameter allows you to make a tradeoff between CPU and the accuracy of 3D rendering. The smaller the value, the less CPU that is used, but the less accurately the 3D image is rendered. You can pick between 512, 1024, 2048, 4096, or 8192.
imm_spatial_output_audio_frames_per_buffer
This parameter relates directly to the “interval” parameter defined in your mod_conference configuration. If you have the mod_conference parameter “interval” set to 10, then you want to set this parameter to 480 because 480 frames at 48 kHz sampling rate is 10 milliseconds. If you have the mod_conference parameter “interval” set to 20, then you want to set this to 960 because 960 frames at 48 kHz is 20 milliseconds. Currently, the Immersitech libraries only support either 480 or 960, please send us a request if you need a different interval.
imm_max_string_length
This parameter states the largest number of characters that will be assumed when saving data to or from strings. For example, if you request to see the current rooms in JSON form, the Immersitech library will provide the JSON string up to imm_max_string_length. Additionally, when the Immersitech library reads in your room_config.json file, it will only read it in up to imm_max_string_length. Most likely the default value of 10000 will be suitable, but if you find yourself using extremely large room config files or hosting very large numbers of participants than you may need to increase this value.
Engage Freeswitch Adapter API commands:
Once the Immersitech Engage Freeswitch Adapter is running, you can apply several API commands in real-time to change the state of the audio effects. You can apply these commands manually through the freeswitch command line (fs_cli) or programmatically through a freeswitch interface. To display a list of all commands available to you in the Immersitech Engage Freeswitch Adapter, enter the command “imm_adapter” and the list of API commands within the adapter will be listed as below:
imm_adapter
Usage:
get_version
show_room_configs
show_rooms
show_room_by_id <room_id>
move_seat <room_id> <participant_id> <seat_index>
change_room_config <room_id> <room_config_id>
set_participant_state <room_id> <participant_id> <control_to_edit> <value>
get_participant_state <room_id> <participant_id> <control_to_edit>
set_all_participants_state <room_id> <control_to_edit> <value>
get_all_participants_state <room_id> <control_to_edit>
get_defined_states
Get Version Information
The command “get_version” will display detailed version information as below:
imm_adapter get_version
Engage Adapter Version: v0.0.7
Engage Core Version: v0.0.0.7
Sound Manager Version: v0.11.004
Show Room Configs
The command “show_room_configs” will display all the room templates defined in the room config json file deployed in freeswitch (room_config.json).
Type “imm_adapter show_room_configs” and the room configurations can be seen as below:
Room configurations: {
"default": "room1",
"room_list": {
"room1": {
"allowParticipantStacking": true,
"desc": "Eight person conference room with round table.",
"id": 1,
"seats": [
{
"x": 0,
"y": 0,
"z": 110
},
{
"x": 78,
"y": 0,
"z": 78
}
],
"stackDelta": 10
},
"room2": {
"allowParticipantStacking": false,
"desc": "Fifteen person classroom with instructor at front. All students at same level; no stadium seating.",
"id": 2,
"seats": [
{
"x": 991,
"y": 992,
"z": 993
},
{
"x": 994,
"y": 995,
"z": 996
},
]
}
}
}
Room Configuration Explanation
Room config contains a list of room templates used to create/update room layout. it contains the following components stored in a json file:
-
Default Template name which will be used when new room is created, such as example as below:
"default": "room1" -
List of room templates, each template contains following components:
-
Room name, example as below::
"room1": {...} -
Room id, example as below:
"id": 1 -
Flag to indicate if the room will stack participants when user count exceeds room seat capacity. If stacking is allowed, multiple participants can share the same seat with vertical distance defined in “stackDelta”, example as below:
“allowParticipantStacking”: True "stackDelta": 10If participant count exceeds the room capacity and stacking is not allowed, new participants will be allocated to position (0, 0, 0).
-
Description of room template, example as below:“desc”: “Eight person conference room with round table.”
-
Seat list with position data, example as below:
"seats": [ { "x": 0, "y": 0, "z": 110 }, { "x": 78, "y": 0, "z": 78 }, … … ]
-
Show Active Rooms
Enter the “imm_adapter show_rooms” command and a list of all active rooms (a room with at least a single user in it) will be displayed as below:
imm_adapter show_rooms
Rooms as Json:
{
"rooms": [
{
"allowParticipantStacking": "true",
"desc": "",
"id": "3500",
"name": "freeswitch conference",
"seats": [
{
"participantId": "4",
"participantName": "1010",
"seatId": 2,
"x": 78,
"y": 0,
"z": 78
},
{
"participantId": "3",
"participantName": "1009",
"seatId": 1,
"x": 0,
"y": 0,
"z": 110
}
]
}
]
}
Show Active Room by Room Id
Type the “imm_adapter show_room <room_id>” command and rooms can be seen as below:
imm_adapter show_room 3500
Room as Json:
{
"rooms": [
{
"allowParticipantStacking": "true",
"desc": "",
"id": "3500",
"name": "freeswitch conference",
"seats": [
{
"participantId": "4",
"participantName": "1010",
"seatId": 2,
"x": 78,
"y": 0,
"z": 78
},
{
"participantId": "3",
"participantName": "1009",
"seatId": 1,
"x": 0,
"y": 0,
"z": 110
}
]
}
]
}
Move Seat
Type “move_seat <room_id> <participant_id> <seat_index>” command to move participant’s seat to seat <seat_index>. You must enter a seat index that has been defined in the room configuration file.
imm_adapter move_seat 3500 1 2
If you try to move a participant to a seat that is already occupied, behavior is defined above in the room configuration section under Participant Stacking and Stacking Delta.
Change Room Configuration
Type “change_room_config <room_id> <room_config_id>” command to change room configuration, room config ids are defined in the room config file.
For example changing from round table to classroom table:
imm_adapter change_room_config 3500 2
Get Defined Participant States
Type “imm_adapter get_defined_states” command to get all defined participant states.
imm_adapter get_defined_states
IMM_CONTROL_STEREO_BYPASS_ENABLE | IMM_CONTROL_MUTE_ENABLE | IMM_CONTROL_ANC_ENABLE | IMM_CONTROL_AGC_ENABLE | IMM_CONTROL_MIXING_3D_ENABLE | IMM_CONTROL_DEVICE | IMM_CONTROL_HALF_SPAN_ANGLE | IMM_CONTROL_MASTER_GAIN
You can find a description of the effects of each participant state is below:
| Control | Description |
|---|---|
| IMM_CONTROL_STEREO_BYPASS_ENABLE | If enabled, this state will cause all of the following effects to be bypassed for this participant, regardless of whether or not they are enabled. Possible Values: 0 (disabled) or 1 (enabled). Default Value: 0 (disabled). |
| IMM_CONTROL_MUTE_ENABLE | If enabled, this state will prevent this participant’s input audio from entering the conference. Possible Values: 0 or 1. Default Value: 0. |
| IMM_CONTROL_ANC_ENABLE | If this state is enabled, noise cancellation will be activated and the listener will hear the other speakers more clearly. Possible Values: 0 or 1. Default Value: 1. |
| IMM_CONTROL_AGC_ENABLE | If this state is enabled, automatic gain control will be activated and the listener will the other speakers at a consistent volume level. Possible Values: 0 or 1. Default Value: 1. |
| IMM_CONTROL_MIXING_3D_ENABLE | If enabled, this state will allow the participant to hear all the other speakers in 3D according to their respective seat positions. Possible Values: 0 or 1. Default Value: 1. |
| IMM_CONTROL_DEVICE | This control allows a user to optimize processing for their type of listening device. Possible Values: IMM_DEVICE_HEADPHONE or IMM_DEVICE_SPEAKER. Default Value: IMM_DEVICE_HEADPHONE. |
| IMM_CONTROL_HALF_SPAN_ANGLE | The angle between the participant’s head and the loudspeaker to their right. Only applies to a listener using stereo loudspeakers. Possible Values: 1 to 90. Default Value: 15. |
| IMM_CONTROL_MASTER_GAIN | This state defines the gain of this participant’s output audio as a percentage. 100 will result in the loudest audio and 0 will result in silence. Possible Values: 0 to 100. Default Value:100. |
Set Participant State
Type “set_participant_state <room_id> <participant_id> <control_to_edit> <value>” command to change participant state.
Below is an example of enabling AGC:
imm_adapter set_participant_state 3500 2 IMM_CONTROL_AGC_ENABLE 1
Get Participant State
Type “get_participant_state <room_id> <participant_id> <control_to_edit>” command to get specific participant state:
imm_adapter get_participant_state 3500 2 IMM_CONTROL_AGC_ENABLE
Get participant 2 state: IMM_CONTROL_AGC_ENABLE value: 1 in room: 3500.
Set All Participants State
Type “set_all_participants_state <room_id> <control_to_edit> <value>” command to change state for all participants.
Below is an example of enabling AGC for all participants in room 3500:
imm_adapter set_all_participants_state 3500 IMM_CONTROL_AGC_ENABLE 1
Get All Participants State
Type “get_all_participants_state <room_id> <control_to_edit>” command to get specific state of all participants:
imm_adapter get_all_participants_state 3500 IMM_CONTROL_AGC_ENABLE
Get states:3
participant id:1 participant name:1010 value:0
participant id:0 participant name:1009 value:0
in room: 3500.
ESL Event from Immersitech Engage Freeswitch Adapter
Specific ESL events will be fired after a certain operation executed by the Immersitech Engage Freeswitch Adapter.
Event id and cdr defined as below:
#define IMM_EVENT_MAINT "immersitech::maintenance"
#define IMM_EVENT_CDR "immersitech::cdr"
Events
All the possible ESL events that may be fired by the Immersitech Engage Freeswitch Adapter are detailed below:
Create Room Event:
Action: immersitech-create-room
Headers: Room-Id
Destroy Room Event:
Action: immersitech-destroy-room
Headers: Room-Id
Add Participant Event:
Action: immersitech-add-participant
Headers: Room_Id, Participant-Id, Participant-Name
Destroy Participant Event:
Action: immersitech-destroy-participant
Headers: Room_Id, Participant-Id
Move Participant Event:
Action: immersitech-move-participant
Headers: Room_Id, Participant-Id, Seat-Id, Seat-Position-X, Seat-Position-Y, Seat-Position-Z
Set Participant State Event:
Action: immersitech-set-participant-state
Headers: Room_Id, Participant-Id, State-Id, State
Set All Participants State Event:
immersitech-set-all-participants-state
Headers: Room_Id, State-Id, State
Change Room Config Event:
Action: immersitech-change-room-config
Headers: Room_Id, Room-Config-Id
Q & A
Why can’t I hear audio?
It might be because the modules were not loaded properly, please check:
⦁ If all files are in correct place defined in chapter “Install Immersitech Engage Freeswitch Adapter”
⦁ If all file names match those defined in the config file “imm_adapter.conf.xml”.
⦁ If all information are correct, please provide version information by using command as below:
imm_adapter get_version
Why doesn’t my audio sound different?
- There some possible causes for this issue, please check:
Is the license file still valid? Please check if any possible error messages showed up when loading mod_imm_adapter module:
⦁ "Unable to open License file"
⦁ "Your trial of the Immersitech library expired on 05/10/2019. Please contact Immersitech to renew your library."
⦁ "Your trial of the Immersitech library does not match your allowed versions. You are trying to use version v3.03.002 of the Immersitech library. However this license only allows use of versions between v1.01.002 and v1.04.001."
- Is the state IMM_CONTROL_MIXING_3D_ENABLE enabled?
Please check if all user’s state IMM_CONTROL_MIXING_3D_ENABLE enabled by using command as below, 3500 is example of room id:
imm_adapter get_all_participants_state 3500 IMM_CONTROL_MIXING_3D_ENABLE
And please enable IMM_CONTROL_MIXING_3D_ENABLE by using command as below, 3500 is example of room id:
imm_adapter set_all_participants_state 3500 IMM_CONTROL_MIXING_3D_ENABLE 1
- Please check users’ seat position in room by using command as below (could be right across from each other), 3500 is example of room id:
imm_adapter show_room_by_id 3500
Why does my audio sound low?
Please check if all user’s state IMM_CONTROL_AGC_ENABLE enabled by using command as below, 3500 is example of room id:
imm_adapter get_all_participants_state 3500 IMM_CONTROL_AGC_ENABLE
And please enable IMM_CONTROL_AGC_ENABLE by using command as below, 3500 is example of room id:
imm_adapter set_all_participants_state 3500 IMM_CONTROL_AGC_ENABLE 1
v4