NOTE: Due to some last-minute discovered bugs, there will be a delay of a few days in the release of the logging/replaying tool. This means that it will not be possible to log any data. However, if you wish to start setting up the architecture in your lab, you may start playing around with Kinect servers and X3D scenes, look into sample data/notebooks of mumodo that come from actual recordings, and patch/compile ELAN according to the instructions. The logger/replayer should be there by then. We deeply apologize for this slight delay.



The FAMEKinectServer software allows the transfer of information provided by the Microsoft Kinect SDK and the Microsoft Speech SDK to Linux and Windows Clients via network. Details concerning the implementation can be found in the code documentation (.../FAME Kinect Server/manual.pdf).


Get FAMEKinectServer for Windows 7. Make sure that all frameworks listed below are installed before running FAMEKinectServer setup file:

Execute FAMEKinectServer_v.2.3.3.exe and install it to a directory of your choice. Afterwards locate instantreality.jar (part of the InstantReality framework, usually found in C:\Program Files\Instant Reality\bin) and copy it into the installation directory of the FAMEKinectserver.

You can now start the server by using the link on your Desktop.

For further information and configuration see manual.pdf.



With our FAMEReplay tool you can both record and replay data. The data is stored in an xml-based file and you can easily load these so called .xio files from previous recordings and replay them with the very same software.


Download FAMEReplay tool.



This is just a small example for you to see how you can easily sent a string over network and visualize it in a X3D scene.


Download GenericInstantIOServer, extract to a folder of your choice and run /GenericInstantIOServer/examples/java/run.bat for Windows or for Linux.


[IMPORTANT NOTE: At the moment this is in an experimental state]


The goal of ELAN.mod is a minor change of ELAN source code in order to synchronously playback a recorded Instant Realtiy scene from the ELAN GUI. This enables the user to watch the actual video recorded in combination with a 3D scene from the InstantPlayer of the same recording.


Get the source code of ELAN annotation tool. Extract the archive to a folder of your choice and get our ELAN.mod patch. Integrate the folders from the into the ELAN source code folder and overwrite the Once this is finished, build your ELAN.mod with ant. Use a command line tool (eg windows command prompt), navigate to the ELAN source root folder and just type "ant", then it will use the existing build.xml to build your patched ELAN.


To configure ELAN.mod there is a file located in the folder structure. See at /src/config/ If there is noch such configuration file you can create on. Standart appearence would be as follows:

#configuration for rpc browser connection(appFAMEReplayer)

#configuration for remote controll connection(mumodo)

#Offset settings

#Frequency of synchronization in (ms)

Note: There is some sample data in mumodo package including all types of files you need (.eaf, .xio, .wav, .mp4) to see a little demonstration of what is possible. The playback scene for InstantReality visualization is located in the x3d package.



Our IPython based data analysis software.


Download mumodo package.


EnthoughtPythonDistribution (not really a dependency per se, but it provides all the packages that are needed (IPython, numpy, matplotlib) in an easy package)


  • install tgt (with python distribute: pip install tgt)

    • via github:git clone

    • cd TextGridTools

    • python install

  • (move the installed package to epd-x.x/lib/python2.7/site-packages)

for ELAN_mod remote control:

Analytical Geometry Library (Vectors, Planes, Quaternions etc.)

X3D files


We have collected some X3D files for you to see the possibilities of the InstantReality framework in combination with our software collection. See the comments in the X3D files for more information. Also visit InstantReality website.


Download X3D files and extract to a folder of your choice. Now you can open them with InstantReality or edit them with a text editor.

Third party software


The instantreality-framework is a high-performance Mixed-Reality (MR) system, which combines various components to provide a single and consistent interface for AR/VR developers built by Fraunhofer IGD.


IPython is an interactive shell for interactive computing in multiple programming languages, especially focused on the Python programming language, that offers enhanced introspection, rich media, additional shell syntax, tab completion, and rich history.


ELAN is a professional tool for the creation of complex annotations on video and audio resources.


This website reports on some results of the "Multimodal Interaction Lab", which is located at Bielefeld University and is led by David Schlangen. The information contained in this website is for general information purposes only; we make no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the website or the information, products, services, or related graphics contained on the website for any purpose. Any reliance you

... Continued

place on such information is therefore strictly at your own risk.

Through this website you are able to link to other websites which are not under the control of us or Bielefeld University. We have no control over the nature, content and availability of those sites. The inclusion of any links does not necessarily imply a recommendation or endorse the views expressed within them.