InproTK has been around for several years and is becoming more widely used in dialogue processing research. It follows the Incremental Unit framework of incremental dialogue processing.

The toolkit has been written in Java primarily by Timo Baumann. In this intro, we will show you how to get InproTK up and running with speech recognition (Incremental Sphinx, part of InproTK), a dialogue manager (opendial, by Pierre Lison; note that opendial can also be used as a fully-functional end-to-end dialogue system, with recently added incremental capabilities, but we will use its dialogue management capabilities here), and a text-to-speech interface (incremental MaryTTS, also part of InproTK).

This intro has two parts. The first part is a rough guide on getting everything “installed” so you can see a working InproTK project called myds. The following video steps you through the myds project code and explains how it all works together. You do not need to install everything to watch the video, but it certainly helps.

Installing InproTK

InproTK is written in Java, so “installing” it means getting the jar (and necessary libraries), or getting the source code. It is advisable to get the source code.

You can the source code from bitbucket by running the following command (sign up for a free bitbucket account if you have not already done so):

• git clone https://[your username]@bitbucket.org/inpro/inprotk.git

The rest of this will explain how to use InproTK within eclipse using a sample eclipse project.

• Download eclipse, unzip, run (you may need to install a Java JDK, make sure it is above 1.5 but lower than 1.8), pick a workspace

• Import the InproTK project ….

  • in Eclipse, file -> import -> Existing Projects into Workspace -> Browse -> (find where you cloned the InproTK project)

  • You should see inprotk as a project in Eclipse. Make sure there are no errors (an error is denoted by red marks). You can click on the “Problems” tab to see where the problems are. Make sure all the libraries are included in the project. To do that, click on Project -> Properties -> Java Build Path -> Libraries -> (select jar files in the inprotk lib folder)

• Download, unzip, and copy the myds project, import in the same way as you did with inprotk

• right-click on myds, go to Properties -> Java Build Path -> Projects -> Add … -> (select the inprotk project)

You will also need the opendial project and its libraries. You can checkout the project from svn:

• svn checkout http://opendial.googlecode.com/svn/trunk/ opendial-read-only

  • either import it into eclipse, or just use the eclipse svn client and create a new project

• right-click on myds, go to Properties -> Java Build Path -> Projects -> Add … -> (select the opendial project)

• Check myds for errors.

Now, you need to get MaryTTS working in InproTK. It’s not required, but getting it working will allow you to use myds. Follow the instructions found at http://sourceforge.net/p/inprotk/wiki/Setup/

• be sure to use Mary 4 for the myds example

• you can also set the mary.base in the app.Main.run() in the myds project.

You should be able to open apps.Main in myds/src and run it by clicking Run -> Run as -> Java Application. It should run without throwing any errors. You will need to change some of the paths in Main.run()

Next, you are ready for the instruction video:

We will post more tutorials on InproTK and incremental dialogue processing in the future.

References

Baumann T, Schlangen D. The InproTK 2012 release. In: Proceedings of the NAACL-HLT Workshop on Future directions and needs in the Spoken Dialog Community: Tools and Data (SDCTD 2012). ACL; 2012: 29–32.

Kennington C, Kousidis S, Schlangen D. InproTKs: A Toolkit for Incremental Situated Processing. In: Proceedings of SIGdial 2014: Short Papers.; 2014: 84–88.

Schlangen, D., & Skantze, G. (2011). A General, Abstract Model of Incremental Dialogue Processing. Dialoge & Discourse, 2(1), 83–111.

Lison, P. (2014). Structured Probabilistic Modelling for Dialogue Management. University of Oslo.

Reference audio is a common method for synchronization of videos from different cameras, when more expensive equipment that does this automatically is not available. The most common use case is of course filming a scene from several different angles, but there are other setups that may require the same technique. Recently, I had to film several scenes in succession, over which the same audio was playing. I needed to extract a part from each scene, so that exactly the same audio would be playing over each scene part (they all have the same duration, of course). So far, nothing is new. However, what is surprising is how easy it is to do that with a handful of open source tools, as I will be showing here. In particular we will be using Praat, ffmpeg and Python.

The tutorial is available in PDF format:

video_syncing_fun