LEGO NXT

The use of LEGO NXT robots as the platform for ROILA automatically means that in an instant there are potentially thousands of robots that would attain the ability to recognize and talk in ROILA.

We have adopted the Lejos platform to program our NXT robot, an obvious choice since Sphinx-4 is also written in Java. You might want to know how to reset the NXT operating system to Lejos. This excellent step by step tutorial would help you a great deal. Alternatively you might also want to explore this tutorial provided by Lejos itself.

Please use the following classes.jar file for lejos, courtesy of the lejos team. Remember to replace your old version of the jar file in two places: lib folder in the projects folder and lib folder where you installed lejos. A simple replace worked for us and we did not have to rebuild any code.

Morevoer, in this section we will now discuss how we established the communication between the NXT and Sphinx-4 using Bluetooth. Once you have Sphinx running on your machine (laptop) and you are able to recognize ROILA sentences you might want to transmit the recognition results to your LEGO NXT robot. We have adapted the sample BTSend and BTReceive java programs provided by Lejos. It is perhaps a good idea to see if you can get bluetooth connected and working between your server (the laptop) and client (the NXT). You will need to add the ROILA Sphinx recognition code in your BTSend program. We have followed the approach of sending the recognized string as bytes over the bluetooth connection.

You will have to include the following bluetooth jar files on the program for the NXT only and not the server side (PC, laptop)

• bluecove.jar
• bluecove-gpl.jar

Add all the four jar files in the lib folder of your Lejos installation. Do this for both server and client programs. Remember to set classes.jar as the topmost jar file to be used by the system (can also be changed in the project properties->build properties, by using the navigation buttons such as top, down). Otherwise Eclipse will act strangely and will use the wrong API for various classes (such as using the NXJ versions of classes instead of the NXT).

It will depend on what you want your NXT to do once it receives the recognition results. In our scenario we simply have the robot navigate through the environment. Please find our two files BTsend and BTreceive

We have not implemented a dialog management system yet but that is our current work where we will have the NXT robots talk back in ROILA via text to speech synthesis. Therefore you might find the current implementation quite trivial as the interaction is just one way.

Remember to hard code information about your specific NXT in BTSend or you may also want to enable open discovery of NXT’s via bluetooth. Secondly, our motor actions might not work correctly for your particular NXT, especially the go left and go right commands. You may have to tinker with the parameters of the steer function.You might still run into some problems, if you do please contact us.

We have tested our code for both 4 and 2-wheeled Lego Mindstorms NXT robots. We hope to upload a video soon to see the robots in action.