Mobile handheld devices have been around for a while and are continuously evolving in terms of both hardware and software capabilities, becoming more practical for a variety of uses. Gone are the days when such handheld devices were used for voice communication or some simple internet browsing and organizing tasks, as miniaturization became more and more significant these devices became packed with communication interfaces, sensors and other peripherals, with powerful processing units to support the added workload and operating systems to coordinate all of it.
In fact nowadays, we can find these smart devices employed in ways not imaginable no more than a decade ago, not without a standard computer at least. As the gap between the capabilities of a standard computer and a smartphone or a tablet diminishes, it is not unusual to see a smartphone or a tablet PC used as a gaming console, as a home media server or as the main method for voice, e-mail or other types of communication. These devices have become very popular when used as full featured remote controls in building automation systems, sensing units or even for development up to a certain level. The list of applications is practically endless so it came only natural to take use of features offered by such a capable device in the field of robotics. Today we will talk about apps created for the two main mobile operating systems, Android and iOS, designed for enabling your smartphone or tablet to interact with the Mindstorms NXT robotic platform. These apps can either turn your handheld device into a remote control for your NXT robot, a data logger or a tool for on the go programming.
Developing software for the Android platform is much more handier thanks to the fact that it is an open-source platform, and an extremely large knowledge base, supported by many communities, is available. It is not a device-specific platform so, at least in theory, anyone can design a device running Android or create a software application – an app – that accesses various resources of the operating system without limitations and thus enabling the developer to add functionality or alter existing features according to purpose, and this determines abundance of Android-based NXT apps that are available. There is a whole different story in the case of the iOS mobile operating system, which is bound to Apple devices and features strict development rules and limited SDKs. This is not necessarily a bad thing, as generally these restrictions are meant to ensure optimum yet somehow limited functionality in most conditions. As we will see later in the article, there are solutions for effectively implementing desired functionality for Apple devices under iOS, while retaining their acknowledged stability and smoothness in operation.
These being said, we will start by reviewing some of the most popular top rated Mindstorms NXT apps for Android, apps that of course work, interspersed with practical configuration and running examples, and later in the article we will talk about NXT apps employing standard iOS Bluetooth functionality and we will also talk about the open-source BTstack software, designed to enhance Bluetooth functionality under iOS, as well as apps that take advantage of this.
Apps for Android devices
NXT Remotes by Ferdinand Stueckler
There are 5 different NXT remote control apps available on the Play Store as free downloads, created by the Austrian professor Ferdinand Stueckler. They all serve the same purpose, to remotely control the NXT robot over a Bluetooth connection, but in different ways.
The NXT Simple Remote enables you to control the NXT by pressing arrows on the screen of your Android device, with speed preset from the slider on bottom. The Sense toggle enables or disables holding a command without keeping the finger on the respective arrow.
The NXT GSensor Remote allows you to do the same thing as above but this time by using the build-in accelerometer sensor of your Android device. The two horizontal bars indicate the level of tilt on X and Y axes, corresponding to the speed of the two motors used for driving. Neither this app or the simple remote app mentioned above do not require initial programming of the NXT brick.
The NXT Numeric Remote app allows you to control the NXT robot by sending numeric or text commands embedded in messages over Bluetooth. This app requires a program that implements mailboxes to be loaded into the NXT’s memory. The advantage of this app is that you can create a wide range of custom commands with actions to be executed by the NXT robot. The Lego NXT-G Bluetooth program block supports only string data. For full support of received data types, NXC programming is recommended.
The NXT Mailbox remote app goes one step further by implementing commands based on logic data that is sent to the NXT robot. Just as for the app above, only string values are supported through NXT-G programming, for full data type support NXC programming of the brick is recommended.
The NXT GMailbox Remote is designed to send accelerometer sensor data from your Android device to three distinct mailboxes, one for each axis, implemented into the NXT brick program. Data can be sent either in string or numerical form and same issues regarding NXT-G programming as above apply.
Below you can see screenshots and operating examples with these apps. The NXT and the Android device need to be paired prior to connecting to the NXT brick from within the apps.
NXT Pad
The NXT Pad app can control the NXT robot over Bluetooth either directly, without preloading any program into the NXT’s memory, or through messages by sending single digit numbers to the NXT brick running a program that implements mailboxes. An NXT-G programming example for implementing mailboxes as well as more information can be found on the app’s website. Speed is varied with the P+ and P- buttons on the direct control pad and you can also customize the on-screen control keypad with messages to be sent to the NXT.
NXT Messenger
The NXT Messenger is a fairly complete app that allows you to interact in several ways with the NXT brick. Remote control through messages sent to mailboxes implemented in the NXT program is possible as well as direct remote control of the motors on the NXT. The app also allows you to start or stop execution of programs loaded into the NXT, play sounds and also provides feedback about the current battery voltage of the NXT robot and the running program. The app is available in two flavors, the free ad-supported version and the paid version which can be purchased for approximately 1.5 Euro.
NXT Speech Control
This speech control app for NXT enables you to control your robot through voice commands. A voice search module for your preferred language needs to be present on the Android device for this app to function. The principle is simple, for each of the four commands listed on the app screen, with adjustable speed and distance parameters, a voice cue can be defined. Then when the user speaks that command it will be executed by the robot. The app is developed by Sebastian Trella and Thorsten Leimbach as part of the Roberta – Learning with robots initiative.
Stream-O-Bot
The Stream-O-Bot app enables you to turn your NXT into a surveillance or spy robot, by using a secondary Android device mounted on it for audio and video streaming purposes. The same app needs to be installed on both Android devices, the remote controller and the one onboard the robot. AV data is streamed over an existing WiFi connection while robot control takes place via Bluetooth. The app can also be utilized as a simple remote control, with on-screen arrow keys or accelerometer control, if you do not have two Android devices available.
NXT simple remote controls
The NXT Remote Control is a simple app for the NXT, developed by Jacek Fedorynski. The app features several interfaces and you can interact with the robot by touch arrows or various areas on the screen. Drive motors can also be synchronized for accurate driving trajectories. The source code for the app can be found on Google Code.
Another simple app is the NXT Bluetooth Remote Control developed by Raphael Kunz and allows you to control all three motors of the NXT by touching arrows on the screen of your Android device.
The NXT Remote by iCount enables you to control the NXT robot either by pressing arrows on the screen, sliding your finger across a touchpad area or by using the buit-in accelerometer of your Android device. The interface is in Polish.
HiTechnic Segway Remote
The HiTechnic Segway Remote app is designed for remote controlling the HTWay NXT robot created by HiTechnic, using the built-in accelerometer of the Android device. The robot employs a gyro sensor, which can be implemented in NXT-G using the according custom block, the text-to-number block in the zip file and the HTWayAR available here.
MINDdroid
This is the official NXT app for Android from LEGO. The app is open-source and code is available on GitHub. Unfortunately the app is no longer available on the Google Play store, this entry is for reference purposes.
The app enables you to remote control your NXT robot over Bluetooth by using the build-in accelerometer of your Android device. Moreover rcd, rso, ric and rxe program files can be uploaded to the NXT through this application. The app features a cute and intuitive interface for easy interaction. Below you can find some screenshots and images with the app in action. The orange square gets bigger as tilt is increased in a certain direction, coresponding to the increase in the speed of the motor. The Action button controls the third motor which can be employed for various actions.
Paid NXT Apps for Android
A few paid Android apps for controlling your NXT robot are also available on the Google Play store. They have not been tested, we are just mentioning them for you to know.
The RobotiX Mindstorms NXT Remote from JGM Software enables you to remote control the NXT robot in two ways, either by touching buttons on the screen or by means of the accelerometer employed by the Android device. It can be purchased from the Play store for about 1.5 Euro.
The NXTRemote app from SmarthoneRemote enables you to control the NXT robot. Apart from movement control through screen buttons or accelerometer you can also control the lamp of the light sensor or the sound speaker as well as get feed back from all the sensors attached to the NXT. Programs and sounds can be started and multi-touch control is also supported. Custom interface layouts can be created through JavaScript. The app can be purchased for approximately 3 Euro from the Play store.
Another app from Smartphone remote is the NxtCamera which allows viewing on your PC streamed video data from an Android device onboard the NXT robot. You can also control the robot from your computer with the aid of the free PC app you can get from the developer website, however the app that is installed on the Android device is available on the Play store for about 3 euro.
Apps for iOS devices
The iOS allows a Bluetooth connection only with a few select devices and headsets with the Mindstorms NXT not being among them. This is why no direct Bluetooth connection can be established between an iOS device with standard functionality and the NXT. For this purpose generally a computer that acts as a Bluetooth server is employed, which receives information from the iOS device via a WiFi network connection and relays it via Bluetooth to the NXT brick. Bluetooth server applications are generally available for Windows and Mac.
iNXT Remote
The iNXT Remote is a complete app that enables you to remote control the NXT robot either by touch or by using the accelerometer on your device. You can also poll and log values from sensors connected to the NXT brick, control the operating mode of the sensors, upload, download or delete files from NXT’s memory, run programs and many more. Of course the app relays information to a Bluetooth server, for Windows and for Mac OS X, which in turn relays information to the robot. The app can be purchased from the iTunes store for US$ 4.99.
nxtBlock
This is a programming app for your NXT robot. The program is developed on your iOS device and then sent to the computer via WiFi which uploads it to the NXT via Bluetooth. You can also add custom blocks created in NXC via the MyCode block. Servers for Windows and Mac OS X are available here. The app is available on the iTunes store for 3 US Dollars.
RoboRemote
The RoboRemote is an app created by Romanian developer Mihai Salavastru and enables remote control of an NXT robot via your iDevice. There are three driving mode, the Puzzle Drive from which you create a track which the robot will follow, and available via in-app purchases are the Pad Drive and Air Drive modes through which the robot can be controlled either by touch or by making use of the device accelerometer. Computer Bluetooth server software, as well as detailed information on how to set up the system is available on the developer’s website.
Workaround for devices running iOS
Yet there is a way to alter standard iOS functionality by jailbreaking, i.e. running various tools that disable limitations in the software. For more information about this procedure you can visit the iPhone Developers community website.
The BTstack library for iOS addresses the limitations in the standard iOS SDK of the Bluetooth interface, as the hardware itself is not limited. For more information about the implementation of Bluetooth in iDevices you can visit the BTstack wiki.
LegoDrive
The LegoDrive v2 app can be found in the BigBoss repository on the Cydia store, which is automatically installed after jailbreaking your phone. The BTstack library is automatically installed, enabling your iPhone or iPod Touch to directly control a NXT robot over a Bluetooth connection. The interface itself is simple and straight-forward, to connect you need to enter the NXT ID, found under Settings/NXT Version on your NXT brick, define left and right motors, sensor connections and that is it. You can drive the robot by using the device accelerometer. The phone or device must be in a vertical position for motor commands to be null, after which the device needs to be shaken in whichever direction the robot is desired to move. A simple sensor readout is also displayed.
Note: We have not tested this app with devices running iOS 7 and above.
hi dan
i’m in my final year in engineering, my task is aout a bluetooth (hc-05) remote controlled car with at89c2051. what i really need is to get java code to work the stepper motor.thanks
i want to send message from NXT to Android device, which program i can use?
@morsi2026 the Android apps listed above are generally based on the same APIs which communicate bidirectionally with the NXT, just look for an app which displays these messages.