Project Number: P13015
Multidisciplinary Senior Design Conference
Kate Gleason College of EngineeringRochester Institute of Technology
Rochester, New York 14623
Copyright © 2012 Rochester Institute of Technology
NAVIGATIONAL AID FOR THE BLIND AND DEAF
Michael BovaComputer Engineer
Kevin AlderferComputer Engineer
ABSTRACTIt can be extremely difficult for individuals who are blind (and deaf) to reach their destinations in a new
building or unfamiliar environment without assisted direction. This project attempts to ease that difficulty by creating a device for use by visually impaired individuals in such circumstances. The device consists of an elastic arm band with sewn in motors and a small mounted housing case that contains a microcontroller (MCU), a keypad, a radio-frequency transceiver, and a lithium battery. The user enters a room destination into the keypad, and the RF transceiver then communicates with other active RF transceivers placed at predefined locations around the building to determine the user’s location. Based on this, the MCU then calculates the shortest path to the destination. As the user traverses the path, tactile feedback is provided via vibration of the appropriate motors (i.e., motor on left of wrist vibrates to indicate a left turn should be made, etc). The user’s position is kept up to date via constant communication between the transceivers around the building and the transceiver in the arm case.
INTRODUCTIONNavigating a new building for a visually impaired individual can be an extremely difficult task. Building
maps usually are not in Braille or relief, and even though room numbers are often marked in Braille, less than 10% of legally blind individuals in the US can actually read Braille [1]. On top of being blind, the Rochester Institute of Technology has a very large deaf population. This leads to a population of blind and deaf individuals, and it is even estimated by the National Association of Regulatory Utility Commissions (NARUC) that there are approximately 70,000 to 100,000 people in the United States alone that are considered deaf-blind. Not many products are available for individuals who are both visually and hearing impaired, and the products that are available are expensive and only affordable for a small number of individuals [2]. The Navigation System for Visually Impaired and Deaf Individuals attempts to address this problem.
This project is the third iteration and attempt to create a working and viable electronic, software, and hardware system. The most recent iteration was comprised of two different teams, P12015 and P12016. This iteration’s final product comprised of an electronic system contained in a black plastic box with a number pad on the top attached to a sleeve that slides over the left arm. The physical system seems to work well, but electronics and software had some major issues. The old system was based on an MCU controlling an RFID reader that would read tags on the walls. This system was so power hungry it could only last about 10 minutes, the reader only occasionally read tags, and many other electronic and software problems were still present. The goal of this current project is to fix the electronic and software problems to create a completely functional prototype.
Figure 1: Assembled Physical Device Figure 2: Structural Block Diagram
NEEDS AND SPECIFICATIONSThe customer requires that the device be able to efficiently navigate an individual from any two locations on a
single floor of a building. Since the device will be used by a visually impaired and possibly deaf individual, it should not use visual or Braille feedback and should be easy to use. The device should be comfortable to wear, yet free both hands when wearing the device to allow use of a cane or a guide dog. The attachment and detachment should be done with one hand, and the process should be simple and take less than one minute. In addition, the device shouldn’t distract individuals near the user and shouldn’t draw too much attention to the user.
The device (not including attachment method) should be compact and weigh as little as possible. The battery should last for at least ten 20-minute navigation intervals and completely charge in less than eight hours. The budget for the project is $200, provided by the National Science Foundation. One of the customer’s main technology constraints is that the device should be built around the current system; hence, the physical assembly, MCU, magnetometer/accelerometer, and the no longer used RFID technology including the antenna, RFID reader, and RFID tags were all provided by the customer.
DESIGN PROCESS Previous designs of this navigation system for the visual and hearing impaired focused on using an RFID reader to read passive tags on the walls of the building being navigated. This system needed to use a large antenna connected to the RFID reader because it needed to actually provide the power to the tags in order to bounce their IDs back. Although this provides the advantage of not having to power the passive wall tags, the voltage required to power the reader and antenna is immense and drained the device’s battery very quickly. Due to this constraint, this iteration of the project decided to move away from this design and create a system using active tags. Using TI’s eZ430-RF2500 transceivers, the old passive tags were replaced with self powered end devices that will periodically look for their pairing: The access point device connected to the MCU. This redistribution of power allows for proper usage of low power settings, because the battery no longer needs to drive a massive antenna, nor does it need to power the initial and return signals from the end devices. These changes in the system design required a few electronic hardware changes and several software changes, which will all be described throughout the rest of this paper.
HARDWARE CONNECTIONS After choosing a new radiofrequency device, a method of interfacing it with the existing MCU (MSP430F5438A, henceforth called the main MCU) had to be decided upon. On the main MCU, pins 39 and 40 were chosen as UART data transmit and receive pins. These correspond to register USCI_A0. Furthermore, pins 87 and 88 were used as digital VCC and Ground to supply the ez430-RF2500. By default, these pins did not provide enough voltage for the device, and in the interest of time an external battery pack was used to power the Access Point on the main device. The transmit (Tx) and receive (Rx) pins were soldered to small solid wires that were then super glued to the PCB in order to secure them in place. These wires were then attached to the ez430-RF2500 in the appropriate ports in order to complete the physical interface between the main MCU and the transceiver device. Each end device is powered by two AAA batteries, and many needed to be fit to custom battery packs (as seen below). Battery packs with attached wires were acquired from Radioshack and used to power the end devices. A replacement magnetometer had to be ordered due to a hardware malfunction in the inherited device. The faulty magnetometer was taken off the PCB and replaced with the replacement device without any problems.
Proceedings of the Multi-Disciplinary Senior Design Conference Page 3
Copyright © 2008 Rochester Institute of Technology
Figure 3: End Devices (nodes)
Figure 4: PCB (bottom) with Attached Access Point Figure 5: PCB (top) with Attached Access Point
TRANSCEIVER
The implementation of the ez430-RF2500 has replaced the outdated Skye M9 RFID reader module. It is a radio frequency transmitter/receiver that transmits at 2.4GHz, instead of the old RFID reader’s 1GHz. This allows for a much greater physical range between devices when compared to the UHF (Ultra High Frequency) RFID reader and its passive tags. Each transceiver device is comprised of a CC2500 transceiver, an MSP430F2274 which is used to interface with other devices and the power source, and an on-board antenna. The transceiver devices each have their own power source in the form of 2 AAA batteries. This evenly distributes the power supply requirements across the entire network, instead of placing the burden on the main wrist device. It also adds another variable to the devices that can be used to calculate the user’s position: The signal strength from an access point (the user’s main device) to an end device (the tags on the wall) can be determined via a given set of instructions from TI. This can be used to determine how close a user is to a given node, and is used to update the path following algorithm accordingly. The ez430-RF2500 was also chosen due to the fact that it is manufactured and supported by Texas Instruments. One of the major concerns of the previous iteration was that the Skye-M9 had shoddy documentation that was incorrect in many cases. This led to large stalls in development time that could have been avoided if a proper support base and documentation had existed.
Figure 6: ez430-RF2500 Transceiver
MAGNETOMETER/ACCELEROMETERThe LSM303DLH is a chip containing both a magnetometer and an accelerometer. It uses an I2C interface to
connect to the MSP430, which can manipulate the chip by setting a number of registers. In order to obtain the heading of the user, the MCU will first poll the accelerometer and magnetometer to get their raw data values, which are each comprised of 3 16-bit values x, y, and z. These values are then normalized. Then the accelerometer values are manipulated with some geometric functions in order to determine the tilt and angle of the device. Next, these
values are used to apply tilt compensation to the magnetometer values. This methodology allows the system to determine the proper heading in relation to magnetic north no matter what the position of the user’s arm is.
For this iteration, the main design concerns were correctly implementing the accelerometer and tilt compensation. After much research and experimentation, it was determined that the magnetometer had been nearly correctly implemented by the previous group. The magnetometer slave address that was being used was not the correct value, and thus needed to be changed in order to get any meaningful data from the device.
Another mistake of the last iteration was that they assumed that values were read from the magnetometer and the accelerometer in the same manner. A read of the magnetometer output will automatically update the register address and therefore send over all 6 bytes that make up the x, y, and z values. This was not the case for the accelerometer. Accelerometer values needed to have each register read separately. The register counter was not incremented automatically, as was assumed. Once a new I2C read was added for each byte to be read separately, correct accelerometer values were acquired. Tilt compensation was not incorporated for the last iteration due to the fact that accelerometer values were not being correctly read and also due to a hardware malfunction in the device. This was determined after research into the initialization procedures of the accelerometer. A replacement device was then ordered and soldered onto the PCB, and the accelerometer then proceeded to work as expected.
In order to acquire correct tilt compensation values, some updates were made to the structure of the code. The normalization routine for magnetometer and accelerometer values was not producing correct results. This turned out to be due to a memory referencing issue on the MCU. The normalizing code was removed from its subroutine and placed in the main method of the program. This corrected the value and allowed for accurate tilt compensation to take place.
MICROCONTROLLER The microcontroller (MCU) is the central component of this whole system, and with every change to the system the controlling software in the MCU had to be changed as well. The first of these changes was removing the code used to control the RFID reader and still make the program compile and run. With the changes to the map design and representation, all of the data structures holding that information had to be rewritten. The work that was done to get the correct and complete tilt-compensated heading has been described above in the magnetometer/accelerometer section. Quite possibly the biggest challenge of all this rework was becoming acquainted with all the inherited code. The new interface that had to be written from scratch posed problems that were just as challenging though. The ez430 access point transmission string was created to transmit two bytes, the node ID followed by the signal strength. In order to receive these on the MCU side, the USCI_A0 on port 3 was set up in UART mode, an interrupt service routine (ISR) was created, and interrupts were enabled on the USCI_A0 receive buffer. The ISR saves off the data sent by the access point to an array that contains all the current signal strengths of the nodes. The ID that is sent first is used as an index into that array.
PROGRAMMING THE MAP The map structure is a proof of concept for now. There are 20 nodes that represent 37 rooms on the second
floor of the James Gleason Building on the Rochester Institute of Technology campus. The map is set up like a traditional graph, consisting of a number of nodes connected by edges to the closest neighbor on each side. This structure is augmented by associating IDs and destinations with each node. Furthermore, each node has a maximum signal strength associated with it in order to determine the user’s distance to the node. During the path following algorithm, this maximum signal strength will be used to create a ratio between the acquired signal strength to the maximum signal strength of that node. The ratios of all the “viewable” nodes will then be compared and the largest will be used to determine which node the user is closest to. Nodes, edges, tags, and destinations are stored in separate arrays in ROM and reference one another by index to reduce memory requirements.
Proceedings of the Multi-Disciplinary Senior Design Conference Page 5
Copyright © 2008 Rochester Institute of Technology
Figure 7: Node Locations on KGCOE- 2nd FloorPATH FINDING ALGORITHM
The path finding algorithm works by first accepting user input through the keypad to tell the device the desired room destination. It then determines which node the user is closest to by using the attached transceiver. After the start and end points have been obtained, the algorithm uses the map structures to obtain the required data to be used in Dijkstra's path finding algorithm. This algorithm looks at the shortest edge from the current node to its neighbors. It continues along this path until it is longer than a previously skipped edge and then goes that path instead. Eventually it will map the entire graph of nodes and find the shortest path from the current position to the desired destination. After the path is found, this algorithm traces the path backwards to create a list of all the nodes along the path which will be used during the path following algorithm
Figure 8: Path Finding Flow Chart
PATH FOLLOWING ALGORITHM
The path-following algorithm describes the process used to guide the user from their current location to their desired destination using the set of directions generated by the path-finding algorithm. The user’s position is first ascertained by polling the receiver (or access point) and checking the list of nodes received against the map’s node list. If the algorithm determines that the user is significantly off course, the path is recomputed and the path-following algorithm is restarted; otherwise, the user’s position is updated and the algorithm resumes. The user’s heading is also continuously determined using the tilt compensated magnetometer readings and it is compared against the heading value required for the user to advance to the next node, which is stored in the map structure. If these values don’t match, the path-following algorithm issues a command to the user to correct their course by turning either left or right. Once the user is facing the desired direction, the command is rescinded.
Figure 9: Path Following Flowchart
CONCLUSIONS AND RECOMMENDATIONS
CONCLUSIONS The ez430-RF2500 end devices were programmed to send both a node ID and signal strength to the access point, as well as to broadcast at a predetermined strength. Nodes are successfully detected by the access point, and data is then passed from the end device to the access point wirelessly utilizing Texas Instrument’s SimpliciTI network protocol. The LSM303DLH magnetometer/accelerometer was successfully implemented to find a tilt compensated heading of the user. The inherited chip was faulty and had to be replaced, and the inherited software was erroneous and needed correcting. Once these tasks were complete, tilt compensated headings were acquired. The map data structures were updated to incorporate the change in hardware (the exclusion of the Skye M9 RFID reader and the inclusion of the ez430_RF2500). This meant removing irrelevant data and incorporating signal strength and node IDs into the design. Furthermore, since only 19 nodes were used on the map, all of the old node values had to be removed and replaced with the new nodes representing new destinations. The new nodes chosen best represent the versatility and validity of the ez430 incorporated design, and were chosen for this proof-of-concept for that reason.
RECOMMENDATIONS FOR FUTURE DEVELOPMENT The main concern for the future development of this project is the rework of the hardware. With the change to the ez430-RF2500 instead of an RFID reader, much of the old PCB has become obsolete. There were also a few
Proceedings of the Multi-Disciplinary Senior Design Conference Page 7
Copyright © 2008 Rochester Institute of Technology
temporary hardware work arounds used in this iteration of the project to get it to a working proof-of-concept. These include things like the wires for the I2C connection to the magnetometer, the wires for the Spy-Bi-Wire interface to the ez430-RF2500, and the external batter pack used to power the access point. These are acceptable for a proof-of-concept implementation, but should be incorporated into the new PCB when designing a finished product. As mentioned in the hardware section above, the Access Point on the wrist device ended up being powered by a battery pack of its own, rather than being incorporated into the power supply used by the PCB. This was due to time constraints, but should be avoided in the future when redesigning the PCB. Adding another trace, like that which powers the MCU, should allow for easy implementation of this change. The temporary physical connection has already been soldered onto the MCU pins and glued down for future use if necessary. One possible issue encountered with the magnetometer/accelerometer is incorrect heading values due to a swinging arm. Since the device sits upon the user’s left arm, it is fair to assume it will swing with the user as they walk. Tilt-compensation works by measuring the overall force on the device, which when at a standstill is just due to the acceleration of gravity. Adding in the acceleration of a swinging arm will affect the overall force on the object, and likely cause incorrect heading readings. For this proof-of-concept, the user must keep their arm in one fixed position. This issue could be avoided by redesigning the physical device into something steadier, such as a belt or other attachment to the torso. Otherwise, different technologies should be researched that will take device orientation into account.
REFERENCES
[1] "Fewer Blind Americans Learning Braille - US News - Life - Msnbc.com." MSNBC. 26 Mar. 2009. Web. Nov.-Dec. 2011. <http://www.msnbc.msn.com/id/29882719/ns/us_news-life/t/fewer-blind-americans-learning-use-braille/>.
[2] "NARUC Passes Resolution Supporting Equal Access to Communications Technologies in the 21st Century." Coalition of Organizations for Accessible Technology. Committee on Consumer Affairs, 20 Feb. 2008. Web. 27 Apr. 2012. <http://www.naruc.org/Resolutions/People%20with%20Disabilities%20Resolution1.pdf>.https://edge.rit.edu/edge/P13015/public/Home (also teams P12016, P11016, P11017, and P12015)
ACKNOWLEDGEMENTSThis material is based upon work supported by the National Science Foundation under Award No. BES-0527358. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.
