1. PhysComp & ICM Finals: Assistive AR for the visually impaired.

    For our Physical Computing final, my project partner Sajan and I decided to work on an assistive project for the visually impaired. Although we had many ideas, we realised that we really had no idea what the particular challenges and needs were that a blind person faced on a daily basis. We then set up a meeting with Walei, a young man who had gone blind much later in life. Meeting Walei put paid to a lot of our preconceptions and ideas. When we asked him to talk to us about his day to day experiences and his opinions on what could be ameliorated, his answers gave us wonderful insight into the unique ways in which he compensated for his lack of sight, including learning his way around New york by memorising it.

    After talking to Walei and to Steven Landau, whose company creates aids for the blind, we decided to work on a device which would allow them to recognise the poeple in front of them or who are addressing them.

    Using AR markers to act as name tags, we thought of using mobile phone cameras that would recognise the AR tags on the person in front of them and give them information back about the person via audio (using the phone itself). The tag could contain information such as the person’s name, profession and any other info that could be useful in helping the visually impaired person to understand the person in frot of the better.

    My partner was also interested in working on a motorised Braille device, that would move to represent the person’s name in Braille as soon as the AR marker was read. He set out to work on this.

    Having never worked with Android before, I asked for help and used the resources on the internet to figure out a way to program the phone to playback audio files and also to do image tracking. In time, and with plenty of help from Craig, I was able to program the phone to play back audio files. We also found an openGL library for the android that allowed us to do marker tracking. The phone’s camera recognised the marker and a 3D rectangle popped up.

    From here, we thought it would be simple enough to change the code to make the audio files play instead of having the shape appear on recognition of the marker. However, it seems it is not quite that simple. While I can get the recorded MP3 files to upload and play separately on the phone and and also have the camera recognise the marker and give back an image, I ran into trouble with merging the two. In order to solve this I need to get deeper into the Android programming, which is where the problem lies. That is Step 2. Once that is done, technically at least, the phone will be able to read different AR markers and give audio feedback about the person wearing the specific AR tag.

    I am continuing to work on the Android to get it to the point where the phone can read the markers and playback the relevant MP3 files. The concept works via Processing, where the recognition of a markers plays back the audio but without mobility, this application would be futile, so the most important step is to get it to work on Android.

    This project has been the hardest and most frustrating one for both Sajan and I, but it’s also been the one we’ve gotten the most out of. Sajan struggled with the mechanics of making the motorised Braille work (more about this on his blog - http://itp.nyu.edu/~sr1971/myBlog/?cat=13). In the process, he learned a lot about different materials and their possibilities and limitations. I struggled with my lack of knowledge with the Android, but it also gave me the opportunity to start learning Android programming and get to know the most widely used mobile platform in the world. A lot of it seemed familiar because both Android and Processing have Java in common. However there are features peculiar to Android programming that I’ll be able to understand only by completely learning it. It was incredibly exciting though to be able to program something and upload it to the phone, only to see it as a feature on the phone! I can safely say I’m addicted :)

  2. ICM Assignment Week 4

    Apple’s Yuan love

    last week, I read how Apple had given into Chinese govt demands for the iPhone4 and installed PRC approved maps which included Arunachal Pradesh as part of China. I ended up venting through my Programming assignment. The symbol is China’s currency, the Renminbi


  3. ICM Lab 1: Ying & Yang, ellipse by ellipse

    No, the background is not intentional, nor is the yang supposed to be outside the boundary of the circle, but I suppose i could always say that it was intentional and that it represents traditional masculine incursion into and dominion over feminine being, but I’m really not that kind of feminist, so we’ll stick with this : Computer Crash; taking with it my homework and list of places that do the best filter coffee under 3 mins. I took this midway through, hence the lack of aesthetics.

    Onto the lab. I LOVED doing this.  It took eons and the amount of code could only be justified if the symbol talked back to you, but nevertheless, it was great practice. The first hurdle was getting the shapes to form by using curves. that didn’t work, so on the suggestion of a second year, Peter, I tried the Bezier curve, which was better. Next came the dilemma of introducing two different colours into overlapping spaces. The only way I could come up with on my limited understanding was to start with two black circles and then colour one of them white, ellipse by ellipse. It seemed like a waste of time and Ariel, the resident who was trying to help me find more sophisticated ways of achieving the same thing w/o complicating it for me thought I was nuts :) It was, but it turned out to be great practice in both, microfocus and to understand where the graph points were. The final product looks a lot better than the above. I had an issue with the colour being stuck on white at one point even though the most recent fill function specified black and Dan Shiffman helped change that by defining the ellipseMode as Centre rather than CORNERS which I’d been working with. All in all, a time consuming lab, but I never thought I’d enjoy it so much, i’d forget to eat!