COVID19

The first time I heard of the coronavirus was on Jan 23 2020. A Chinese postdoc at a UPenn robotics lab I was planning to join was telling me about how a mysterious virus had been rapidly spreading through her home country. At the time, I vaguely remembered reading about a new virus originating from Wuhan, China a couple of weeks back, so I made a mental note to check if the two viruses were related after the meeting. I later learned that the coronavirus in question had originated from Wuhan. At the time, it seemed like a disease endemic to China and I put it out of mind till the end of February, when things started to get slightly more serious. Though the US only had a few cases at the time, it was quickly starting to become clear that the disease was a) no longer endemic to China and b) very contagious. I started obsessively checking coronavirus tracking web dashboards, much like how I used to check live cricket scores during the 2019 Cricket World Cup.

Two weeks later, the situation started taking a turn for the worse. We got an email from UPenn on March 12, stating that the campus would be closed for the remainder of the semester. In hindsight, this was a prescient move, though at the time it felt like an overreaction to a disease which wasn't fully understood.

Things rapidly started spiralling out of control in front of my eyes. Day by day, the confirmed cases in the US kept mounting till it had overtaken Italy as the most infected country in less than ten days since I had received that fateful email from Penn. This was no longer something I could casually regard like cricket match scores or the weather. It had become something very real.

The last six weeks have been the hardest I have experienced in a long time. Graduate courses continued in flow, and although there were some concessions in the workload, the work that was there took much longer to finish. Motivation and productivity plummeted. It was amazing anything got done at all when you factor in the hours spent everyday checking for live updates on the virus. Cutting out my daily half-an-hour walk to campus and back was making me go stir-crazy. That said, I was seldom bored, for it's hard to be bored when you're stressed.

On some level, it's hard to believe that the world is in disarray. With millions of people under lockdown, things are unusually peaceful outside. It feels like the onslaught of data about new infections has come to define the pandemic and how to react to it.

I am glad that I stumbled upon a Harvard Business Review article, That Discomfort You’re Feeling Is Grief as it helped me contextualize feeling something which I never had before. At the moment, I would say that I have reached a kind of wary acceptance with the current situation. However, I am frustrated that several opportunities (including an internship!) and plans are now shuttered because of the virus. Many decisions I made during the start of the semester were not made considering a global pandemic on the horizon. Suffice to say, most of these decisions turned out to be the wrong ones.

I have also been thinking about this virus on a more meta level. In no particular order, some of the things I have been thinking about are:

• In the last few years, there has been an increase in awareness about the deleterious effects of social media. The post-mortem of the 2016 US presidential election and Brexit highlighted how the terabytes of personal data mined by Facebook and its cronies could be used to manipulate people. Yet, now with life virtually online all the time, we are far more dependent on (and thereby influenced by) social media than ever before. With a particularly crucial US election this year, it is troubling that we could see a repeat of what happened in 2016.
• The American life could not be worse set up for dealing with a pandemic. In the US, excessive consumption is a way of life and freedom is an uncompromisable right. The pandemic, on the other hand, necessitates frugality and strict impositions on personal movement. It is telling how much at odds this is from typical life in the US from how confused the government's initial response to the disease was in the beginning.
• The speed at which awareness about COVID19 has spread is impressive. Yes, the media cherry picks reporting cases of people blatantly defying stay at home orders, but the general compliance of mask wearing and staying at home has been good. Anecdotally, I've noticed that mask wearing still hasn't caught on in Philadelphia even though there are many more people outside these days.
• The pandemic has resulted in a large drop in vehicular emissions. The air quality in Indian metropolitan areas such as Bangalore and Delhi has never been better. Will this inspire environmentally friendly policies and increased usage of public transport in the long term?
• "Somebody is to Blame for This". It is much more satisfying to think that this is the product of some organizational failure than just a freak accident of a nature. Who can one blame? The CCP for suppressing news about the outbreak? The US government for botching the coronavirus response in the crucial first weeks of the pandemic? The WHO for its "noble lie" of suggesting that masks were ineffective in containing transmission? It is exasperating how the issue has been fucking politicized. These failures will be case studies for future generations of what not to do during a health crisis.
• (More optimistically) As a budding roboticist, will the pandemic kickstart innovation in automation? Could we finally be seeing robots integrated in our daily lives in the next few years?

The only thing worse than the pandemic itself would be if the world goes back to the status quo after it's over. This is perhaps the only opportunity we will get to reflect on what really matters and how that should define policy-making decisions over the next decade.

Despite how grim things are at the moment, I remain optimistic for the next year. At least in the US, medical facilities are no longer overcrowded with patients. There are thousands of smart people researching possible treatments. Some vaccines have shown promising results in animals and some, such as the Oxford/AstraZeneca's ChAdOx1 have even started Phase 2 trials in humans. It is expected that we will learn more about the response of the vaccine in the next few months. Until then, we will have to do something we all have become intimately familiar with doing recently, wait.

Masters vs Undergrad

Before starting my master's degree in robotics at UPenn, I had many unanswered questions about what a master's degree would be like. I was certain that it would be considerably more difficult than undergrad but inspite of asking many current masters' students, I didn't have an idea of what to expect. As with many considering a master's degree abroad, my main concerns were not knowing how hard the coursework was going to be and adapting to the cultural differences in the US. I was also anxious about not having an exit plan figured out before starting my degree. Fortunately, each of my concerns were unfounded throughout the course of my first semester, and with this blog post, I want to share the key differences I experienced between undergraduate education in India with my first semester at UPenn.

Difficulty

For my first semester, I registered for the three courses - Introduction to Robotics, Computer Vision and Computational Photography, and Machine Learning. Though these courses are considered to be relatively heavy (even by UPenn's standards), I found it hard to believe that three courses with ten hours of classes a week could be stressful, especially when we were required to handle more than double the number of courses in undergrad.

It didn't take long to realise why though. Compared to undergrad in India, master's coursework in the US places a heavy emphasis on homework and assignments. The weekly individual assignments for each course I had in my first semester at UPenn are comparable to the monthly "group" programming assignments we used to have in undergrad. Evaluation policies for assignments are strict and late submissions for two of my courses were penalized at a 25% deduction in the maximum score per late day.

A master's degree will test the limits of your throughput and the rapid succession of assignment deadlines can easily lead to burnout. With the end of semester final project submissions, the last four weeks of the semester are particularly brutal for everyone. I don't think I would've made it through without compromising on sleep and inadequately compensating for it with the free coffee available at the Grad Students' Center. That said, the struggle wasn't for nothing. I feel that the knowledge I've gained through this process is much more durable than the kind gained for getting through a couple of semester exams. As an aside, the three courses I took this semester were not math heavy beyond needing to know some linear alegbra properties and solving the occasional differential equation.

The grading itself is not difficult - in most courses, scoring above the median will land you an A and scoring below will give you a B. C's and D's are only given if you neglect to turn in multiple assignments. Since the level of competition is high, even crossing the median score can be a challenge!

Atmosphere

The atmosphere at UPenn is more collaborative than what I had experienced before. You will frequently need to approach professors, TA's, and other students for help. Collaboration in projects here goes beyond the division of labour of work as it even involves finding solutions to problems for which there aren't any right answers. I also found other students to be more competitive and motivated. It's the norm to spend several hours a day banging one's head against figurative walls. A master's degree is also a significant financial commitment and it also comes with the opportunity cost of working for two years. This tradeoff is a good reason to be diligent about the coursework.

The pace of instruction is faster as well and it requires much more attentiveness in class. In the first four weeks of the Machine Learning course at UPenn, I learned more than I had in the entire semester of the Machine Learning course in undergrad.

Free Time

Due to the heavy coursework, household chores (especially cooking!), and part-time TA jobs that most students end up doing, free time ends up becoming quite limited. One of my gripes so far is that I haven't been able to explore research outside of my coursework despite the ample opportunities for the taking at UPenn's numerous robotics labs. It is possible to manage three courses with research, but I feel taking an independent study or equivalent course is more sustainable.

Preparation Required

I think it's a good idea to brush up on the tools you will need before starting a master's degree in engineering. First year fundamental math courses are important. Gilbert Strang's textbook and OCW series are good resources for linear algebra. There is a good chance that you will need to use LaTeX for homework and scientific software such as MATLAB and Python with scipy, numpy, sklearn, and pytorch among many others. Learning to use these tools effectively can save you a lot of time later. Though I feel it isn't required to prepare beforehand, you can find most of the course material of graduate courses online if you wish to preview the course content. However, the best way to know what you're getting into is by asking a helpful senior at the college you're planning to attend :-)

T+4 Years And Counting: Turning over a new leaf

On the eve of the day I start my masters degree in Robotics, I am definitely in a better state of mind than I was when I started undergrad.

I have to start by saying that the last year was blissful. My last semester on campus had very little in the way of any academic pressure and my thesis in Singapore at the ARL was everything I wanted it to be and much, much more. It has given me cold feet about whether the sharp dip in academic workload from my third to fourth year will become a detriment to how I will be able to handle the high workload of a master's degree. On the other hand, the break from frantically jumping from project to another has given me ample time to relax and evaluate what I want from this degree. Despite speaking with many people who are treading the same path, my goal of what a career in robotics will look like isn't entirely clear to me. Yet, I have high hopes that this industry will be entering a golden era not long after I graduate.

I expect a lot of changes in doing a program which demands exacting knowledge of three major fields of engineering, but there is plenty in the comfort zone of computer science that I can sink my teeth into. I am excited to see what the University of Pennsylvania and its Robotics program has to offer.

One of the few pictures I have of Penn's lovely campus

It's quite different from how I felt when starting undergrad in 2015, when there was some negativity in the air, a sentiment which was often shared with several batchmates who felt they could've gone to other colleges or taken other branches had things gone their way. That said, BITS Pilani's vision and policies awed many an impressionable fresher, me being one of them. Unfortunately, it can't be said that anyone graduating now would look back as fondly on their alma mater now as they did back then. It's clear that the academic utopia presented to us during the orientation was indeed too good to be true. It's not for lack of trying on the part of faculty or administration however, but instead of a combination of pedagogy not being able to keep up with the times and the difficulty in filling academic positions with highly qualified professors in India. For whatever reason, the students and the faculty were always at odds with one another and that has been detrimental in developing a healthy campus culture. Giving credit where it's due, college gave me the freedom to try out, succeed and fail at a number of pursuits with very little impact to future prospects. This is not a luxury I can take for granted anymore, for now the costs are far greater.

My priorities changed the most in my first year of college and I'm glad that I didn't fixate on any goal till I had a visceral feeling that a good idea (read GSoC) could work out. I had initially fancied myself graduating with a 9+ CGPA at the end of college, but my first semester's result made that vision a little too ambitious. Winning (yes, winning) the ACM-ICPC was another fascination I had during my first semester, but after a few months of grinding through various online archives, I realised that the pressure of competitive coding wasn't for me. There was still magic in programming for me and I had an idea during the previous summer vacation that I felt could realistically turn into a GSoC project. But that kernel of an idea only flourished when I pulled an all nighter coding a prototype of my idea during my first Diwali vacation on campus. From there on, I wasn't chasing an abstract idea, I was doing something far more tangible and that gave me the thrust I needed to get in touch with numerous KDE developers to put in a solid proposal and formally become a GSoC student that very year.

However, that led to hubris in my second year where I felt that any project could be conquered. I then later learning the hard way that perhaps I need to be a bit more discerning about my own capabilities after having wasted a lot of time working on projects that had little to show in the end. I think that in the most general sense, college is a good time to figure out your personality traits and to see what works and what doesn't. Time is usually abundant and the repercussions of bad decisions are temporary. It's hard to imagine any other time one would get a chance to explore like this. Looking back, there is plenty I will cringe about the 2016 version, or heck, even the 2018 version of me. But isn't that what personality development is all about? I would say that cringing on a past version of yourself (while uncomfortable), is a good thing because it shows that you are able to identify undesirable traits about yourself that you couldn't previously.

I would be remiss not to mention all the awesome people I've met along the way. Much of what I've been able to achieve in college and what has been crucial in bringing me to UPenn is because of these experiences of building strong connections with people who have gone out of their way to help me. I'm also eternally grateful to those people who told me when I was out of line or when I wasn't thinking rationally. It's also been great to have the privilege to travel the world during undergrad thanks to a couple of KDE conferences. My time with the ARL in Singapore was a distillation of everything I could have wanted from an internship and I feel much more prepared for master's after my time there.

In conclusion, I am quite looking forward to my time as a master's student and getting back on the treadmill of academic productivity. Graduate school will be another major reset of sorts, of the same kind as which happened when switching from school to undergrad. I hope that it won't be long before I develop some perspective on what life as a graduate student is like so that I can share it here. I also look forward to getting reacquainted to life in the US and seeing all that this country has to offer. Above all, I'm looking forward to seeing what education is like when done at its highest level.

The Summer of Pokemon

This has been the first summer vacation in a while in which I've had absolutely nothing to do. It's a far cry from the last three summers which had been filled with (at least in 2016 and 2018) interesting projects to do.

I have a tremendous opportunity waiting for me on the other side of this vacation – pursuing my Masters in Robotics degree at UPenn. It is remarkable that with hardly a month to go to MS, the specifics of life after graduation are still hazy. That said, if undergrad has taught me anything, perhaps things are best left unplanned. It's not to say that I haven't had my concerns about MS, but my initial trepidation is gradually turning into excitement.

One of the main reasons which made shift from CS to Robotics was that the greater interdisciplinary focus in Robotics would give me more options to figure out where I would really want to do. In some sense, it is giving me a second chance at postponing an important decision of which path of engineering to go down.

It's nice to be back at home after the internship in Singapore and it's been great to catch up with people from school and college. If anything though, this summer was the Summer of Pokemon. I was lucky to snag a used 3DS XL in near mint condition for S$45 on my last day in Singapore and it's made playing my library of 3DS games a joy.

For the first time ever with my Pokemon game collection, I deleted my old save and replayed Pokemon Moon from the beginning till the end of the postgame. Shortly after that, I borrowed a friend's cartridges to finish off Pokemon X and Pokemon White 2, to bring my count of completed Pokemon games to nine. Or rather 8.5 because I only vicariously played through about half of the Kanto part of SoulSilver. With this experience, I think it can be said that Pokemon is not a series to be binged. It is to be enjoyed slowly, one bit at a time. The three games which I played over the summer felt very similar and by the end, I blitzed through the last third of White 2 in a five hour marathon.

My second playthrough of Moon only supported the opinion I had after finishing my first playthrough - that the series was heading down a troubling path. Even after trying to make the game harder for myself through a Nuzlocke, it was hardly a challenge because of the change in the X-item mechanics. The story wasn't bad, but it felt very hamfisted and I dreaded the repetitive cutscene interruptions on going to each flag on the map. Speaking of the map, Alola looks a lot larger than it really is because the beautiful but short routes do not lend themselves to the same half an hour exploration fests that Platnium used to have. Also, the less said about the performance of this game, the better. After a hundred battles the delay between turns becomes less apparent but it's impossible to go back to Moon after experiencing how fluid Gen V was on the DS. Double battles are so poorly optimised that I couldn't bring myself to do more than ten battles of the Battle Tree before just switching to another game.

First impressions of Pokemon X were more positive. The game's performance was markedly better than Moon. X was also one of the more traditional Pokemon games with plot and objectives, only having Mega Evolutions thrown into the mix. Kalos reminded me a lot about Unova. X also tried to make better use of the 3DS's superior 3D capabilities over the DS, but the implementation left much to be desired. The camera angles were always off in Lumiose city and only certain areas of the map supported the 3DS's 3D mode which was very confusing. Unlike the O3DS, the 3DS XL doesn't have a 3D mode indicator LED so it requires pure guesswork to figure out which parts supported the 3D effect. Meaningful NPC dialogue in this game is hilariously lacking and it's left me wondering if the translation effort to make the worldwide release was using a briefly edited version of Google Translate. Random trainer battles felt obstructive and barring a few well produced cutscenes, netither the story nor the rivals are nothing to write home about. Another pain point were all the Battle Frontier references in X, which as we now know, never materialised and doesn't look like it will in the near future either. My personal theory is that the Battle Frontier was intended in a third version of XY, but the game was cancelled to make way for Gen VII.

After finishing X, I had high hopes for White 2. Pokemon Black has always been my favourite Pokemon generation, although some part of that can be attributed to the fact that 14-year old me could finally relate to a Pokemon game which didn't treat its players like small kids. It took me some time to get used to the DS era graphics, though this game made the best use of the console it was released on among all Pokemon games. It was so nice to see a new Unova after a two year absence (well, eight years for me anyway). I have to say, I did really like White 2. The handholding and cutscene spam which plagued Gen VII is much reduced and the difficulty of the game (even on Normal mode) is quite enjoyable. I suspended my Nuzlocke for a normal play after getting wiped out by Burgh's Leavanny. It was the first time I had to actually strategise and make a suitable team for a gym leader ever since FireRed which I played back in 2005. I found a Darumaka outside Castelia who fit the bill and swept Burgh in no time. I didn't find an EXP Share for some reason and the game was all the better for it. The regional Pokemon selection is excellent and there are plenty of great Gen V Pokemon to use. The story was not quite on the same level of the revelation the first Gen V games had, but after playing through X and Moon, this was a forgivable shortcoming for the better gameplay it had. There are a lot of subtle nods to the previous Gen V games through NPC dialogue which is miles ahead of what X has.

This little discourse on Pokemon games is years out of date, though it has been pretty interesting to play through half the series in reverse order to figure out where GameFreak started regressing on its best features to give us what looks like a rather uninspired effort for Sword & Shield. I do hope for the best, but I am not holding my breath for these games and I have no plans to invest in a Switch based on what I've seen so far.

Using the Logitech F310 gamepad with the DJI Tello

I really like the DJI/Ryze Tello.

It's a great nano drone and its feature list is hard to match at its price point. The camera/video quality is very good and for a drone which solely relies on its WiFi hotspot for control and video streaming to a smartphone, the range is pretty decent as well. It's impressively stable outdoors and the 10-15 minute battery life is adequate. As a trainer drone for new pilots, there's very little to complain about.

However, the main thing which drew to me to buy the Tello on impulse was its marketing with an emphasis on being "programmable". Unfortunately, I bought it without really looking into what this meant. I blindly assumed the Tello was hackable, but it turns out that the "programmable" features of the drone are limited to a few Scratch commands for writing basic scripts for movement. I wanted more control over the drone and initial impressions suggest that it has the technology to do a lot more than what the Scratch API exposes. The tiny drone has an infrared sensor for autonomous landing and uses optical flow for maintaining its position, which makes its sensor array unmatched for its price point.

The onboard Intel Myriad 2 SoC is advertised to have "14 Programmable SHAVE cores" but details on how to develop for this SoC are scarce. It's a shame, because I would've really loved to see the autonomous capabilities of drone by implementing my own algorithms for it. The Tello uses this SoC for its Visual Processing Unit to keep it stable in flight, so I think it's reasonable to speculate that this chip could be used for a larger variety of image processing applications. Moreover, the drone is clearly tuned for stability and this makes it feel sluggish despite the drone having the hardware to be much more nimble.

But I digress. I spent some time playing with different approaches to reverse engineer the Tello, and the main Python projects I found were:

Tello-Python: DJI's official Python API for interacting with the Tello. Promising, but I found their sample apps glitchy.

TelloPy: This library is amazing. It appears to be a port of some of the reverse engineering work on the Tello conducted by the Gobot folks. The API supports everything offered in Tello-Python and much more. To top it off, I found the sample TelloPy apps far more reliable than their Tello-Python counterparts

I also suggest checking tellopilots.com for more mods and discussion about hacking the Tello.

The nice thing about TelloPy is its controller support using pygame. However, this is limited to the PS4, PS3, XBOne, and Taranis controllers in gamepad mode. My Logitech F310 wired controller wasn't supported, so I figured out the button mappings in pygame and added it to the source of the joystick_and_video.py sample app. I've created a pull request for the same too. To download my fork of TelloPy, clone it from GitHub here:

git clone https://github.com/shortstheory/TelloPy/tree/F310 

After this, build and install tellopy. The one currently available on pip seems to be outdated .

cd TelloPy 

python setup.py bdist_wheel 

pip install dist/tellopy-*.dev*.whl --upgrade 

Now switch on the Tello and connect your computer to its WiFi hotspot. The Tello can be controlled using the F310 joystick using:

python tellopy.examples.joystick_and_video.py 

Press RB to takeoff and LB to land! The left stick is used for yaw and altitude control and the right stick handles roll and pitch.

Raspberry Pi Controller

It would be nice if this could work without the need of a laptop. I tested this by following the same installation steps on a Pi Zero W running Raspbian Lite and the setup works well. I added the script to the Pi's crontab so it would start on every boot of the Pi:

sudo crontab -e 

Followed by adding:

@reboot python /home/pi/TelloPy/tellopy/examples/joystick_and_video.py 

To make the Pi Zero W automatically connect to the Tello when switched on, add the following lines to your /etc/wpa_supplicant.conf on the Pi:

network={
   ssid="TELLO-XXXXXX"
   key_mgmt=NONE
}

The Pi Zero is small enough to be tucked in behind the controller, even with a small Li-ion battery for power. However, I imagine the range of the Pi Zero W's Wi-Fi connection would only make it suitable for indoor flight. Fortunately, this can always be fixed by using a USB Wi-Fi dongle.

Things to be done

While the TelloPy API supports the directional flip capability of the Tello, it doesn't map these controls to the joystick. I plan on dedicating the D-Pad for directional flips. I would also want the video streamed to the Tello to be saved directly to the Pi's internal storage. It would also be interesting to see if I could get the Pi to handle the image processing tasks and control the Tello remotely.