I have published a paper!

I am delighted to announce that my work on Delay Tolerant Networks has culminated into a paper, published 11 months since the project started!

In this project, I worked with a faculty of the CS department, Mr. Abhishek Thakur and a college senior, Tejashwar Reddy. Here, we showed that it is viable to send specially encoded videos using an opportunistic network of Android devices. We called it VECTORS, short for “VidEo Communication Through Opportunistic Relays and Scalable video coding”. Since it’s published under Elsevier’s Open Access policy, it does not require any subscription or institutional login to access.

DOI: https://doi.org/10.1016/j.softx.2018.12.006

As a continuation of this, I am currently at the Acoustic Research Laboratory at NUS for a project on underwater DTNs. But that merits a blog post of its own another time ;)

My Favourite Videogames (Part 2)

One post isn’t enough to do justice to the number of fantastic games I’ve played of late. I’ve played a large number of games from 2017-18 and I have three more that make the cut for the Greatest Of All Time in my books.

EarthBound (SNES)

Released 1995. Played 2017.

Released as Mother 2 for the Nintendo SNES in mid-90’s Japan, EarthBound is the quirkiest game I’ve ever played. Despite being over two decades old, the humour still feels fresh and the game mechanics are remarkably prescient of what future RPG’s would continue to use.

At its core, EarthBound is a 2D JRPG where 13-year old Ness has to the save the world from the wrath of an evil…force(?)…called Giygas. It has most of the core JRPG elements such as turn-based battles, multiple party members, inventory management, and a Health/Battle Points system. What really distinguishes EarthBound is the absolutely insane variety in the enemies. You battle absolutely everything - from a pile of puke to Blue KKK Cultists to taxicabs and Salvador Dali clocks. In a stark contrast to the brain-dead NPC dialogue in modern RPG’s, EarthBound has some of the funniest lines of NPC dialogue you will ever find. The soundtrack is enjoyable and surprisingly detailed in places, especially given the 4MB size of the game. To truly appreciate EarthBound, it’s best to spend some quality time playing it slowly. Every single frame of this game just oozes with charm.

Just EarthBound things

Perhaps the best thing about EarthBound is how fearlessly creative it is. Unlike contemporary videogames, EarthBound doesn’t hold back on breaking new ground for using familiar elements which would appeal to a broader audience. This fearless approach did result in EarthBound’s commercial failure on launch in North America, where it was puzzlingly marketed with the tagline “This Game Stinks!”. Perhaps Nintendo of America was banking a little too heavily on the gross fascinations of pubescent boys to turn EarthBound into a blockbuster success.

However, EarthBound is far from a perfect game. Being old as it is, not all of its mechanics have aged well. Put mildly, the inventory management system is a downright terrible. The turn-based combat gets very tedious after a while (though there is an AUTO battle mode to somewhat fix this). Moreover, the player characters move very slowly in relation to the size of the game world. The Run button from Pokémon games is sorely missed here. In all, it’s quite clear that the game was designed for an era in which there were far fewer competing distractions for one’s attention. As I played the game in an emulator, I 100x’ed some of the slower parts to get through the game. Nevertheless, a single playthrough of EarthBound was enough to convince me that this is a severely under-appreciated Nintendo classic. If videogames were works of art, EarthBound would be the forerunner of the Surrealist movement. It’s one of Shigesato Itoi’s and Iwata’s finest masterpieces.

EarthBound was a key inspiration for Toby Fox’s Undertale released in 2015. At first Undertale felt like a distilled version of EarthBound, sharing the same sense of humour, but with significantly better pacing and an interesting combat system. Yet, on finishing Undertale, I was more of the opinion that it was a watered down version of EarthBound. It didn’t capture my imagination the way EarthBound had.

After the release of EarthBound, Itoi and Nintendo worked on Mother 3. Mother 3 endured development hell for twelve years, being cancelled and reannounced for nearly three generations of Nintendo consoles. In the end, Mother 3 was finally released for the GBA in Japan in 2006 (two years after the release of the DS!). Unfortunately, Mother 3 still hasn’t been officially translated for an international release, making the series a lost Nintendo legend. If you’re interested in the story of EarthBound’s development and about the game itself, AVGN’s video is a great 40 minute primer:

Portal 1 (PC)

Released 2007. Played 2018.

I had a blast playing Portal 1 this January after picking it up in the Steam winter sale. It’s easily the most recognisable game on this list and it needs no introduction.

Portal 1 is the epitome of good game design. There’s minimal hand-holding and the levels are perfect in length and difficulty to give the player an intuitive understanding of the game mechanics before moving on to the more complicated puzzles. Instead of being ornamentally detailed, the visuals are just sufficient to convey essential information without being distracting.

There are no low points during the game and the entire experience is thoroughly enjoyable. Portal 1 demonstrates perfect pacing in a videogame. So many things in Portal 1 are just right. Given it takes only six hours to finish and is available for as little ₹30 on sale, it’s an excellent addition to anyone’s Steam library.

Portal 2 expanded on the formula established in Portal 1 with more complicated puzzles and new mechanics. The voice acting is even better with the snappy Wheatley and the hilariously deadpan delivery of Cave Johnson. However, the perfect pacing of Portal 1 didn’t carry over to Portal 2. In several mid-game levels, the cluttered environment and confusing lighting effects made it difficult to find a white tile to open a portal. I would have much rather had the simpler lighting model of Portal 1 as it was far easier to visually parse. That said, Portal 2 still reaches the mark for a notable mention in my list.

Spelunky (PC)

Released 2011. Played 2018 - Present.

The final game on this list is also the one I’ve played the most this year. I was lucky enough to snag it on the Steam winter sale last year and it’s easily the best ₹50 I’ve ever spent.

Spelunky is an insanely difficult 2D platformer with 16 randomly generated levels in each playthrough. There is no concept of saving progress in Spelunky and death is permenant. These two features make it impossible to play memorising the levels as you can in a conventional 2D platformer. Every moment of Spelunky is filled with compromise and multiple branching decisions. Picking the wrong one can send you all the way back to the beginning. As most monsters in the game have between 3-10 HP, the Spelunker isn’t even much stronger than common monsters with just 4 HP, 4 Bombs, and 4 Ropes at the start of each run.

This makes death an innate part of the progression system of the game. With the lack of in-game saving, the progression is the evolution of the player in becoming better at playing Spelunky. Each death teaches the player a bit more about its mechanics. Death in Spelunky is almost always fair, implying that more often than not, death is due to your own mistakes.

The randomly generated levels give Spelunky infinite replayabilty for no two runs in Spelunky ever share the same levels. Certain patterns and obstacles may become familiar after a while but clearing an entire level requires coming up with a plan on-the-fly for each section. The best moments of the game are had in using a unfamiliar new item to sweep through the game in ways I would have never imagined.

I think I’m done for the day.

Getting better in Spelunky is the most satisfying feeling I have ever experienced in a game. An entire run of Spelunky from 1-1 to the final boss in 4-4 can be done in as little as 20 minutes, but it take just as many hours to get good enough at the game to do so.

Spelunky is also chock-filled with comic relief and dark humour. I absolutely adore the cartoonish art style which belies the incredible difficulty of the game. Nothing in the game takes itself seriously and even dying thanks to a chain reaction of events can be absolutely hysterical at at times.

You can rescue Damsels in distress for a point of health, but you also have the option of sacrificing them to the altar of Kali for a free item. Damsels are also effective projectiles and can be used for killing other enemies by throwing them. Shopkeepers, the ornery merchants of the Spelunky world, are another source of great amusement. Items from shops can either be bought or stolen. Stealing items puts you at the contempt of all the shopkeepers who will do everything to kill you for the rest of the run. It is this kind of crucial decision making which really gets at the heart of what makes Spelunky great. Mark Brown’s excellent video explains the level generation system and goes into the crucial decision making process which makes Spelunky what it is:

The game is also incredibly mechanically sound. There are so many options for traversal of a level in which you need to go from the top to the bottom. Each tile of the Spelunky world can be destroyed using a well-placed bomb, making resource management an important theme of the game. Controls are also razor tight. Thanks to the low inertia of the Spelunker, playing the entire game on a keyboard is completely feasible. In fact, it’s my preferred way to play Spelunky.

Not only is it super addicting to play, it’s addicting to watch as well. My early forays in Spelunky were with my wingies jeering every death. I ended up becoming a far more careful player as a result. I tried to learn as much as I could about the game mechanics in every run. In some ways, this single player game brings people together more than some multiplayer games can.

Lastly, Spelunky also hosts a number of secrets under its 2D tiled world. The most significant of these is the much vaunted Hell run which requires true mastery of the game to complete. At the time of writing, I have clocked up 1500+ deaths in Spelunky, out of which I’ve reached Hell just five times.

I will go as far as to say that Spelunky is probably the best videogame I’ve ever played and I am absolutely HYPED for Spelunky 2 coming out in 2019.

The Last Lecture

I’ve read a lot of books in the last two months.

Page for page, I’ve already far surpassed my 2017 reading throughput this year. Along with my usual cache of fiction and space-flight books, I have read some beautifully written books about cancer and the process of dying such as “When Breath Becomes Air” and “Being Mortal”. As much as I hope that advances in medical science will outpace mortality in the next fifty years, death is the only certainty we can contend with in life. I found that reading these books helps me understand what happens at the End Of Life. It’s strangely comforting to have a deeper insight into death.

This is what brings me to The Last Lecture, a talk I stumbled upon when binging on Coding Horror.

Dr. Randy Pausch was a professor at CMU and was in the vanguard of the development of virtual reality in the 1990’s. Unfortunately, he was diagnosed with pancreatic cancer in 2006. After being told that his cancer had turned terminal, he delivered his final lecture at CMU about “Achieving Your Childhood Dreams” (linked above). It’s an incredibly inspirational talk and Randy Pausch is so charismatic that you can’t help but to watch the entire video in a single sitting.

Despite the premise, the talk is not about cancer or dying at all. Instead, Dr. Pausch shares anecdotes from his life about he chased his own whimsical childhood dreams.

I loved many parts of this lecture, so I have copied the parts of his excellent book which resonated me the most:

On failure:

Over the years, I also made a point of telling my students that in the entertainment industry, there are countless failed products. It’s not like building houses, where every house built can be lived in by someone. A video game can be created and never make it through research and development. Or else it comes out and no one wants to play it. Yes, video-game creators who’ve had successes are greatly valued. But those who’ve had failures are valued, too—sometimes even more so.

Start-up companies often prefer to hire a chief executive with a failed start-up in his or her background. The person who failed often knows how to avoid future failures. The person who knows only success can be more oblivious to all the pitfalls.

Experience is what you get when you didn’t get what you wanted. And experience is often the most valuable thing you have to offer.

It’s a cliché that experience is the best teacher. But, as Randy Pausch would have said, it’s a cliché for good reason, because it’s so damn true. From one of the most influential talks I’ve had the pleasure of attending in person, I have to come to realise that failing spectacularly is better than being a mediocre success.

It’s a point which I try to shoehorn into every public talk I give to juniors. Sure, you might get laughed at and trampled on for failing terribly, but once you’ve experienced that, you will realise that failure isn’t something to be all that scared about. You’ll learn from the mistakes you made when you failed, and eventually, you might even learn to laugh at yourself for failing the way you did.

On brick walls - the insurmountable obstacles that stop you from getting what you want:

The brick walls are there to stop the people who don’t want it badly enough. They’re there to stop the other people.

There’s a lot of meaning in those two sentences and it has really changed my perception of “not taking no for an answer”. The “other people” refers to those who don’t have the tenacity or the zeal to achieve their childhood dreams. The brick walls are a test for seeing if you’re really sure that you want something, be it a degree, a job, or maybe a way to finance your latest hobby. I feel like a lot of challenging decisions in life can become a lot clearer after perceiving it this way. If you don’t find the motivation to attempt climbing the brick wall between you and your dreams, then maybe those dreams weren’t yours in the first place.

On the ultimate purpose of teachers:

In the end, educators best serve students by helping them be more self-reflective. The only way any of us can improve—as Coach Graham taught me—is if we develop a real ability to assess ourselves. If we can’t accurately do that, how can we tell if we’re getting better or worse?

Self-reflection and introspection are very underrated. It’s easy to reduce education to a commercial service (only more true these days than ever) and teachers as the providers. But, education is something of a “head fake”. As Randy Pausch describes it, head fakes are a type of indirect learning, often more important than the content of what is being taught. He elaborates on this by saying:

When we send our kids to play organized sports—football, soccer, swimming, whatever—for most of us, it’s not because we’re desperate for them to learn the intricacies of the sport.

What we really want them to learn is far more important: teamwork, perseverance, sportsmanship, the value of hard work, an ability to deal with adversity.

I can’t remember the conditions needed for each oxidation state of Manganese, but I do remember learning how to learn and how to get through some very sticky situations. Most of the benefits I’ve gleaned from studying from school are less quantifiable than a number of exam results, but I wouldn’t trade those life lessons for anything. It’s this kind of meta education which I find sorely lacking at BITS. I miss the deeper perspectives on my subjects which made me question the way things are. Fiercely independent college students may not need the same hand-holding as naive school students, but the need for some kind of guidance is just as necessary.

On hard work:

A lot of people want a shortcut. I find the best shortcut is the long way, which is basically two words: work hard.

As I see it, if you work more hours than somebody else, during those hours you learn more about your craft. That can make you more efficient, more able, even happier. Hard work is like compounded interest in the bank. The rewards build faster.

On apologising to people:

Apologies are not pass/fail. I always told my students: When giving an apology, any performance lower than an A really doesn’t cut it.

Halfhearted or insincere apologies are often worse than not apologizing at all because recipients find them insulting. If you’ve done something wrong in your dealings with another person, it’s as if there’s an infection in your relationship. A good apology is like an antibiotic; a bad apology is like rubbing salt in the wound.

My Favourite Videogames (Part 1)

I love videogames. I can’t call myself a serious gamer, but I’ve played my share of good and bad games ever since I started playing Super Mario Advance 4 on a GBA SP fourteen years ago. Some games have left a deeper impression on me than others. Objectively, the games in this list aren’t necessarily the best I have ever played, but they are the ones with which I’ve made the most memories.

Microsoft Flight Simulator 2004 (PC)

Released in 2003. Played from 2005 - 2013.

This is the first major PC game I’ve played. My dad got this for me with a Logitech Extreme 3D Pro joystick on one of his business trips. Eight year old me was blown away by the packaging of the game - it came with several booklets in a box, and the game itself was split in as many as four disks! I fondly remember being awed at the number of pre-loaded airplanes, the 3D graphics (and gorgeous virtual cockpits!), and how hellishly difficult it was to play on our vintage 1998 desktop. The poor desktop’s Pentium 3 CPU could only just sputter out 5-6 fps at the lowest settings on a good day. Not to mention that the game’s relatively enormous 2.5 GB installation size took up more than half of the computer’s 5GB HDD. Many years later, I realised that dealing with these absurd hardware limitations is what got me into computers in the first place.

Nevertheless, my desktop’s measly hardware hardly deterred the tenacious pilot in me. I enjoyed piloting small aircraft like the Piper Cub just as much as I enjoyed flying the 747-400. There was something so very satisfying in landing a plane safely with the sun creeping over the horizon at dawn. It didn’t take me long to find the very active modding scene for FS2004 on websites like Simviation.com. Several add-ons are still actively developed to this day.

However, it wasn’t until we got a new computer in 2008 that I really got to see the game for what it was. The new computer was more than capable of pushing out 60 fps in Ultra High quality. Scenery details, water shaders, and even the runway textures had a photo-realistic effect to them. I scoured every glitched nook and cranny of the Flight Simulator world. I mastered the art of flaring commercial aircraft before landing and taxiing it back to the terminal gate.

At this point, my add-on collection had swollen to over 10GB, nearly four times as large as the original game. There were several oddballs in my collection - including a Mitsubishi Pajero, an Asuka Cruise Ship, and a Bald Eagle which flew like a glider for all intents and purposes. My favourites among my add-on collection were the MD-9, DC-10, OpenSky 737-800, XB-70 Valkyrie, and a an amazing Concorde pack complete with dozens of different liveries. I also liked flying the Tupelov Tu-95 and the Lockheed U-2 “Dragon Lady”. Alas, I couldn’t shift my joystick setup after moving to college. Had I been able to, there is a good chance I would still be playing it - fifteen years after the game’s initial release.

Need For Speed: Most Wanted (PC)

Released in 2005. Played from 2008 - 2011.

The only racing game in this list and my favourite overall by far. NFSMW won’t win any awards for realism in its driving physics (go play DiRT if you’re a stickler for realism), but it certainly will for mindless fun. There is so much that EA got right with this game - the fresh, overbright visuals, excellent world design with plenty of shortcuts and obstacles to crush unsuspecting cop cars, a good progression system, and a wicked smart cop AI to make cop chases true white-knuckle affairs.

The storyline is simple and the game has plenty of expository cutscenes to make you hate your main rival, Clarence “Razor” Callahan for stealing your brand-new BMW M3 GTR. There are a total of fifteen bosses. On beating each boss, you get a chance to “steal” their uniquely styled car through a lottery system. Winning a boss’s car made it much easier to progress through the game. Each car in the game’s roster can be customised in every aspect and I spent several hours tuning and buying parts for all the cars in my garage to suit my playing style.

The sheer excitement in cop chases is something I haven’t found in any NFS game I’ve played since MW. On maxing out a car’s “Heat”, the cops’ cars would be upgraded from the bovine Ford Crown Victoria to the absurdly fast Chevy Corvette. Being able to tap on to the police chatter was a superb game mechanic, letting one plan their way around road-blocks and spike-traps. The game also offered plenty of room for creativity to get out of tight situations in cop chases. Lastly, beating Razor at the end is right up there with the most badass moments I’ve ever had in a videogame.

I did play a handful of NFS games after MW, but they just fell flat on the fun aspect which MW had in droves. NFS ProStreet brought car damage and photo-realistic graphics, but the serious racing style was a huge tonal shift from the absurd shenanigans of MW. NFS Shift was cut from the same cloth as ProStreet. Hot Pursuit looked like a return to form for EA, but sadly it couldn’t live up to my expectations in gameplay, which just felt clunky compared to MW. I do look forward to playing the 2012 remake of MW to see if it can replicate the charm of the first one.

Pokemon Black (DS)

Released in 2011. Played from 2011 - 2014.

Pokemon Black is not the first Pokemon game I’ve played, but it is the first I have played on a Nintendo console after rekindling my love for Pokemon. Coming off playing Platinum (which I had also enjoyed), Pokemon Black was a breath of fresh air. For perhaps the only time in the series’ history, GameFreak took a number of risks, most of which paid off. In Black & White only brand-new Unova Pokemon are available in the wild before beating the Elite Four. This forced players to learn the game’s new mechanics and adopt new battling strategies. The visuals were the best I had ever seen from a DS game with lovingly animated Pokemon sprite-work in the battles. Even the storyline was actually thought provoking and it was far better than the “Ten Year Old Beats Team After World Domination" narrative that had been done to death in the earlier games.

The overworld was rich and varied and the game’s season mechanic changed the appearance of several areas over the course of a year. The music was fantastic in several places and a major step up from the Gen IV themes.

Being a Pokemon game, it wasn’t completely free of hand-holding, but at least it was far better than Sun & Moon which came half a decade after the release of Black & White. My major gripe with BW is that Unova is much too linear and literally forces you halfway around the map to reach the Elite Four.

However, the regular gameplay was just a distraction for the wonderful Competitive Battling scene. This was the first Pokemon game in which I put my new-found knowledge of EVs and IVs to fruition. The Gen V Wi-Fi Competitive Battling scene was amazing thanks to Team Preview, BattleSpot, and International Tourneys held online every few months. Thanks to RNG manipulation and the help of an equally geeky friend, I managed to breed a full team of 6x31 IV Pokemon with ideal Natures, all obtained through completely legal means. I discovered my love of Double Battles through the International Tourneys, which I still maintain is the best way to play Pokemon. My go to team in the Tourneys was an immensely bulky Tailwind Suicune with a wall-breaking Life Orb Gengar, backed up by a Physical Fighting Gem Virizion, and a Toxic Orb Gliscor or a Sitrus Berry Gyarados for shutting down any physical attackers. This unconventional team checked a wide variety of the common suspects in Gen V Doubles, most importantly the nearly omnipresent Rotom-W.

Pokemon also gave me my first taste of online community. I joined Smogon in early 2012. My Smogon username still perpetuates in every onlineaccount I’ve made since then. I watched the birth of the Gen V metagame with my own eyes. Movesets, suspect tests, and analyses were uploaded and discussed at a furious pace on the Dragonspiral Tower sub-forum. Through briefly working for Smogon’s C&C team and #grammar on Smogon IRC, I made some great online friends, some of whom I still speak with to this day. The community was warm and supportive. It’s one of the many things I can fondly remember being a part of back in 2012.

After Pokemon Black, I played Pokemon Alpha Sapphire in late 2015. AS was a delightful blast from the past and it fully satiated my decade-long nostalgia for Hoenn. This was in no small part thanks to the post-game Delta Episode which is one of my favourite sequences of any Pokemon game.

About a year later, I picked up Pokemon Moon but I could not get into it at all. I was thoroughly underwhelmed by the constant hand-holding which severely detracted from the core experience of the game. My aging 3DS died at about the same time I finished it, marking a fitting end to my relationship with the series for the time being. Here’s hoping Gen VIII on the Switch is truly a special experience which Pokemon players have been missing since Black came out.

Introducing APStreamline!

Download APSync with APStreamline built in from here (BETA)!

I’m very excited to announce the release of a network adaptive video streaming solution for ArduPilot!

Links to the code

• APStreamline
• APWeb server
• My APWeb PR

About

The APSync project currently offers a basic video streaming solution for the Raspberry Pi camera. APStreamline aims to complement this project by adding several useful features:

• Automatic quality selection based on bandwidth and packet loss estimates

• Selection of network interfaces to stream the video

• Options to record the live-streamed video feed to the companion computer

• Manual control over resolution and framerates

• Multiple camera support using RTSP

• Hardware-accelerated H.264 encoding for the Raspberry Pi

• Camera settings configurable through the APWeb GUI

I’m chuffed to say that this has not only met the requirements for the GSoC project but has also covered the stretch goals I had outlined in my proposal!

Streaming video from robots is an interesting problem. There are several different use cases – you might be a marine biologist with a snazzy BlueROV equipped with several cameras or a UAV enthusiast with the itch of flying FPV. APStreamline caters to these and several other use cases.

While APStreamline works on the all network interfaces available on the Companion Computer (CC), its main value lies in the case of streaming on unreliable networks such as Wi-Fi as in most cases, we use the Companion Computer (CC) in Wi-Fi hotspot mode for streaming the video. Due to the limited range of 2.4GHz Wi-Fi, the Quality-of-Service (QoS) progressively gets worse when the robot moves further away from the receiving computer.

This project aims to fixes problem by dynamically adjusting the video quality in realtime. Over UDP we can obtain estimates of QoS using RTCP packets received from the receiver. These RTCP packets provide helpful QoS information (such as RTT and packet loss) which can be used for automatically changing the bitrate and resolution of the video delivered from the sender.

Running the Code

Hardware

All the following instructions are for installing APStreamline and APWeb on the CC. A Raspberry Pi 2/3/3B+ with the latest version of Raspian or APSync is a good choice. Intel NUC’s are excellent choices as well.

Do note that the Raspberry Pi 3 and 3B+ have very low power Wi-Fi antennae which aren’t great for video streaming. Using a portable Wi-Fi router like the TPLink MR-3020 can dramatically improve range. Wi-Fi USB dongles working in hotspot mode can help as well.

Installing APStreamline

Install the gstreamer dependencies:

sudo apt install libgstreamer-plugins-base1.0* libgstreamer1.0-dev libgstrtspserver-1.0-dev gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly python3-pip 

Install meson from pip and ninja for building the code:

sudo pip3 install meson 

sudo apt install ninja-build 

Navigate to your favourite folder folder and run:

git clone -b release https://github.com/shortstheory/adaptive-streaming 

meson build 

cd build 

sudo ninja install # installs to /usr/local/bin for APWeb to spawn 

On the Raspberry Pi, use sudo modprobe bcm2835-v4l2 to load the V4L2 driver for the Raspberry Pi camera. Add bcm2835-v4l2 to /etc/modules for automatically loading this module on boot.

Installing APWeb

The APWeb server project enables setting several flight controller parameters on the fly through the use of a Companion Computer (e.g. the Raspberry Pi). We use this APWeb server for configuring the video streams as well.

Clone the forked branch with APStreamline support here:

git clone -b video_streaming https://github.com/shortstheory/APWeb.git 

cd APWeb 

Install libtalloc-dev and get the MAVLink submodule:

sudo apt-get install libtalloc-dev 

git submodule init 

git submodule update 

Build APWeb:

cd APWeb 

make 

./web_server -p 80 

In case it fails to create the TCP socket, try using sudo ./web_server -p 80. This can clearly cause bad things to happen so be careful!

Usage

Video livestreams can be launched using RTSP. It is recommended to use RTSP for streaming video as it provides the advantages of supporting multiple cameras, configuring the resolution on-the-fly, and recording the livestreamed video to a file.

APWeb

Start the APWeb server. This will serve the configuration page for the RTSP stream server. Connect to the web server in your favourite web browser by going to the IP address of the Companion Computer.

On navigating to the new video/ page, you will be presented with a page to start the RTSP Server:

On selecting the desired interface and starting the RTSP Server, the APWeb server will spawn the Stream Server process. The stream server will search for all the V4L2 cameras available in /dev/. It will query the capabilities of all these cameras and select hardware encoding or software encoding accordingly. The list of available cameras can be refreshed by simply stopping and starting the server.

From here, the APWeb page will display the list of available RTSP streams and their mount points:

The video quality can either be automatically set based on the available network bandwidth or set manually for more fine-grained control.

The APWeb page also presents an option to record the video stream to a file on the CC. For this the video stream must be opened on the client. This works with any of the manually set resolutions but does not work with Auto quality. This is because the dynamically changing resolution causes problems with the file recording pipeline. An exception to this is the UVC cameras which can record to a file in Auto mode as well.

The RTSP streams can be viewed using any RTSP player. VLC is a good choice. Some GCS such as QGroundControl and Mission Planner can also stream video over RTSP.

For example, this can be done by going to “Media > Open Network Stream” and pasting in the RTSP Mount Point for the camera displayed in the APWeb configuration page. However, VLC introduces two seconds of latency for the jitter reduction, making it unsuitable for critical applications. To circumvent this, RTSP streams can also be viewed at lower latency by using the gst-launch command:

gst-launch-1.0 playbin uri=<RTSP-MOUNT-POINT> latency=100

As an example RTSP Mount Point looks like: rtsp://192.168.0.17:8554/cam0. Refer to the APWeb page to see the mount points given for your camera.

Standalone

Launch the RTSP stream server by running:

stream_server <interface>

The list of available network interfaces can be found by running ifconfig. Streams can be connected to using gst-launch.

Things to Try Out

• Use a variety of different cameras and stream several at the same time

• Try recording the live-streamed video to a file on the CC

• Play with the Auto quality streaming feature on different types of network interfaces