21 Jul 2016
I’m a lot closer to finishing the project now. Thanks to some great support from my GSoC mentor, my project has turned out better than what I had written about in my proposal! Working together, we’ve made a lot of changes to the project.
For starters, we’ve changed the name of the ioslave from “File Tray” to “staging” to “stash”. I wasn’t a big fan of the name change, but I see the utility in shaving off a couple of characters in the name of what I hope will be a widely used feature.
Secondly, the ioslave is now completely independent from Dolphin, or any KIO application for that matter. This means it works exactly the same way across the entire suite of KIO apps. Given that at one point we were planning to make the ioslave fully functional only with Dolphin, this is a major plus point for the project.
Next, the backend for storing stashed files and folders has undergone a complete overhaul. The first iteration of the project stored files and folders by saving the URLs of stashed items in a QList in a custom “stash” daemon running on top of kded5. Although this was a neat little solution which worked well for most intents and purposes, it had some disadvantages. For one, you couldn’t delete and move files around on the ioslave without affecting the source because they were all linked to their original directories. Moreover, with the way ‘mkdir’ works in KIO, this solution would never work without each application being specially configured to use the ioslave which would entail a lot of groundwork laying out QDBus calls to the stash daemon. With these problems looming large, somewhere around the midterm evaluation week, I got a message from my mentor about ramping up the project using a “StashFileSystem”, a virtual file system in Qt that he had written just for this project.
The virtual file system is a clever way to approach this - as it solved both of the problems with the previous approach right off the bat - mkdir could be mapped to virtual directory and now making volatile edits to folders is possible without touching the source directory. It did have its drawbacks too - as it needed to stage every file in the source directory, it would require a lot more memory than the previous approach. Plus, it would still be at the whims of kded5 if a contained process went bad and crashed the daemon.
Nevertheless, the benefits in this case far outweighed the potential cons and I got to implementing it in my ioslave and stash daemon. Using this virtual file system also meant remapping all the SlaveBase functions to corresponding calls to the stash daemon which was a complete rewrite of my code. For instance, my GitHub log for the week of implementing the virtual file system showed a sombre 449++/419–. This isn’t to say it wasn’t productive though - to my surprise the virtual file system actually worked better than I hoped it would! Memory utilisation is low at a nominal ~300 bytes per stashed file and the performance in my manual testing has been looking pretty good.
With the ioslave and other modules of the application largely completed, the current phase of the project involves integrating the feature neatly with Dolphin and for writing a couple of unit tests along the way. I’m looking forward to a good finish with this project.
You can find the source for it here: https://github.com/KDE/kio-stash (did I mention it’s now hosted on a KDE repo? ;) )
21 Jun 2016
This project has been going well. Though it was expectedly difficult in the beginning, I feel like I am on the other side of the learning curve now. I will probably make a proper update post sometime later this month. My repo for this project can be found here: https://github.com/shortstheory/kio-stash
For now, this is a small tutorial for writing KDE I/O slaves (KIO slaves) which can be used for a variety of KDE applications. KIO slaves are a great way for accessing files from different filesystems and protocols in a neat, uniform way across many KDE applications. Their versatility makes them integral to the KIO library. KIO slaves have changed in their structure the transition to KF5 and this tutorial highlights some of these differences from preceding iterations of it.
Project Structure
For the purpose of this tutorial, your project source directory needs to have the following files.
- kio_hello.h
- kio_hello.cpp
- hello.json
- CMakeLists.txt
If you don’t feel like creating these yourself, just clone it from here: https://github.com/shortstheory/kioslave-tutorial
hello.json
The .json file replaces the .protocol files used in KIO slaves pre KF5. The .json file for the KIO slave specifies the properties the KIO slave will have such as the executable path to the KIO slave on installation. The .json file also includes properties of the slave such as being able to read from, write to, delete from, among many others. Fields in this .json file are specified from the KProtocolManager class. For creating a KIO slave capable of showing a directory in a file manager such as Dolphin, the listing property must be set to true. As an example, the .json file for the Hello KIO slave described in this tutorial looks like this:
{
"KDE-KIO-Protocols" : {
"hello": {
"Class": ":local",
"X-DocPath": "kioslave5/kio_hello.html",
"exec": "kf5/kio/hello",
"input": "none",
"output": "filesystem",
"protocol": "hello",
"reading": true
}
}
}
As for the CMakeLists.txt, you will need to link your KIO slave module with KF5::KIOCore. This can be seen in the project directory.
kio_hello.h
#ifndef HELLO_H
#define HELLO_H
#include <kio/slavebase.h>
/**
This class implements a Hello World kioslave
*/
class Hello : public QObject, public KIO::SlaveBase
{
Q_OBJECT
public:
Hello(const QByteArray &pool, const QByteArray &app);
void get(const QUrl &url) Q_DECL_OVERRIDE;
};
#endif
The Hello KIO slave is derived from KIO::SlaveBase. The SlaveBase class has some basic functions already implemented for the KIO slave. This can be found in the documentation. However, most of the functions of SlaveBase are virtual functions and have to be re-implemented for the KIO slave. In this case, we are re-implementing the get function to print a QString when it is called by kioclient5.
In case you don’t need special handling of the KIO slave’s functions, you can derive your KIO slave class directly from KIO::ForwardingSlaveBase. Here, you would only need to re-implement the rewriteUrl function to get your KIO slave working.
kio_hello.cpp
#include "hello.h"
#include <QDebug>
class KIOPluginForMetaData : public QObject
{
Q_OBJECT
Q_PLUGIN_METADATA(IID "org.kde.kio.slave.hello" FILE "hello.json")
};
extern "C"
{
int Q_DECL_EXPORT kdemain(int argc, char **argv)
{
qDebug() << "Launching KIO slave.";
if (argc != 4) {
fprintf(stderr, "Usage: kio_hello protocol domain-socket1 domain-socket2\n");
exit(-1);
}
Hello slave(argv[2], argv[3]);
slave.dispatchLoop();
return 0;
}
}
void Hello::get(const QUrl &url)
{
qDebug() << "Entering function.";
mimeType("text/plain");
QByteArray str("Hello world!\n");
data(str);
finished();
qDebug() << "Leaving function";
}
Hello::Hello(const QByteArray &pool, const QByteArray &app)
: SlaveBase("hello", pool, app) {}
#include "hello.moc"
The .moc file is, of course, auto-generated at compilation time.
As mentioned earlier, the KIO Slave’s .cpp file will also require a new KIOPluginForMetaData class to add the .json file. The following is used for the hello KIO slave and can be used as an example:
class KIOPluginForMetaData : public QObject
{
Q_OBJECT
Q_PLUGIN_METADATA(IID "org.kde.kio.slave.hello" FILE "hello.json")
};
CMakeLists.txt
cmake_minimum_required(VERSION 3.5)
set(QT_MIN_VERSION "5.4.0")
set(KF5_MIN_VERSION "5.16.0")
find_package(ECM ${KF5_MIN_VERSION} REQUIRED NO_MODULE)
set(
CMAKE_MODULE_PATH
${CMAKE_MODULE_PATH}
${ECM_MODULE_PATH}
${ECM_KDE_MODULE_DIR}
)
include(KDEInstallDirs)
include(KDECMakeSettings)
include(KDECompilerSettings NO_POLICY_SCOPE)
include(ECMSetupVersion)
include(FeatureSummary)
add_library(kio_hello MODULE hello.cpp)
find_package(KF5 ${KF5_MIN_VERSION} REQUIRED KIO)
target_link_libraries(kio_hello KF5::KIOCore)
set_target_properties(kio_hello PROPERTIES OUTPUT_NAME "hello")
install(TARGETS kio_hello DESTINATION ${KDE_INSTALL_PLUGINDIR}/kf5/kio )
Installation
Simply run the following commands in the source folder:
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr -DKDE_INSTALL_USE_QT_SYS_PATHS=TRUE ..
make
sudo make install
kdeinit5
As shown above, we have to run kdeinit5 again so the new KIO slave is discovered by KLauncher and can be loaded when we run a command through an application such as kioclient5.
Run:
kioclient5 'cat' 'hello:/'
And the output should be:
30 May 2016
I have officially started my GSoC project under the mentorship of Boudhayan Gupta and Pinak Ahuja.
The project idea’s implementation has undergone some changes from what I proposed. While the essence of the project is the same, it will now no longer be dependent on Baloo and xattr. Instead, it will use a QList to hold a list of staged files with a plugin to kiod. My next milestone before the mid-term evaluation is to implement this in a KIO slave which will be compatible with the whole suite of KDE applications.
For the last two weeks, I’ve been busy with going through hundreds of lines of source code to understand the concept of a KIO slave. The KIO API is a very neat feature of KDE - it provides a single, consistent way to access remote and local filesystems. This is further expanded to KIO slaves which are programs based on the KIO API which allow for a filesystem to be expressed in a particular way. For instance, there is a KIO slave for displaying xattr file tags as a directory under which each file marked to a tag would be displayed. KIO slaves even expand to network protocols allowing for remote access using slaves such as http:/, ftp:/, smb:/ (for Windows samba shares), fish:/, sftp:/, nfs:/, and webdav:/. My project requires virtual folder constructed of URLs stored in a QList - an ideal fit for KIO slaves.
However, hacking on KIO slaves was not exactly straightforward. Prior to my GSoC selection, I had no idea on how to edit CMakeLists.txt files and it was a task to learn to make one by hand. Initially, it felt like installing the dependencies for building KIO slaves would almost certainly lead to me destroying my KDE installation, and sure enough, I did manage to ruin my installation. Most annoying. Fortunately, I managed to recover my data and with a fresh install of Kubuntu 16.04 with all the required KDE packages, I got back to working on getting the technical equivalent of a Hello World to work with a KIO slave.
This too, was more than a matter of just copying and pasting lines of code from the KDE tutorial. KIO slaves had dropped the use of .protocol files in the KF5 transition, instead opting for JSON files to store the properties of the KIO slave. Thankfully, I had the assistance of the legendary David Faure. Under his guidance, I managed to port the KIO slave in the tutorial to a KF5 compatible KIO slave and after a full week of frustration of dealing with dependency hell, I saw the best Hello World I could ever hope for:
Baby steps. The next step was to make the KIO slave capable of displaying the contents of a specified QUrl in a file manager. The documentation for KProtocolManager made it seem like a pretty straightforward task - apparently that all I needed to do was to add a “listing” entry in my JSON protocol file and I would have to re-implement the listDir method inherited from SlaveBase using a call to SlaveBase::listDir(&QUrl). Unbeknownst to me, the SlaveBase class actually didn’t have any code for displaying a directory! The SlaveBase class was only for reimplementing its member functions in a derived class as I found out by going through the source code of the core of kio/core. Learning from my mistake here I switched to using a ForwardingSlaveBase class for my KIO slave which instantly solved my problems of displaying a directory.
Fistpump
According to my timeline, the next steps in the project are
- Finishing off the KIO slave by the end of this month
- Making GUI modifications in Dolphin to accommodate the staging area
- Thinking of a better name for this feature?
So far, it’s been a great experience to get so much support from the KDE community. Here’s to another two and a half months of KDE development!
23 Apr 2016
Wow!
I have been selected for the Google Summer of Code!
For the better part of the summer vacation, I will now be committing myself to write code for KDE to implement my project idea of implementing a virtual folder in Dolphin to make it easier to select files.
As a primer, the Google Summer of Code (GSoC) is an annual event organized by Google for drawing students to work on open source projects with a nice stipend, goodies, and fame. The GSoC is a term one would hear pretty frequently when talking about the technical prowess and coding culture of a college.
This year, BITS Pilani - Hyderabad Campus had a record number of a total of 7 selections! This is more than double of our previous record. This in some ways, might be the start of the technical culture wave this campus was looking for so long.
The File Tray idea for the GSoC came at a difficult time, hardly a month after the entrance exams in 2015. It was at a time I was frustrated with everything I had done and I didn’t have any energy to pursue anything at all having been completely drained out by the entrance exams prior to it. From there, the project sat on the list of “Things I Might Do In The Distant Future”. The project idea was known only to a few close friends and my tiny programming diary.
It was only till November 2015 when I stumbled across the GSoC. I began looking at open source file managers for which I could implement my project idea. I had been using Linux with various desktop environments for about 4 years at that point, so I had a pretty decent idea of what to look for. Writing this feature for GNOME’s Nautilus was the first thing I looked into as I had been using Nautilus for a while and I was a big fan of Nautilus’s simple to use interface. But, the problem was that Nautilus was a C/GTK+ project and I had no desire to move on to using C after having C++ in my comfort zone for a very long time. Fortunately, Dolphin, one of the best file managers I had used since my days of using KDE, used C++/Qt, a toolset I am much better with. I felt my project idea was a natural fit for Dolphin’s Split view mode. KDE also had an excellent record in the GSoC with a very good number of slots and a high percentage of successful projects. This began my tryst with Open Source development.
From there on, I taught myself Qt and during a Diwali vacation on campus, I managed to make a very rough prototype application of my project after coding for 6 hours straight from 11pm to 5am the next day. Following this, I subscribed to KDE’s mailing lists and after lurking around for a while, I started asking for feedback on my GSoC idea. With surprisingly positive feedback from numerous KDE developers, I realized that there might just be a non-zero chance of getting selected.
Things quickly began falling into place and I then moved on to the next step of hunting around for bugs I could fix and new features I could implement for Dolphin. The bug-fixing was as enjoyable as it was occasionally frustrating. Reading over 20000 lines of code certainly took its toll when I had no idea when how different parts of the application meshed together. In the end, thanks to the guidance of Dolphin maintainer, Emmanuel Pescosta, I managed to fix a couple of things for Dolphin and moved on to the next step of making a proposal for my GSoC application.
Starting off with making a competent proposal was like launching off ground zero as there were very few people who had successfully completed the GSoC from our campus and most of these people had graduated well before this time. I started digging around for proposals accepted by KDE in previous GSoC’s. What I couldn’t get from all the proposals was some sound advice from seniors. In particular, Naveen Jafer bhaiya (who also went on to achieve a GSoC project of his own!) helped me with making my proposal as good as possible. In the end, after painstakingly checking every word in my proposal for what felt like the fiftieth time I submitted it on 25 March, only to spend an anxious month waiting for the results which came out at 1230am IST on April 23. While it still hasn’t sunk in yet (!), I am sure that this will make for an awesome summer vacation!
13 Apr 2016
I spend a lot of time thinking since I’ve joined college.
I had postponed a lot of introspection during the two years I spent slogging away for the JEE. But now, with ample free time and practically no requirement to go to classes, I’ve finally got some time to look back and see how things have turned out. College started off on the back foot, and while 1-1 had its charms, I wasn’t exactly happy about how things had turned out at the time due to a bad time with the entrance exams for all the wrongreasons.
But I digress, as time has passed, I have learned to live with my failure a little better every day, though it still sticks out like a sore thumb on an otherwise decent academic profile. Despite this, it has been an interesting exercise is to compare what I expected from college a year ago and reality.
To be fair, BITS Pilani (this applies to all campuses, but in this case, Hyderabad Campus) has some of best internal systems among all Indian colleges. Optional attendance, good grading system, decent infrastructure, and a lot of freedom is more than what can be asked for in a lot of other colleges. Despite some glaring flaws such as the lack of a solid technical culture, this college has punched above its weight for a new institution.
But as an engineering utopia? I feel like we are way short of the mark AaronSwartz mentioned in his blog:
“Perhaps it’s natural, when doing something so greedy and practical as a startup, to pine for the idealized world of academia. Its image as a place in an idyllic location filled with smart people has always been attractive; even more so with the sense that by being there one can get smarter simply through osmosis. People describe a place of continual geekiness, of throwing chemicals into the river and building robots in free time. A magical place for hackers to just enjoy themselves.”
This aside, I am of the opinion that the version of me a year ago would have been sorely disappointed by the version of me today. I feel that I was much more hardworking and efficient back at that time. The two years in JEE preparation were undoubtedly the worst years of my school life but now looking back, those dark days brought out the best in me in the briefest of moments. Had I not prepared for JEE at all, I would have had no idea just how driven and hard-working I could be for a goal that would always be just a touch out of view.
Despite my frustrations with life during JEE preparation, the epiphanies I used to have on weekly basis with studying physics kept me going. It was a positive feedback loop with no goading required. On the other hand I can’t remember the last time I actually enjoyed learning something in class in this college. I hope it isn’t a sign of things to come when I start “engineering” coursework in my second year but as of now, I have pretty much lost all motivation to study. The unbridled enthusiasm I used to have when studying for the entrance exams and the giddy thoughts of making batshit crazy projects in college has dwindled. In my first semester, it was a convenient excuse to blame this on burnout after pushing my limits for two years but I’ve come to realize that the reason is probably shallower than that. It’s not just with academics though - wasting time still feels painful but I have nothing I want to do to fill in the gaps. Is there a cause for this? Probably. Have I figured it out? Absolutely not.
It feels like an artificial conflict of time between these misguided academic pursuits and to actually work on something worthwhile. I could put up with it in school with the thought that there would be enough free time to pursue this in college - and while there is - it begs the question why such artificial restraints on time in the form of exams are always looming in the first place.
At this point of time, I don’t know what to do. With compre in half a month but a GSoC project and a couple other projects I’ve planned in the pipeline, it’s a pretty easy decision to make the choice of which one of these two things I would want to work on. For a CGPA for which I cannot care for anymore, it might be one of the worst decisions I can make.