The urge to do

Programming and debugging are  activities where we should think a lot and do little. Commonly what happens is the reverse. We are driven by the urge to do something.
It's particularly true for debugging. We see a  problem, the first instinct is to print something, build and rerun. And then print some more. In the end when we discover the problem we go: Ah, that wasobvious. I could have gotten there faster instead of these x iterations. And on complex systems these iterations take a long long time.
It often helps to resist the urge to do  and think what could be going off here. It saves a ton of time.
So also for programming.
A lot of programming time is spent in the edit build debug cycle. We make numerous iterations through this cycle to get to our goal. How could we optimize this? What if we are allowed only 1 iteration through this?
To ensure that I put enough thought in the process, recently I created this game for myself. After breaking the problem into subproblems I take it as a challenge that every submodule that is built will run completely in the first iteration itself. I get to decide the size of the submodule, but my code should do exactly what I intended in the first shot. Over time I have found that this leads to a lot of efficiency by forcing me to think of all the dependencies and side effects upfront.
Of course not all problems can be addressed with this method. Especially if it's a new technology or domain you learn much faster through each iteration. But for a lot of cases this approach does magic. You should try it out...

Defining Objects

Earlier in my career, I loved browsing through the Linux kernel's source code. Mostly to satisfy my curiosity about how certain things are internally implemented. As I read through more and more files of code, I started noticing one common pattern. For the most part, the code was organised around  abstract concepts (spin-lock, work-queue, kthreads etc). And every abstract concept had

• An object representation, a C structure that holds details about that structure
• An init/alloc call, that initialises the object
• The init/alloc call also returns a handle to this object
• A number of operations that accept this handle as a parameter and then operate on that object
• And finally a deinit/free call that destroys that object and also makes the handle invalid for any further use

Oh yes, you say, we had learnt it in our classes, no big deal! No, no, no. I mean, I had also learnt about it in my classes, but I had never ever used it practically in my code, apart from the stack example in my class assignment.

This was a good way to see it in practice in the Linux kernel, and that too used at multiple places. We even use it in our everyday programming lives. For example,

•  File IO:
• fd = open(...);
• read(fd, ...);
• write(fd, ...);
• close(fd);
• Socket IO:
• socket_fd = socket(...);
• bind(socket_fd, ...);
• connect(socket_fd, ...);
• send(socket_fd, ...);
• close(socket_fd);
• Pthreads: (pthread_create)
• OpenSSL: (SSL_new)
• Semaphores: (sem_init)
• and numerous others

So what is really my point here? Thinking in this way, helps you focus on defining these objects. And thinking about these objects helps you have a more modular and cleaner approach to software development. Design your data structures first, and the rest automatically follows. 

Next time you add a global int/char variable, think if there is a relationship among those data structures. Do they belong into a structure together? Can you make things easier by defining an abstract concept/object and operations around that object?

Here is what Linus Torvalds (the founder of the Linux kernel and git) has to say about this:

I'm a huge proponent of designing your code around the data,
rather than the other way around, and I think it's one of the
reasons git has been fairly successful (*).
...
(*) I will, in fact, claim that the difference between a bad
programmer and a good one is whether he considers his code or
his data structures more important. Bad programmers worry
about the code. Good programmers worry about data structures
and their relationships.

Milestones

Let's do an experiment. I assume you have been working on a project (academic or professional) for some time now. Today, setup some realistic goal as to what you want to achieve at the end of this week. And in the morning, every day of this week, setup a sub-goal as to what you wish to finish at the end of that day. At the end of the week, evaluate whether you were more productive than the weeks before this. Repeat this exercise for another week and check if you were more productive.

To me, I find myself more productive when I do this. Moreover, I find that things just keep sliding when I do not do this. It just goes on and on, with no concrete outcome.

For one, setting milestones help you pace yourself through the day. Without it, its just work-in-work-out, a chore. With a target at the end of the day, your whole approach is different. You are focused on results, and achieving them is so satisfying.

Secondly, setting milestones helps you focus on the prioritisation and planning activity, which is particularly overlooked early in your career. Nothing fancy, just a small enough time frame to experiment and improve week over week.

It has so often happened to me, that I go: "Hmm, if I hadn't given it this thought, I would have continued pushing down a path that would waste so much time"

So, try it out, and let me know how it goes...

In Pursuit of Efficiency

The first few years of your career you learn a huge amount. Make the best out of it, and develop an attitude to observe and learn from your own experiences. After every task, take a moment. Look back at what you did. How you did it. Feel good about it. And importantly, think how could you could have done it better and faster.

This look-back ends up becoming very critical, and helps you evolve much faster with each iteration. And act on, what you think, could have been improved upon.

Typical things you should think of:

• Can I optimise the time spent switching windows
• Could I save the build time by running multiple threads 
• Could I overlap this activity with that
• If a tool keeps taking most of my time, I should be a power-user of that tool
• I got these code review comments, what was the rationale behind points 2 and 5? Search or ask the reviewer
• Could I have done this faster if I had approached the problem differently?
• Are there any recurring patterns across the tasks that I have done, and the deductions I have drawn so far?

What is important is having that perspective, almost like zooming out of your body and looking at yourself, trying to identify where could you improve your efficiency.

Programming with Maturity

As a computer engineering student, you learn multiple languages in your college.

You understand the syntax and concepts of this language.

You work on a few assignments to understand some unique features of this language.

Most of the focus in these assignments and studies is on getting it done. Implementing quick sort an sorting an array, or  traversing a tree in pre/post/in-order.

But what about the qualities of software?

Beyond the customary paragraph on each of the software qualities, how do you learn about those? How do you acquire the right taste in software? Because at the end of the day, its not about whether you can implement it. Of course, you can, any so-called software developer can. But its about whether you can implement it with the best qualities. That is what sets your deliverable apart. Once you join an organisation this will happen through your mentors at the organisation. But what can you do while you are still a student?

One interesting way to develop this taste is to track open-source projects in your domain of interest. These are treasure troves of knowledge, from some of the most experienced folks in the world. Imagine being able to read the source code of the Linux Kernel which powers almost everything, from the OS in your phone, to large-scale servers.

And what do I mean by track an open-source project?

• In its simplest form, try to understand a small enough module from that project. And document it for the world (this also acts as a good milestone to work against). You will have to read multiple files and cross-references in the process. And when you finish one module, try a more complex one. Not only this exercise make you good at reading code, but also the best practices and the development philosophy of that project starts rubbing on you.

• As a next step, subscribe to the project's development mailing list, or their git repositories pull/merge requests. Read about the various proposals contributors make, the arguments against these proposals. You might not understand everything, but you start understanding what members of that project care about beyond functionality.

• Best yet, pick a small portion of the TODO list of that project and send out a patch for inclusion. While reading has its benefits, doing it yourself exposes you to a world of other questions, that you hadn't considered before. And the comments on your work, that's golden. 

If you are interested in System Programming, track the Linux Kernel. Or if its too large for you, track Zephyr.

Why work (hard) on a BE Project?

(Note: Although I mention BE Project below, this applies to any project that you are working on during your student days. )

Answer some questions for me:
1. What programming language are you strongest in?
C, C++, Java?

2. How complex a program have you written in this language? Any programming beyond the assignments?

Programming is what you are going to be doing for a long time in your career. How have you really prepared yourself for this? Scoring well in theory exams is in no way an indicator of how well you can program.
Lets take cricket to emphasize the point further:
a. People who score excellent in cricket theory exams cannot become cricketers. They will tremble and sweat standing on a pitch when facing a fast bowler. A situation not unlike many I have seen in IT companies.
b. Excellent cricketers have spent a long time practicing on the pitch. You are automatically comfortable, you know you are on your home pitch. You may be nervous, but clueless you are not.

Of course, I don't mean theory is not important. It is crucial. But so is programming, and unfortunately our examination systems, do not quite test you well on that. Thankfully, within the first 10 minutes, a good interviewer can identify a kaun-kitne-paani-main-hain... (This gets quite close to the point I am trying to make).

3. Do you think after 4 years of your BE, you are a mature technical professional who is hireable?
Towards the end of my third year, I always thought we have hardly done anything significant during Engineering. I mean anyone reading a few books can get here. What makes me different?

Here are some of the important benefits of a great Project:


1. Choose your co-workers, choose your work
This is something which is closer to the hearts of the students and so lets look at that first.
Rarely in life do you get a chance to choose your co-workers. This is a golden opportunity to work alongside your friends on something constructive. Many of my students often remember and miss their days of the project when they were designing for their project, brainstorming an idea or hunting for a bug for days at end.

The culmination of all these gives you a high, a sense of achievement, like nothing else. Its that feeling of accomplishment that stays with you.

Not only your co-workers, but this is also your opportunity to choose your domain of work, or pick an idea that you are passionate about. Many people that I know end up working in more or less the same domain that they did their BE project in, , even after years of passing out of BE.

2. Preparing for the future
Here are a few things that you'll be doing once you join any organisation
a. get familiar with the tools
  - revision control system
  - bug tracking system
  - development environment
  - netiquette
b. get familiar with the code
  - understand the existing architecture, code base
  - debug and fix bugs
  - design, implement and test a component


And this is going to be your profession. Surprisingly, as a student you have never had to do any of the above during your engineering. The academic assignments are too small to count towards this.

Your project acts as the sufficiently complex entity where you can get to experience all of the above first-hand. You yourself have to consider the breaking of problem into sub-problems, scheduling, risk mitigation and other strategies. These no longer stay concepts from Pressman, but you get to put them in practice. As you experience these concepts, you start developing ways and procedures of dealing with problems yourselves. This is the important learning curve. This is what contributes to the "experience".

There are many things that you can learn and absorb during the various phases of your project. We'll cover that probably in another article.

3. Foot in the door
The resumes of most BE passouts are just about the same. As I mentioned above, good marks aren't necessarily an indicator of mature developers. So how are you different from the thousands of engineers passing out with you?

Your BE project acts as your foot-in-the-door. That is what separates you from the rest. Working on your BE project also counts towards your experience in that particular domain. You now have something more concrete to talk about in the interview, and having spent time on that you are an expert in that field.



Acknowledgements: Thanks to Amey Inamdar and Amod Jaltade for their suggestions.