Category Archives: weekly-update

Backhoes, beliefs and software re-use

  • Backhoes Don’t Obey Moore’s Law: A Story of Convergence  – An article from the way back – 1996 that talks about Moore’s law and the computer industry’s dependence on telecommunication and the implication of this dependence.
  • Survival Guide for Junior Developers – Particularly, I can’t stress #2 enough. It not only helps the junior developers themselves, but others as well. When I joined my current team, there wasn’t a lot of documentation for several things. I documented everything(starting from how to set up the dev environment, documentation of my understanding on several systems and how-tos) on the wiki. As time went by, the team started reaping benefits – it was much easier and quicker to ramp-up the new people who joined our team(4 full timers, 3 of them were new grads and 4 contractors)
  • The Fallacy Of ReUse  – The title of the article was such a bait that I had to check it out. The highlight of the article for me was the distinction between ‘use’ and ‘reuse’ of software. Many times, when we say we are ‘reusing’ a piece of code, we are actually ‘using’ that piece of code – the distinction here is a little subtle.

If we were to (re)use a piece of code in only one part of our system, it would be safe to say that we would get less value than if we could (re)use it in more places. For example, we could say that for many web applications, the web framework we use provides more value than a given encryption algorithm that we may use in only a few places.

So, what characterizes the code we use in many places?

Well, it’s very generic.

Actually, the more generic a piece of code, the less likely it is that we’ll be changing something in it when fixing a bug in the system.

That’s important.

However, when looking at the kind of code we reuse, and the reasons around it, we tend to see very non-generic code – something that deals with the domain-specific behaviors of the system. Thus, the likelihood of a bug fix needing to touch that code is higher than in the generic/use-not-reuse case, often much higher.

The lessons from the article, which I agree based on my experience so far, are

– “stay away from reuse” – that is, don’t write specific/non-generic code

– “minimize use” – write generic components, but be careful and minimalistic while adding dependencies between generic components.

 

  • The Newsroom – started watching The Newsroom – I know I’m late to the party, but man what a great show! What I like most about the show is how optimistic and non-cynical it is. And, with what is going on in the world right now, I don’t think there could be a more ironic time to watch this show if you haven’t watched it already.

  • Crony beliefs -this was a super interesting post. The author’s idea is that beliefs can be viewed as ‘hired hands’ for the brain and their job is to make sure we survive. He classifies beliefs into two types:

– merit beliefs(beliefs we want to be factual and accurate so we monitor them actively) and

– crony beliefs(beliefs that are essential to for us to continue be a social animal. we don’t actively monitor these for accuracy).

The post further talks about how to identify crony beliefs and what can we do about them.

Service workers, code smells and hiring

  • Service workers – an Introduction – learnt about service workers .
    • You can think of service worker as a static web-app(js and css) that gets cached and ‘registered’ to your web-app url in the web browser. You can register a service worker for a whole domain (like www.example.com) or for a specific path (like www.example.com/example). Think of them as offline code that is stored in the browser.
    • As they are ‘registered’ to your web-app url, they can act like a network proxy for calls from/to your web-app url domain – i.e they can see the request you send to your server and the response the browser receives and can manipulate them both.
    • Because of #2, you need https to register a service worker.
    • They use javascript web workers underneath- so, they can do multi-threading stuff within your browser.
    • If you think “they are cached in the browser so I can use them to store global state” – don’t.
  • A Method I’ve Used to Eliminate Bad Tech Hires – Overall, I agree with the idea of giving the candidate a take-home test , then bringing the candidate on site to discuss the solution and paying the candidate for her/his effort. However, I highly doubt if such an approach will fly in big companies (that are comfortably set in their old ways). Additionally, this approach is also biased against people who are on H1B visa like me – we can not take on such contract work since it is a violation of our visa status. Perhaps, a gift card instead?
  • 8 ways to become a better coder – a simple and straight-forward article on eight things you can do to become a better programmer. My favorite quote was

 “The code works” isn’t where you stop; it’s where you start.

….

Part of that process is defining what “better” means. Is it valuable to make it faster? Easier to document? More reusable? More reliable? The answer varies with each application, but the process doesn’t.

It might sound obvious, but I have met many who think that’s about it. I liked how the article kept what ‘better’ means as open-ended.

  • Code smells – A good list of code smells to watch out for. Something to share and discuss with your team.

Simple problems, Ray Allens’ letter and practice

  • The most complicated simple problems  – a nice little article on how problems that appear simple on the surface can be quite complicated. A lot of these complications don’t have anything to do with the problems themselves, but with the availability of data, our ability to process available information and most importantly our inability to know whether or not we have all the data that we need. My most favorite quote from the article was:

It’s hard to know the difference between incentives and motivations. Here’s the simple difference: Mark Zuckerberg in the early days of Facebook was motivated. The Wells Fargo employees who opened fake accounts to meet their targets were incentivized. Both worked on a system of, “If I do X, I’ll be rewarded with Y.” But one wanted to earn the reward – it was like an emotional mission for a cause larger than himself – while the other only wanted (or needed) to get the reward. Without hindsight it can be hard to tell which is which.

  • Letter to my younger self – Ray Allen, a former ten-time all-star NBA player and two time champion writes a letter to his 13 year old self. This is one of the most inspiring pieces of writing that I have read in the recent times. There are too many amazing lines to quote here, however, my most favorite part comes towards the end:

When you walk in the door, the receptionist looks at you and says, “Ray? What … what are you doing here?”

“I couldn’t sleep.”

“But … you just won the title.”

“Yeah, I just wanted to get out of the house.”

“But … it’s eight in the morning. And you just won the title.”

“Well, I still got some work to be done on this tooth. Is he in?”

Your dentist walks out of his office.

“Ray? What are you … what?”

“Couldn’t sleep.”

This is what success looks like for you. You’re the kind of guy who goes to the dentist the morning after winning an NBA title.

I know, man.

I know.

But in order to achieve your dreams, you will become a different kind of person. You’ll become a bit obsessive about your routine. This will come at a heavy cost to some of your friends and family.

Most nights, you won’t go out. Your friends will ask why. You won’t drink alcohol, ever. People will look at you funny. When you get to the NBA, you won’t always play cards with the boys. Some people will assume you’re not being a good teammate. You’ll even have to put your family on the back-burner for your job.

Most of the time, you will be alone.

That won’t make you the most popular person. Some people simply won’t understand. Is the cost worth it?

Only you can answer that.

Who am I supposed to be?

Tomorrow when you get off that school bus in South Carolina, you’ll have to choose.

Every day for the rest of your life, you’ll have to choose.

Do you want to fit in, or do you want to embark on the lonely pursuit of greatness?

I write this to you today as a 41-year-old man who is retiring from the game. I write to you as a man who is completely at peace with himself.

….

Get your work in, young fella.

Most people will never really get to know the real you. But they’ll know your work.

There’s one discipline they all share as well, which appears only in varying degrees in the earlier levels: Without exception, they possess a very deep and intimate knowledge of their tools. Be it the editor, the compiler, or the framework, they know the ins and outs of its features and they navigate efficiently. They use a much wider array of features because they know exactly how the tools work.

Knowledge of tools, coupled with an extensive, tested palette of programming techniques, and the ability to remember large amounts of the code base, while relating low-level details to the whole with unconscious ease — these are the traits I see most often in great programmers. And they’re the skills I continue to aspire to in my work.

Particularly, the ability to remember a significant section of the code base while keeping track of low-level details really does separate great programmers from the rest. Funnily, the current trend of asking generic problem-solving questions doesn’t really test for this trait. We, as an industry continue to do this because that’s what everybody is doing(so I don’t really have to put thought into the process and blindly follow)  and more importantly we think we are ‘measuring performance‘ and have a ‘scale‘ and are ‘being objective‘, so this process is better. (I’m guilty of asking these problem-solving questions and making a decision when I interview people, but I have never been completely convinced or comfortable with it. This is something I’m going to re-evaluate and change myself)

 

 

The Secret rules of Modern Living: Algorithms

I watched a documentary on Netflix called “The Secret rules of Modern Living: Algorithms” produced by BBC 4. I thought it was a super fun Algorithms 101 documentary. If you want to introduce your kid to Computer Science and Algorithms, this is an excellent starting point.

The same documentary is also available on Youtube and you can watch it below here.

Why I started writing here

@patio is one of my favorite people on Twitter and he often tweets great stuff. He recently tweeted this:

This tweet made me think that I should  write about why I started writing here. I have had this DigitalOcean droplet since late 2014, but I mainly used it to host a few personal projects; I did not have any blog posts here even though I have always wanted to write. It all started with writing a private, every day journal.

I started writing a journal sometime last year. I started it because I realized I simply consumed too much information – it kind of struck me that in spite of consuming so much information – via blogs, twitter, reddit, hackernews and books I created very little. Particularly, after a couple of weeks into writing a journal, I realized it doesn’t matter how much I read – unless I write about it – give my thoughts some kind of physical form, they disappear very quickly.

The journal was first on a Google doc and I continued using it for a couple of months. Once the habit caught on, I started needing additional features that Google doc doesn’t support. For example, I started consolidating ideas under a title; then I wanted to add tags to each title; then I wanted a way to search for titles so that I can add and edit stuff; I wanted to be able to search by tags – clearly, a Google doc wasn’t sufficient anymore. So, I moved my journal to a private wordpress installation. I have been writing my journal for about 7-8 months now and it is one of the best decisions I have ever made. (One motivation for starting to write a journal is the Quiet: The Power of Introverts in a World That Can’t Stop Talking – an excellent book that I can not recommend enough)

Still, I couldn’t bring myself to start writing a public blog. Because, once I started writing my journal, I found out that my writing sucked (still sucks). So for a long time, I kept postponing the idea of writing a public blog. Things changed when I came across the blog post  by @patio.

Particularly, the above sections of the post convinced me that  excellent communication skills are far more important than technical chops to be a successful programmer; and instinctively, writing seemed like the easiest way to improve my skills since I like to read. Writing allows me to put down all my thoughts and go back and edit/revise until I’m satisfied with the outcome to publicly publish. The surprising part of this process is how my thoughts and ideas actually change during it. Writing allows me to see the trail of my ideas and thoughts, be more thoughtful and think about them deeply than before – it has helped me reduce my mindless consumption of information and spend more time creating some, however insignificant they may be.

I also see a lot of parallels between programming and writing. Both result in a very tangible product; it is easy to create a shitty product in both and extremely difficult to create a aesthetically beautiful one; both require lots of hard work and years of training to get good at. I’m hoping to get better at both of them by continuing to write here.

Setting up Apache Nutch with ElasticSearch, Naval’s podcast and new books

I made some good progress on my Apache Nutch set up. I finally got Nutch to fetch and parse walmart.com. I also managed to get Nutch store index on ElasticSearch. You would think that setting up a basic web crawler using Apache Nutch in 2016 would be an easy, a couple of hours worth of effort. Turns out it isn’t.

One of the issues I ran into while trying to set things up was specifying certain config values across a few files for Nutch and Hbase to work together correctly. You can grab these config values at https://github.com/balajiathreya/nutch-hbase-config-setup

View post on imgur.com

The above is a screenshot of my local ElasticSearch instance containing an index created by my crawler. The next step is to figure out how to get Nutch extract and parse a specific section of the web page – particularly, the item name, price and number of items available.

Finished the book – Colorless Tsukuru Tazaki that I picked a couple of weeks back. I quite liked the book even though the book ended with some loose ends not tied up. I gave the book 3/5 stars on goodreads. (A random thought that popped into my head while reading this book – I have read quite a few books by Murakami and I don’t remember any one of these books ever mention the atomic bombings even though the stories take place in Japan – not even a casual, off-hand mention. I thought it was quite weird. May be, Murakami indeed has mentioned the atomic bombing in his other books that I have not read yet.)

The Rational Optimist

Picked up two new books from my to-read list. The first one is “The Rational Optimist: How Prosperity Evolves” by Matt Ridley. This non-fiction book was a recommendation in a podcast by Tim Ferriss with Naval Ravikant, the founder of Angel-list. In this podcast, Naval shares his thoughts on life, habits and start-ups; once you get past the first 20-25 minutes, it gets really really interesting and perceptive.  My favorite moments from the podcast:

The best way to prepare for the future in 20 years is find something you love to do. Build an independent brand around it with your name. Make creative work, so that you stay interesting, you can stay ahead of the game. Anything that is not creative society can replicate and not pay you full value overtime, so it’s better always solving new problems and doing new things. Get comfortable with working in a boom/bust fashion where a couple of weeks at a time you can have a lot of work and then a couple of weeks at a time you’re on vacation.

The future will be gradual and then it will be sudden. The best way to prepare is just not to give up your independence in a first place.

 

At the end of the day, I think you have to work on your internal state until you are free of as many biases and conditioned responses as possible…. these are extremely hard skills to build; they are not things you are gonna build by reading one book and ah ha… I don’t believe in the epiphany theory of self development… you read one book, you read a phrase and thats it… this changes myself…. you scrawl on it a paper and look at it for a long time… you make it desktop background.. life doesn’t work that way… what you kinda have to do is build skills. I think happiness is a skill, dieting is a skill… skills get built over decades with feedback loop and you keep working on it.

True happiness comes out of peace. And peace comes out of fundamentally understanding yourself. It comes from looking inside yourself.

 

The act of judging something separates you from that thing. Overtime as you judge, judge, judge, you invariably judge people, you judge yourself. You separate yourself from everything and then you end up lonely. That feeling of disconnection, loneliness is what eventually leads to suffering. And then you struggle, you resist the world the way it is. Happiness is the absence of suffering. It comes from peace.

 

The most important trick to be happy is to realise that happiness is a skill that you develop and a choice that you make. You choose to be happy and then you work at it.

 

Individual entrepreneurial efforts often fail, but individual entrepreneurs over their careers rarely fail. As long as you can keep taking shots on goal and you keep getting back up eventually you’ll get through.

 

It’s only after you’re bored that you’re going to have good ideas. It’s never going to be when you’re stressed or busy or running around or rushed. Make the time. Same way with people. You need to have space in your life where you’re not booked with the people that you already know. You have to be pretty ruthless about saying no to things, about turning people down and leaving room in your life for serendipity.

This podcast became so popular that Tim and Naval met for a second time – I’m yet to listen to this one.

View post on imgur.com


The second book I picked up is Dune by Frank Herbert, a popular science fiction book. I started reading it last year, but had to return since someone had made a hold on the book and I couldn’t renew.

My fiancee visited me for the weekend and we spent sometime preparing for interviews. I couldn’t help but think that the interviewing dynamics would be quite different and interviewers would be a lot more empathetic if he/she doesn’t know the solution to the  problems already.

Other than that, the weekdays were quite uneventful and passed quite fast.

Weekly update – 3/14

This week was quite a busy week at work – the code I have been working on for the past 2-3 months went live and I was busy in preparing for the release, smoke-testing, fixing last minute bugs and making sure nothing is broken in production. The smoke-testing will go on for a couple of more days before the code is fully rolled out. It was quite an interesting few months of work and a great way to begin this year!

My contributions involved:

  1. migrating 3 business rule engine policies(related to managing seller risk) from an old platform to a new platform.
  2. updating the new platform to process these policies (adding new datapoints and code to handle what to do with the outcomes of policies)
  3. and some clever maneuvering(at least, I think so 🙂 ) in a legacy codebase so that sellers can be seamlessly made to go through the new platform instead of the old platform.

As a result, I had little time to read “The Count of Monte Cristo” during the weekdays. I’m kind of in the middle of this 1400 page book and I hate to not finish a book after picking it up. However, I made good progress with “Colorless Tsukuru Tazaki” – the other book I have been reading. I think I’ll probably finish the book sometime in the middle of the next week unless it gets too crazy at work.

Interesting reads from this week:

  1. Peter Norvig, the director of research at Google on making errors:  Link
  2. The usual H1B visa discussion you see on hackernews every couple of months – Link – the anti-H1B sentiment on hacker news is nothing new, but I can’t deny that I learn new information/perspective every time one of these threads crop up.
  3. StackOverflow’s developer survey – Link . I’ll save you click – 2015 has essentially beenAnd, referrals continue to be the number one means of getting jobs(Link) – one thing I really wish changes.
  4. Make your own bubble in ten easy steps – Link
  5. turkeypants comments on What phase are you going through on reddit – Link

I also made some progress with my side project with Apache Nutch but not too far – setting it up turned out to be more difficult than I expected. I got Nutch to fetch and parse web pages, but I’m still having issues with storing the index on elastic search. It is probably a couple of hours of work –  I need to spend some time reading the docs to figure out what I’m doing wrong. I’m hoping to work on it again next weekend.