Category: Uncategorized

The Three Constraints of Modern Software Development – Part1: Time Constraints

Software has eaten the world and your business too. We will be addressing three major constraints and challenges to software delivery for your business in our blog series, starting today with…

Time Constraints and Deadlines

Every business involved with software has a short-term plan or strategy if you may say, that requires code to be written in a particular time frame. Planning is a legitimate affair for any business, and so is putting tentative dates on them.

However, what is not legitimate is that they assume that some piece of software that they have in their planning can be delivered on the planned date without considering the state of existing software, the team capabilities and without roping in the engineering team into those planning meetings.

Looking from the other side, engineers might not like to be in those meetings, and with that mindset they are never the useful in those meetings either. Even though it is an ideal situation that engineers see the bigger picture, time and again it has been proven that they do not always do that. This might not be the right thing, but it is what it is. Engineers are excited by bits and bytes, and not by financial statements and business milestones.

From an engineer’s perspective, the deadline is unreal. From the business point of view, the deadline is important. So, what we are actually dealing here is with an unreal but important date to be met for the software piece to be delivered.

If we can fill this gap of unreal but important date, we can meet the unreal important date. This requires taking a step back and rethinking on why has this never worked well in the history of software development and how can this be addressed.

Solving the unreal but important deadline syndrome?

As a business you need software to do certain things in order to accomplish a business goal. The business goal is important and the features you are recommending are unreal. It could be unreal from a technology standpoint or time-wise. It is the business that makes things unreal. Let’s accept it. You should not be deciding on that.

The meeting you need with your engineering team is to share the business goal with them, the timeline you have in your plan and work with them to come up with the correct requirements. This is in contrast to getting your CTO or the engineering team in the business planning meeting altogether.

Example:

Business goal is to allow your application users to create a deep profile page. You want it done in 2 days. Of course 2 days is unreal. They would say it is a 2 week project. However, now that you have shared the purpose, they might be able to recommend that they could import people’s LinkedIn profile, and showcase all the details available and only put a design together and show all of that information in 3-days’ time. That creates a purposeful certainty.

What changed?

The focus shifted from pushing to meet a deadline to creatively thinking on accomplishing the purpose in the needed timeline. You are empowering your development team to help you make decisions, and are laying your trust on them. Apart from solving the business problem of delivering software on time, now you are also improving the synergy with your development team. You have roped the engineering team into the business world by discussing an engineering problem. Now that even though they do not fully comply, they have a feel for the big picture and are going to be completely aware of it. Partial problem solved!

Microservices at Idyllic

Over the last one and a half years we’ve been constantly working towards finding newer ways to build more resilient software, faster. And, we’ve found that the lines between enterprise software development and building for startups has significantly blurred. The value of iterating and releasing software quickly has been universally acknowledged and our traditional methodologies for building applications fall short in more than a few areas. At Idyllic, our experiments with Burst Mode Development (TM) brought us to Microservices and we haven’t looked back since.

Rarely has there been so much excitement about an architectural design philosophy but we’ve learnt that the hype around it is, in fact, well founded. Concepts like Event Sourcing and CQRS alone, are worth their weight in gold and have completely changed how we look at software systems. Our quest towards anti fragile software has never been stronger. Over the next few weeks we intend to blog and talk about our journey in adopting and migrating towards micro services based systems in order to enable others who have been on the fence for a while.

Last Friday, we decided to share our year long journey by creating the first micro services specific meet up group in Pune, sharing our experiences in micro services adoption and some general best practices. While we focus on this, we’ll be working towards several open source tools and platforms that enable adoption of these principles and we hope to seek community support in building these products.
micro1 micro2 micro3

 

Hack with Rack

Hello everyone, Hope you are having a good day.
Ok, some years ago, one of our clients wanted us to implement Social Network (FB,Twitter,Linkedin) sharing functionality. Knowing all the OAuth implementation that one has to deal with, we decided to use this awesome gem called OmniAuth that does all the heavy lifting tasks of OAuth Authorisation with very basic minimum implementation.
So with OmniAuth we manage to get the work done in time but then we wanted OAuth Authorisation to work with Ajax call as well. I remember going all out crazy (Googling + debugging OmniAuth code) to find a way to do it but honestly we couldn’t find any way do it. (Note: This was some years back I’m not sure about the status as of now.)
So we decide to write our own “Rack Middleware” that tampers with the Omniauth Response and treats the ajax request the way we wanted.
The Middleware

 

and then we insert the Middleware where we wanted it

 

Let look at our middleware stack
Yup that’s it- a “Hack with Rack” to get a concrete solution for our problem that isn’t too fancy but still does the task we needed getting done.

Building an OCR using PDF.js and PDFText

A couple of years back I was handpicked (not really 🙂 ) for a project that required me to work and build an application that would extract text from the PDFs uploaded into the system, using a bunch of baseline co-ordinates mapped against a sample pdf at start. The Project was named Invoice Analyser (IA) – since all the PDFs were actually invoices of our client.

I still vividly remember the day when I was made aware of the requirement on call and soon after the call ended, I knew it was going to tough but I kept faith in myself.

If I had to sum up the requirements, it would fit in 3 important points.

1. Display PDF Online.(baseline/sample pdf)
2. Map a set of co-ordinates of mark up area in a sample pdf (which is to be displayed online).
3. Extract the text from all the specified co-ordinates in any pdf uploaded henceforth (this is important because I needed those sets of co-ordinates to create a baseline sample, that once created, would be used against all similar looking PDFs to extract text out of them from relevant co-ordinates).

Now knowing all the above scenarios, I started with something that I believe everybody does (pray, no 🙂 ): googling. Luckily, I managed to knock off task 1, thanks to the amazing library called PDF.js

Those who don’t know about PDF.js well it’s an amazing Open Source library (perhaps maintain my mozilla) that focuses on a general-purpose, web standards-based platform for parsing and rendering PDFs. As far as I remember, it uses HTML5 canvas to display the PDF online (so a non HTML5 browser beware it might not work for you ).

Now, with 1 down, I concentrated my focus on tasks 2 and 3 and this was challenging. Upon my examination, I found that the co-ordinates mapped on PDF.js displayed(online) was way different from the actual pdf. Here is a link in which I asked this question on Stack Overflow (seriously after reading the first answer by @DanielLi, my confidence was deeply dented.) until, one fine day I finally got it. “That’s it. I found it. Eureka Eureka!” (was my initial reaction).

So to map the precise co-ordinates in PDF.js with respect to the actual pdf, one thing that played an important role was ASPECT RATIO [1](even though PDF render by PDF.js had different co-ordinates from the original PDF, it still maintains the same aspect ratio). So, with a little bit of jQuery and a little bit of Mathematics, I finally managed to nail task 2. Look I have also answered my own question here. (describing how I did it)

Now with 1 and 2 knocked off, task 3 was actually pretty easy. I came across 2 amazing libraries,
iText
PDFbox

that would return the specified text for the supplied co-ordinate. I went with iText (choosing iText over PDFbox was a personal choice)

End Result:

Disclosure : –
The project is almost 3 years old now. I’m aware that PDF.js has matured quite significantly now. For all those whose end result differs from mine(where I need to create baseline co-ordinates from the sample to use it to extract text from any pdf uploaded in future) you might want to look at the following example.

Also if you have a requirement similar to mine, you might just want to look at this answer as well

I stumbled upon this issue as the OP did and the answer helped me a lot.

So to map the extract co-ordinate in this case, just interchange your length and breadth based on orientation.

All in all, the project was seriously a challenge and being able to knock it off within the specified time frame is something I’m still proud of.

I have open sourced a demo version of the OCR which can give you a head start in building something similar to what I did.

GITHUB SOURCE

Dealing with disillusionment in tech.

As someone who has been in the tech industry for a while now, I’ve seen engineers getting disillusioned, overwhelmed and burnt out to a point where they couldn’t return to their original selves. Battling burnouts is still a subject that isn’t widely discussed in our field and a lot of those who’ve successfully survived it barely understand what they’ve been through. In an industry where we evangelize hackathons and sleeping under the desk, it’s hard to sustain as an individual that doesn’t associate failures in business or code as personal failures. This leads us to the general distinction between passionate programmers and the clock watchers.

The Passionate Programmer

The general notion of a passionate programmer is the guy who comes online at 1am to fix a production issue that he didn’t cause. The guy who ensures he is always up to speed with the latest libraries and tools. Someone who is willing to contribute to the project beyond the usual 8 hours. It’s also (kind of) obvious that he would be the more likely candidate for leadership roles and the one to be allocated stock options for putting his skin in the game, over the guy who prefers to check in fixes the next day.
The 9 to 5 engineer is the one who is in there for his day job. He is uncool and generally overlooked. The irony is, he is the one who is going to be more productive in the long run.

The passionate programmers usually succumb first.

There usually comes a point where it becomes impossible to keep a steady learning curve. This usually results in the plateauing of the graph in terms of the new things you learn, to the conventional stuff that needs to get done. Now, there is a gradual but growing resentment towards the job that holds him back from keeping up with all those things he could’ve learnt.

Let us assume that all the time needed to keep up with his learning was made available to him by the company; there still remains a sense of resentment as there is no way he can sustain that pace. Though his rational mind would reject the causation for the situation, it’s definitely much easier to place the blame on an entity that’s easy to dislike.

People who have a life outside work deal better with these situations.

In one of the 37signals’ blog posts(https://signalvnoise.com/posts/996-hire-family-people), DHH mentions how working with people with families seemed to be a better choice as they had to be somewhere, with someone more important at the end of the day. There were no honor badges available for killing it 12 hours a day at the office. And this is one of the most important distinctions I’ve learnt as a source to overcome the disillusionment associated with tech – having a life outside work which adds a sense of fulfillment or accomplishment could be invaluable.
Even mundane challenges like forcing yourself to a regular exercise regime or a book a week could go a long way in solving a lot of the problems associated with this.

Startups thrive off this

Most startups proudly describe how they’ve worked against Maslow’s hierarchy of needs to make the world a better place. My take on it is that it’s unsustainable and hence will eventually fall apart. Self actualization is a higher goal and at times may force you into thinking differently but eventually it does adhere to the laws. I’ve also come to realize that this is part of the reason why some of the best engineers that I’ve admired, teach.
The rewards of watching someone you’ve mentored succeed far outweigh the efforts that go into it. It inculcates several valuable qualities like patience, humility and empathy.

Why didn’t this pay off?

The worst reasons for disillusionment are failed projects, disgruntled co-workers and unhappy clients. This brings in a sense of failure that becomes hard to disassociate from. It’s usually never about the code, though it eventually transcends to it. Usually the problem lies in communication, conveying the vision, the goals and creating a sense of belonging. In one of my earlier posts, I’ve tried to highlight the importance of having the right project/product managers in the team. The value of having the right project managers (or leadership team in this case) is to ensure you embrace your team right from the beginning. Extroverts usually shine in such roles as they easily embrace their team forcing everyone to participate. Embracing your team right from the beginning helps deal with failures as a team and individuals aren’t singled out when things go bad. This is a more cultural problem and trickles down from the leadership team. For a quick low down on how to manage engineering teams here are some great reads by Greg Brockman of Stripe https://blog.gregbrockman.com/figuring-out-the-cto-role-at-stripe and Slava Akhmechet from RethinkDb. Every single post on his blog are words to live by if you work in tech (http://www.defmacro.org/).

Introverts have an uphill battle

Introverts have a constant battle in figuring out this balance. As a textbook introvert, it’s a problem I constantly have and one I am yet to find great solutions to. One of the problems I’ve frequently battled with, is coding in silos and finding ways to work alone. While the productivity boosts are hard to deny, it introduces a level of monotony and stagnation that’s hard to shake off immediately. I would love to hear about how you deal with these situations as an introvert.

Not being a slave to routine

Working from different places and meeting new people every now and then could introduce a fresh perspective to your problems. Speaking with people who have dealt with similar problems or even people with a fresh perspective towards approaching your work problem could infuse newfound energy in your approach towards it.

Attending conferences

Conferences are (IMO) less about the tech and more about meeting people who tackle the challenges you face in entirely different ways. And generally, I find a new sense of purpose and energy after every conference I attend.

Subscribe To Our Blog

Get access to proven marketing ideas, latest trends and best practices.

Next up home

Contact

Lets build cool stuff

Share your contact information & we will get in touch!

I want (Tell us more about your dream project)