Anil is confused and upset. He thinks life has been unfair to him. He is 28, has worked for six years in the software industry, and is livid with his former company for sacking him. Anil was the technical lead on a software development project for a large retailer in North America. He led a team of ten developers who were building an important part of the website, which has millions of shoppers using it every day.

The team faced many challenges, like changing requirements, high expectations and stiff timelines. It was unable to cope, and the customer terminated the project after six months. Senior management was concerned; a post-mortem was conducted and heads rolled, with Anil’s being the first. When Anil asked his manager the main reason for his being laid off, his manager said, “You don’t know programming well enough!”

Anil was astonished to hear this – – he had been in the software field for six years and no one had said this to him before. But his manager’s words, though harsh, were true; all the people assigned to the project, including Anil, were shallow on complex programming.

There are many Anils in the Indian software industry today. The dearth of engineers who can write good software code that will withstand a thorough code review is an open secret in the software business. An ideal software engineer is well versed with the basic concepts of programming and computer science fundamentals. He also excels at analytical thinking, logical reasoning and problem solving.

The average software engineer today, however, does not have enough experience in writing software, and has an inadequate understanding of computer science fundamentals. His problem solving skills and the analytical abilities are by and large at an average level. And he often gets deployed on a live project without enough grounding in hands-on programming.

In short, a large percentage of the software engineering workforce today essentially consists of “softicians”, persons who can tinker around with an existing piece of code or who can write code using established frameworks. When it comes to complex programming tasks, however, they’re all at sea.

The Genesis

So, how exactly did softicians come into being? And what does the future hold for them?

Till early 90s, most companies recruited software engineers from reputed colleges which had good acedamic curriculum in computer sciences and the demand supply equation was well balanced. Indian software industry revenues consisted of a limited to domestic market and a small export market. The number of companies serving this market was very small and the best companies recruited from computers and electronics streams of leading institutes and trained them well in software programming.

Smart collaboration between Industry and Government and extensive lobbying by bodies like NASSCOM created a quantum jump in the software exports revenue over the last 15-20 years. The revenues lept from a neglibile $0.3 Bn in 1992-93 to 31 Bn in 2007-08 ( a 100 fold increase in 15 years). This 100 fold increase in the last 15 years in revenues forced the industry to move from recruiting BE computer science only to BE/BTech in any streams ( like metallurgy/civil/chemical), and persuade acedamia to develop courses like postgraduation in computer sciences like MCM/MCA/MSc. Companies have taken the approach of recruiting freshers and training them intensively before deploying them.

This also helped because the demand was huge and recruiting freshers in large numbers helped the bottomline. The model has worked commercially but in the process the skill set bar has been dropped a few notches to help cope up with the demand.

This breed rapidly grew in number thanks to the creation of robust software frameworks like J2EE and .NET, which made it easy for engineers to write software despite lacking depth in basic concepts. The availability of a large number of open-source software components, and new trends like Web 2.0 mashups, has negated the need to write software from scratch all the time.

Increasingly, the web is becoming the platform for development (like Windows), and underlying products are getting “shrink wrapped.” Services like S3 and EC2 of Amazon allow one to build on the Web, and create distributed applications that can scale up easily and are extremely reliable and available.

The new trends offer businesses new ways to improve efficiency and throw up a different set of opportunities for services vendors. It has become easier to write more software in less time when one needs a particular application.

Thus, business and technology models are gravitating towards a direction where you may not require all software engineers to be great or “true” software engineers. Trends like this will only create more softicians.

So does this mean companies should take up only low-end work and make do with softicians? What happens when a higher level of programming is needed, as was the case with Anil?

Let’s look at the Indian software services industry to get some answers. The industry today is worth more than $40 billion. The export industry initially took off by offering to do low-end coding and maintenance work cheap. However, over the last 20 years, many companies, especially the big ones, have climbed up the value chain and are doing complex work as well as simple coding, testing and maintenance work.

Today, the Indian software industry has figured out a way to succeed in the global services market by creating hybrid teams that are a mix of excellent engineers (about 30%) and average engineers (around 70%). The industry today commands 85% of the global offshoring market, so the model is obviously working well.

…..Now the Global Services Industry is transforming

This model is, today, democratic in nature and many “softicians” get paid the same salary as the few “true” software engineers. The industry is set to consolidate and margins are going to shrink. As new business models makes it easier to buy or stitch software out of open-source components, instead of writing everything from scratch, the run of good salaries enjoyed by all and sundry is set to change.

Good software engineers who have solid programming skills and who can write an excellent piece of code will always be in demand. Those jobs will reduce in number and their salaries will become better. Average software engineer jobs will transform into “softician jobs” as skills get commoditised.

As in the mechanical and electrical industry, where there is a clear line dividing mechanics and electricians on one side and mechanical and electrical engineers on the other, crisp distinctions will emerge in the software industry too. The demarcation has already begun and the line will become clearer over the next 5-10 years. A caste system will emerge over a period of time where architects and system programmers will be at the top, with application developers next followed by softicians who will be able tp tailor and maiatain the code.

The interesting thing here is that this new class of softicians will not come from diploma colleges or college graduates (they will only account for a portion of it); instead, many of them will emerge from the existing pool of software engineers. Engineers who have not taken pains to understand the basics of computer science and programming and are sticking to routine tasks or very narrow technical skills are the most exposed.

The software services industry employes more than a million engineers (not including the BPO) today. The number of “softicians” would range from 30 to  50% today. As the technical and business models change over the next 5-10 years, this number might only increase.

It will not be long before we see ads that have different job descriptions for software engineer and “softicians”. It is up to each engineer himself to decide which bracket he wants to end up in.