9 Mar 2013

The Swamp of Low-end Engineering

Ravi Iyer (disclosure: my professor and mentor at college) has an interesting article comparing engineering and scientific mindset in the context of software development. (I had an email discussion with him, where he corrected some of my mistakes, but we still disagreed at the end, so I'd like to put forward a different view here.)

His main point is that engineers operate under a need-to-know mindset, learning only as much as needed to get their present job done, and not needing or wanting to know how things work under the hood. They would use a vector class knowing only its interface, and not understanding how it actually works. On the other hand, scientists, in Ravi's telling, want to understand how everything works, what they are built out of, how those components work, etc. Ravi does qualify his comment about engineers by saying that it doesn't work for developing core building blocks like OSs, system libraries and programming languages — it works only for using existing components to build applications.

Unfortunately, this characterization still paints all engineers with too broad a brush — talking of an "engineering mindset" is misleading because high-end engineers perhaps have more in common with scientists than with their low-end brethren. So, a three-way categorization is more useful:

  • low-end engineers
  • high-end engineers
  • scientists
High-end engineers typically work for the top tech companies in the world like Google (disclosure: my employer), Apple and Amazon, or startups that are equally competent, whereas low-end engineers typically work for out-scourcing companies like TCS or Infosys, where cost is the primary factor.

(The classification of engineers into high- and low- end may be uncomfortable for some people. My intent is to have an honest discussion, which requires calling a spade a spade, not to insult.)

These people are doing software only that's the best paying job they could get, not because they have a passion for it. If civil engineering, or movie-making, or marketing paid more and they could get a job, they'd be doing that instead. Low-end engineers are not the ones you'd find reading a paper on distributed systems, on the weekend, on their own time, for the love of it. Whereas both high-end engineers and scientists might, which is another way high-engineers are closer to scientists than they are to low-end engineers.

Another factor is scale — high-end engineers can build systems used by hundreds of millions or even a billion people. A related factor is technical complexity — are you building a system that can serve a petabyte of traffic at peak, or are you building a web site consisting of some Perl scripts running on an Apache server for Canara Bank? High-end engineers earn more, multiple times the salary of low-end engineers, in some cases, as a consequence of their greater abilities.

Both the high-end engineer and the scientist want a complete bottom-up understanding of the software stack, but their motivations are different — ultimately, the high-end engineer wants to learn so that he can build a better system, while the scientist may build a system so that he can learn more.

Ravi seems to mis-characterize this by saying:
Component-Based-Development delivers solutions quickly and at far lower cost than reinventing-the-wheel solutions
That's true by itself, of course, but almost a tautology — reinventing the wheel is by definition understood to be bad. More importantly, this statement is misleading — someone who wants a complete bottom-up understanding of the software stack isn't doing so because they want to reinvent the wheel. Just the opposite — if you know how things work at each layer, what is done at each layer, and why, how else we may build our software stack, and what the pros and cons of that alternate approach may be — if you understand all this, you are less likely to reinvent the wheel, because you'd know that you could reuse the existing component and that it would work for you. Similarly, a bottom-up understanding means that you are also less likely to blindly reuse a component when it wouldn't work for your use case, when the situation genuinely calls for a re-implementation. A person who doesn't understand why he's doing what he's doing is obviously going to produce better work than a person who doesn't.

More concretely, going with Ravi's example of understanding the vector's interface and not how it works, if all you know about the vector is its interface, if you think, "oh, I can invoke vector.add(4, object) and it will add the object at the fourth position [1]", and if you don't know about asymptotic complexity, you could insert things at the beginning of the vector when you could just as well insert them at the end. This wouldn't work when the vector got really large, and your program would then grind to a halt. That doesn't happen for every program, or for every vector in a program, of course, but the point is that someone who doesn't understand how the vector works is liable to building a broken program in his ignorance, like a civil engineer who builds a mall that collapses when it's packed, during the peak shopping season.

Time frame is another difference. Low-end engineers usually works at a shorter time-frame, and high-end engineering may work at either a shorter or a medium time frame, like a few years for Google (or, I assume, Apple). Science can go on for a long time frame, like a decade or longer. For example, the LHC took 14 years to go from proposal to when it was used for the first time. As another example, Knuth has been writing his books since 1962, and still hasn't finished.

Engineering, whether low- or high-end, is ultimately driven by solving people's problems, while science is driven by advancing the world's knowledge. So high-end engineering attracts people who get a kick of solving real-world problems, but who would otherwise have been scientists.

Both engineers and scientists can work in one of two directions: they can work backwards from the system that needs to be built, or forwards from the theory. Both directions are valid and have their place, and both high-end engineers and scientists work in both directions, but I think scientists work forward from theory more often than do engineers.

Science does run the risk of intellectual masturbation, doing theory for the sake of theory. I'm not talking about fundamental research like the properties of matter, or doing research with plausible benefits, but more about work like the kind one of my professors at IIT Bombay did. He'd say that if you have a struct:

struct Something {
    int a;
    char b;
};

and if you represent the set of int values by A, and the set of char values by B, then the set of values of this struct is A X B. Which is fine, but how does it help me write better programs? I remember arguing with him about this, saying that I want to do something useful, whether it's formal or not, but he seemed to want things that are formal, whether they are useful or not.

Paul Graham writes insightfully about this:
[...] Instead of doing what they really want to do, which is to design beautiful software, hackers in universities and research labs feel they ought to be writing research papers. 
In the best case, the papers are just a formality. Hackers write cool software, and then write a paper about it, and the paper becomes a proxy for the achievement represented by the software. But often this mismatch causes problems. It's easy to drift away from building beautiful things toward building ugly things that make more suitable subjects for research papers.
Unfortunately, beautiful things don't always make the best subjects for papers. Number one, research must be original-- and as anyone who has written a PhD dissertation knows, the way to be sure that you're exploring virgin territory is to to stake out a piece of ground that no one wants. Number two, research must be substantial-- and awkward systems yield meatier papers, because you can write about the obstacles you have to overcome in order to get things done. Nothing yields meaty problems like starting with the wrong assumptions. Most of AI is an example of this rule; if you assume that knowledge can be represented as a list of predicate logic expressions whose arguments represent abstract concepts, you'll have a lot of papers to write about how to make this work. As Ricky Ricardo used to say, "Lucy, you got a lot of explaining to do." 
The way to create something beautiful is often to make subtle tweaks to something that already exists, or to combine existing ideas in a slightly new way. This kind of work is hard to convey in a research paper.

Engineers have their value measured by the market — you won't get paid if you don't solve real problems. And you get paid more if you solve important problems. The market doesn't pay for intellectual masturbation. Of course, no one's arguing that the market is perfect, and indeed a lot of technological advancement has come from science, so we need both the market-driven engineering approach and the scientific one.

The real problem with low-end engineering is that it's often crappy — it does not really satisfy users. I remember the Indian Railways web site charging me money without issuing a ticket, and a dozen emails to their customer service department yielding no solution, beyond a "Please note that this transaction was cancelled." No, you doofuses, the transaction went through half-way — you charged me money.

Another fuck-up is the Airtel web site, where nothing works as you expect it to, where you can't find the thing you want, and even if you are able to find and change it, it doesn't reflect in the rest of their systems, forcing you to spend tons of time dealing with their customer-service incompetents. Airtel still emails me at my older email ID, more than a month after asking them to update their records, both on the web site and on phone, multiple times.

Ravi writes:
[...] The real world is driven by how society's needs are satisfied quickly and at acceptable cost.
Fair point, but in many cases, frustrating the user doesn't save money. If anything, as in the example above, it increases costs both for the user and the provider (who has to deal with support calls and negative reputation). A lot of things in the world suck not because of any intrinsic economic necessity, but for lack of imagination and empathy and a desire to make things better, even when such improvements have little or no cost.

These systems are aggravating and generate feelings of frustration, alienation and, at worst, rage, as opposed to the feeling of joy and satisfaction and competence people feel when they are able to accomplish a task. People use Airtel only because there may not be a better ISP. And that's a bad place to be in, both business wise and from a creative point of view. I used to happily tell friends what great service I used to receive from Airtel, but I no longer have any loyalty left for them. And I'm not the only one. Airtel leaves itself open to attack from any competitor who meets the low bar of merely not aggravating their paying customers.

Business aside, as a creative person, as a person who wants and hopes to make a positive contribution to the world over the course of his career, you're a failure if all you can do is build systems that at worst aggravate people (as with Airtel) or are often merely unpleasant to use (Indian Railways, SBI, Canara Bank...).

Low-end engineering is where ambition and creativity go to die, and where discontentment lives. In that sense, low-end software engineering is no different from the low-end of other industries. Taking transportation as an example, the low-end consists of traveling in a crushed bus, surrounded by other people and their sweat, for 10 rupees, while the high-end may be a spacious, air-conditioned chauffer-driven car. Why would expect the software industry to be any different, for a lot of low-end software engineering to not suck?

[1] Fifth, since it's zero-indexed.

11 comments:

  1. @Kartik. Thanks for sharing your views on my article on your blog. Here is my response in multiple parts due to comment size limitations.

    BTW I (Ravi S. Iyer) am not a Professor :). I am a software consultant who taught Lab. courses and acted as a technical consultant for project work of students in a Maths & Computer Science department of a deemed university, mainly as a "visiting faculty"/"honorary faculty".

    I used the term "practical solution mindset" and did not imply that all software engineers would have a similar mindset. In the article I further narrowed down this term to refer to "component-based-development software solution approach".

    I also avoided referring to scientists and instead used the term "scientific mindset". To clarify, some software developers/engineers may have a purely "scientific mindset" as I defined the term in this limited context, some may have a "practical solution mindset" and some may have a mix of both.

    I don't think the typical software developer/engineer can accurately be referred to as a scientist as I believe that term to be understood commonly. The scientist does research and publishes scientific papers on his/her research. The software developer/engineer builds software and typically does not publish research papers on the software that he/she has built. [I also largely agree with the Paul Graham quotes/extracts you put up in your article.]

    But the approach the software developer/engineer uses to build software may involve a "scientific mindset".

    Instead of using terms like low-end and high-end to differentiate the categories of software developers/engineers I prefer to use the term systems software developers or infrastructure software developers (to use Prof. Stroustrup's term as described in "Software Development for Infrastructure", IEEE Computer, Jan. 2012, though Prof. Stroustrup's defintion is more sweeping than what I limit it to) for the category that deals with low-level software work like Operating Systems development, database development (as against database applications), system library components etc. The other category of software developers can be referred to as Applications developers (Prof. Stroustrup uses that term too) or high-level software developers. The latter category developers may typically combine applications development skills/knowledge with domain knowledge of the area related to the application (e.g. Medicine, Law, Enterprise Resource Planning, Shipping Port Logistics etc.)

    ReplyDelete
  2. Regarding the point about knowing only the vector's interface what I had written was "He simply needs to know the programming interface and the pros & cons of the vector class". The pros & cons part is very important and a well documented vector class should have guidance information to its class users on how best to use it and the efficiency and other problems that may crop up when used in some particular ways. Expecting a vector class user to know such matters by knowing/studying its implementation details and not through good documentation, IMHO, is an example of a poor vector class/component (a class/component is its code/implementation as well as its documentation).

    Regarding application software that does not function properly at times like the experience Kartik had with Indian Railways website, it is very unfortunate that the quality of a lot of application software out there is not very good. Small and not-so-small bugs are commonplace. Here, I think it is the software industry culture that is to blame. In the nearly three decades that I have seen and experienced it, the software industry worldwide seems to operate as a Without Warranty Wild West industry, with current generation of released software seeming to have far more bugs/problems than software on Mainframes/Mini-computers two to three decades ago. Forget application software bugs, the very operating system on which the whole software stack runs is without warranty and has a host of bugs including some terrifying security weaknesses. If you get hit by a malicious virus then it is your bad luck, that's it. You cannot hold the software company accountable. It is this lack of accountability of the software industry that, IMHO, is at the root of the excessive software failures that users have to suffer from.

    Once the software industry becomes like other mature engineering industries who are accountable for their products (e.g. a Television set manufacturer is accountable for its product), then, IMHO, the entire software ecosystem including the teaching/academics part of it will be forced to become more "professional" about software development/engineering. But will most software companies, or perhaps any software company, offer warranty for their software easily and of their own accord? I strongly doubt that will happen without outside or public pressure/intervention.

    Meanwhile we have to continue to live with the Without Warranty Wild West software world and produce software applications, within time & money constraints, of reasonable quality with some bugs here and there being tolerated like some security weaknesses here and there in the OS itself are tolerated. Is this the ideal software world? Certainly not! But can we freeze application software development till the software world becomes ideal? I don't think so! Very, very few people are even thinking and writing and talking about these fundamental issues with software development (e.g. Prof. David Parnas, Prof. Bjarne Stroustrup). And most people in the software industry or software academy don't seem to bother about or even know what they are saying!

    So, in this far-from-perfect software world, for application software, component based development software solution approach which uses a "need to know" mindset, thrives. You can't stop it. As an individual you can't change it. If you want to provide acceptable application software solutions to people who are willing to tolerate some bugs, you have to go with it. Or else you can simply stay away from providing such solutions. Others will provide them in your place. I could be wrong but that's the way I think it is.

    ReplyDelete
  3. Thanks for your comments, Ravi, and for taking the time to discuss this with me.

    I know that you are not formally a professor, but I learnt more from you in the five years at Sathya Sai University than I did from any of the supposed professors, so it seems to not call you a professor.

    Sorry for misunderstanding your point about the vector. I agree that a well-written vector class would tell its users about the efficiency of inserting things at the beginning vs at the end. But my general point was that the quality of work done by someone who understands at least roughly surpasses the quality of work done by someone who doesn't. Whether or not that high quality is required for a particular project is a different point.

    I also now understand your point about mindset, and that an engineer may have a scientific or an engineering mindset. Sorry for not understanding that earlier. Still, I find it simpler to not have this distinction between what one is doing (like one is an engineer or one is a scientist) and mindset. Instead, I find it simpler to just divide people as low-end engineers, high-end engineers and scientists. Besides, my post talks of several ways in which high-end engineers are closer to scientists. I agree with you that the typical software developer cannot be referred to as a scientist. I never said otherwise. When I say scientist, I refer to people like Parnas who push knowledge forward.

    I will stick to my terminology of low- and high- end engineers. Your suggested replacement of system- and application-level engineers is totally inaccurate to me and confuses rather than clarifies the issue. It's not about whether you're building an OS or applications, it's how good you are at it, both as an individual and as an organization. Simplenote, Google Docs and Android are all high-end engineering work. Two of those (Simplenote and Google Docs) are applications, while the third (Android) is an OS. But they are all high-end engineering work. Whereas the Airtel website is just an application, but is low-end engineering. So, I repeat: it's not about whether you're building an app or a system-level thing, but about whether your work is great or sucks.

    ReplyDelete
  4. Regarding your point about warranty, I find the greatest frustration with web sites for a real-world service like the bank or railways. These are the ones that have captive users. Most people wouldn't use a particularly bad social network or blog service, since the barrier to switching is low. In cases like the bank web site, warranty does not apply, because the web site is only a part of a larger service. Besides, it would be hard to write a warranty that covers bugs, confusing UI (some of these guys are masters at this), etc. In any case, I find rarely come across software that totally doesn't work, so the warranty would do little good. Finally, software is often free or distributed for a nominal fee (like all the 99 cent mobile apps), so it doesn't seem appropriate to impose warranties, and run the risk of killing the golden goose.

    Things like OS bugs and security holes are scary, but it's a much smaller problem with modern operating systems like Android and iOS. Some people believe that we will switch to these OSs in a few years, and these security features (like sandboxing) are spreading to desktop OSs — the latest versions of both Mac OS X and Windows have these facilities in place, so a random app you install from the app store cannot steal all your data, let alone delete it. Chromebooks don't even run apps in the traditional sense.

    In other words, the market is already fixing the problem, so a legal approach is not needed, and risks unintended side effects in a rapidly evolving industry. Besides, to impose a warranty, one must have a clear definition of what's expected from the product. That's straightforward for a washing machine, but not for as complex a device as a computer. If you get a warning about running an infected program, and you click Yes and run it, whose fault is it? The traditional PC approach says that it's the user's, but Apple takes a different view, not even giving users the choice. Is the Apple app store approach right? But then you lose your freedom and Apple decides what you can or can't run. The point I am trying to make is that when isn't a broad consensus on what the right path is, one can't legislate warranty requirements.

    In any case, warranty is no substitute for people who don't care about building things that work. Basically a crappy workman is going to produce crappy results. There's no substitute for individuals caring about the quality of their work. My driver, for example, drove over someone's foot last year, and then stared angrily at the pedestrian as if to ask, "why the hell were you in my way?" Whether you are a driver or a software engineer, if you don't give a shit about the quality of the work you do, it's hard to get you to do good work.

    We don't see this in most fields because one would have to be really dumb to make a washing machine that doesn't work. But in a newer field like software, without well-established standards, a clueless person can do a lot of damage to users.

    Your point that standards were higher in the minicomputer and mainframe worlds is interesting.

    Of course, no one's arguing that we freeze software development till we reach an ideal state. And, of course, I can't change the quality of software.

    ReplyDelete
  5. Regarding your point that few very people like Prof.Parnas are working on addressing fundamental issues with software development, I am not setting such high standards when I dismiss low-end software as crappy. All I ask is that I can figure out how to login to the Citibank web site. Yes, they screwed this up, as hard as it is to imagine. Or that the railways web site either issues me a ticket or if it can't do so, at least does not charge me. When even these low expectations are not satisfied, that's what frustrates me and makes me rail against the low-end engineers. Forget Parnas or Stroustrup and their work in advancing the state of the art. At least meet my low expectations. BTW, out of curiosity, is there some accessible work from Parnas or Stroustrup that I can apply in my day to day work? Is it the gap between industry and academia that's to account for the lack of adoption of their ideas? Is it that they just theorize and don't put in enough effort into packaging their work in a form that can be readily adopted?

    ReplyDelete
  6. A colleague of mine wrote this delightful mail a while ago. I think this is another reason why low-end engineers in India do a crappy job.

    They cannot identify with great work, because in India, given that we are still a developing country, a lot of things are not as good as in the west. Take transport. Imagine traveling in a crushingly full and dog slow bus, in the 45 degree heat with no air-conditioning vs taking a metro in London or barreling down a freeway. The roads are pot-holed, pavements are uneven or non-existent, the power isn't always there, the auto guys rip you off... My guess in that in such an environment, it's hard to imagine great work. And if you can't imagine it, you can't do great work. Going abroad to developed countries was an eye-opener for me, seeing everything so clean and orderly and comfortable and running well and... perfect (of course it's not, but it seems a bit like that when you go abroad the first few times).

    Another factor may be our cultural factor -- we blindly do what we are told, without asking why. Take the security at the Google office building, RMZ Infinity. They check the underside of cars for bombs, and the dicky, but not the interior. People sometimes come with big suitcases, when they are going abroad or have just returned. One can easily hide a bomb there and kill a couple dozen people.

    I have seen one of the security guards scolded by his supervisor -- you must keep standing, or I will slap you. People are treated like animals, and they naturally behave in a mindless manner. The supervisor tells me to stand, so I should stand. Not: why am I doing this, how else can I do my work, would that be better or worse, etc.

    But when I see the security people in London or the US, they carry themselves so well. They don't behave like servants. They are fewer in number, far more effective, they have a brain, they interact with us at a higher level than the Indian security guards. The US ones are our equals, the Indian ones one level below, it seems, by their behavior.

    So, long story short, I think that Indians culturally are told to shut up and follow instructions.

    BEGIN QUOTE:

    I have fond memories of eating veggie (max) Subway sandwiches in the US. Back in India, I've rarely been satisfied with the Subway sandwiches the franchisees make. They either overheat the patties or scrimp on the black olives. You can give specific guidance on these but still they don't get it right. Six years and not a great Subway sandwich yet...

    I've wondered if this was because many of the Subway employees cannot actually identify with the taste of a well-made sandwich. They are following a process someone taught them but don't have a sense for what a fine sandwich is - they probably don't appreciate it and perhaps can't even identify with someone who appreciates it. (Most of them do have a dour face as they make and serve the sandwiches. Subway franchisees in the US are also usually immigrants - greek and so on - and they seem to do a better job at both aspects of "service with a smile".)

    This suboptimal output is probably true of many service providers in India. Unless they've seen and experienced in first person fine quality in a service or a product, they don't even think they or anyone else can do better. (The quality of service and decor at malls and shops and movies and other commercial places improved dramatically in India after people got exposed to US/UK/Singapore etc. in the last 10 years.)

    Your *ability* for perfection/quality is capped by your *conception* of perfection or super-high quality or delightful product experience. If you've worked with carpenters or masons or builders in India, and especially if you've had to argue with them to get the small details done the right way, done your way, you know what I am talking about.

    END QUOTE

    ReplyDelete
  7. Sorry for the delay in response, Kartik. But ... better late than never :).

    Regarding quality issues with the two categories of software that we are discussing, [with different terminologies :)] : Yes, of course, you can have poor quality application software like a poor mobile company web app. And you can have poor quality infrastructure software like a poor quality operating system as well. Both categories of software, IMHO, can and do suffer from poor quality issues today.

    I continue to hold the view that, at least for paid/significant cost software, providing a warrranty like other fields of manufacturing, will bring order to the chaos that pervades the software industry today. Whether it can be pushed through, is practical enough, is a different matter. But if you ask the hapless user s/he does not care about whether it is practical enough or not, s/he is fed up of paying significant amount of money for software and having to live with painful bugs in what s/he has paid for!

    About market already fixing the problem: Well, I don't know how many people will buy that. I don't want to get into the techincal stuff but simply go by the sordid history of the software industry in this regard. If changes are afoot in the software industry so that these bugs and security issues in infrastructure software like the operating system become history, wonderful! But I will accept that such a change has happened only when I see it proved in the marketplace in significant quantity/at large. Till such time I will hold on to the view that warranty should be provided for software. If bugs hit the customer, the warranty should spell out how the customer will be compensated based on the impact of the bugs.

    About poor workmen being a problem, I agree. But till the poor quality of output produced by poor workmen is penalised the market/industry will employ such workmen and make money at the expense of hapless software customers. Prof. Parnas and others seem to hold the view that introducing licensure for software engineers will control the poor workmen problem like in other engineering sectors in the Western world. Maybe it will. But despite tremendous efforts of Prof. Parnas and others they have not succeeded so far in the Western world adopting licensure for software engineering in a big way.

    ReplyDelete
  8. About work of Parnas & Stroustrup: I have read only a limited part of their writings, and my interest was limited to improving the practice of software development in academia. In that limited scope of my readings of their works they have been trying to promote models in academics which will lead to more knowledgeable and better skilled software developers/engineers being produced by academia (Computer Science and Software Engineering graduates). I cannot, off hand, remember from this limited scope of my reading of their works of stuff that may be of immediate practical use to a software development practitioner like you.

    About quality and attitude aspects of India vis-a-vis the West: Well, some centuries ago, before the West embarked on ruthless colonization using its better weapons and strategies of war, some parts of Asia including India seemed to have far better quality products than the West. Attitude in terms of quality consciousness also may have been superior to that of the West. Colonization for around a century in India (and maybe similar period in other parts of Asia), and rapid strides made by science and technology in the West, combined to result in Asia biting the dust. We essentially became servants of the colonial masters, who looted the wealth of the colonies and used them as cheap labour and captive markets for their flourishing industries. Now Asia is re-emerging. Maybe one or two generations in the future will show a markedly different attitude in India, even at more menial kind of jobs like private security guards, drivers and sandwich makers. Already there is substantial change at 'higher' levels. You can see it in the cricket team. They don't get pushed around like they used to, two decades ago. You can see it in our industry leaders. They now buy and run companies in the West. Unheard of and perhaps even undreamt of, two or three decades ago.

    ReplyDelete
  9. No problem about the delay in responding. Please do so at your convenience and only when your schedule allows, and not feel obligated to be quick.

    Regarding warranty, the practical matters are exactly where many ideas fail, don't they? I agree with the point of view that users are fed up, but we have seen a lot of laws that don't take the practical matters into account and make things worse. Our IT Act of 2008 provides for up to three years in jail for sending a message that's "annoying". The US SOPA and PIPA were criticized as breaking the internet. Will service providers be liable for bugs? What kind of bugs? What if their server goes down for a few hours in a year? Will you be able to sue them? What if it's confusing to use? Who defines "confusing"? Saying "I want a law" without giving details makes no sense. If you are proposing a law, the onus is on you to define it at least somewhat, and illustrate what it would and wouldn't cover, without resorting to hand-waving.

    > But till the poor quality of output produced by poor workmen is penalised the market/industry will employ such workmen and make money at the expense of hapless software customers.

    Agreed.

    Regarding mandatory licensing of software practitioners, one needs to be careful to do it in a way that does not require onerous formal training. As you know, many of the visionaries didn't have a college degree, or didn't have one in CS. What's appropriate for a doctor may not be appropriate for a software engineer. I'm not opposed to it, and as you remember, I supported the idea in one of our email exchanges, but I am just saying we need to be careful to avoid unintended consequences, as with your warranty idea.

    Besides, there's the risk that a formal certification requirement will become bureaucratic and be co-opted by people interested in other motives than users' benefit. For example, I was told how the body that certifies doctors in the US keeps a strict limit on the number of people applying for a medicine degree, so as to ensure an artificial scarcity and thus high demand. I was shocked to hear that a visit to a specialist in the US can cost $300, as compared to Rs. 300-500 for the specialists at a good hospital in Bangalore.

    Let's hope India becomes a more egalitarian society in the future.

    As for imperialists stealing our resources, yes, that's true, but that's supposed to have played in a minor role in the standard of living of our country today, 6 decades after independence. I don't want to argue about this point with you, but I read a fascinating book called The Birth of Plenty, which is an analysis of 2 millenia of economic history, all the way to the Roman empire, and an inquiry into the causes of prosperity. This book argues that the primary factor for a country's prosperity is its own policies: the rule of law, the strength of capital markets, intellectual freedom, transport and communication, etc. In other words, maybe you can blame the British for our economic weakness in 1947, but not for the situation in 2013. Why did countries like Singapore, Hong Kong and China grow so quickly? And if the invasion and stealing of resources theory is correct, countries that were never invaded, like Mongolia (IIRC) should be rich, which is not the case.

    ReplyDelete
  10. Thanks for your understanding about delay in my response.

    Regarding warranty and practicality of it: I don't want to get into a detailed discussion on the practicality of warranty for software as I have not examined the matter in depth (and neither have the time to do so now). But then let us at least acknowledge that the software industry folks (including me in the past and perhaps in the future as an Open Source developer) are amateurs as compared to professionals from fields like civil engineering or mechanical engineering. Yes, most of us are/were highly paid for our software development work but we are/were highly paid amateurs. We don't have the right to call ourselves "professionals" as we cannot give any warranty/guarantee about the software solution we provide, in most, if not all, cases.

    Please note that warranty does not mean error-free. A Television set may develop faults within its warranty period. Warranty, in my understanding, implies accountability to the extent of replacing a faulty product with a working product at no extra charge to the customer and also compensating a customer for significant damages incurred due to a faulty product.

    Regarding licensure: I am not an expert on it. I just mentioned efforts of Prof. Parnas and others.

    Regarding imperialists. You wrote that you don't want to argue on it. We both have expressed our views. So let us leave it there.

    ReplyDelete
  11. I disagree with the "we are all amateurs" view. That's true of low-end engineers and the products they build, not high-end engineers or their products, like Mac OS X, iOS, Android, Gmail, Chrome, Facebook, Simplenote, etc. These products work, work great and reliably, with delightful UIs.

    I understand that warranty means accountability and not that the product is error-free. But that doesn't answer my questions -- will service providers be liable if their web site goes down for a few hours in an year, but you happen to need it exactly at that time? Will they be liable if their products are confusing? Who defines "confusing"?

    It's fine that you don't want to discuss this. But in that case, you cannot find fault if your proposals are dismissed as superficial or fanciful :)

    BTW, no warranty AFAIK includes consequential damage like you are saying. If your phone doesn't work, they will give you a new phone, but if you lost something because of the phone not working, they won't compensate you for that.

    ReplyDelete