Who moved my comfortable IT job (who moved my cheese)?

One of the best books on change is Who Moved My Cheese?  The book chronicles what happens to those that anticipate and welcome change and those that refuse to change.   The software industry is changing rapidly and I am sure that many are wondering who moved my cheese?  Who moved my comfortable IT job.   It is no longer about the technology it is about the business and customer.

The first wave of data processing is coming to an end which was internal development and B2B.  The second wave. B2C,  is underway and we are just at the start of this wave .  Business to consumer covers self service software applications deployed to the actual customer.

It use to be interviews with internal customers would suffice the requirements process.  Today, knowledge of the actual customer and the business is critical to the success of the requirements process and the success of a software application.  The big change is trying to understand the business and customer as much or more than the technology.

Read more at Reboot! Rethinking and Restarting Software Development.

Published in: on August 18, 2009 at 09:53  Comments (1)  

The Heart of The Matter

I am asked all the time, how can I sell a metrics program in my organization.  The key word here is sell.  Any good salesperson will tell you the key to sales is benefits.

1. There is limited time management has to focus their attention.

  • Management knows when to stay the course or change direction for projects.
  • Control charts can be built with calculated upper and lower control limits.
  • When any indicator is goes beyond a control limit corrective action can be taken.
  • If the project indicators are within control limits, then management can focus on other projects needing their attention.

2. Clients will experience higher levels of satisfaction because communication improves.

  • This will allow clients to have a better sense a project is on time.
  • Clients will be notified of slippage in schedules sooner than later.  They will be able to notify those they have made commitments to.
  • IT staff will be the recipient of better planning and project coordination.
  • Management will make commitments based upon past historical performance not on best guessing.
  • This prevents IT staff from trying to achieve and meet unreasonable deadlines.

3. Metrics allows IT staff members to become empowered.

  • The likelihood of a lot of overtime at the end of a project decreases, so IT staff will not be required to work a lot of overtime.
  • If an IT staff member (or team) is not performing up to standards, then corrective action can be taken
  • Training
  • Reassignment

Just Google it! Huh?

There is a misconception it is okay to have documentation scattered about individual computers and a “shared drive.”  The myth is documentation can be found via searches and it does not need to be organized.  Some organizations want to implement a Google style search mechanism.  The beauty of the Google search engine is it seems so simple.  Google relies on creators of websites linking information together and some relatively complex algorithms.  The founders of Google wrote a paper entitled, The Anatomy of a Large Scale Hypertextual Web Search Engine outlining how the Google search engine would work.   It is not as simple as it looks.  Most organizations with disorganized documentation cannot get their developers to use consistent terminology nor can they get them to embed hyperlinks in a document.

If a specific document is referenced by many other documents, then the document is significant.  If the document is not be referenced via some linking mechanism, then there is no way to determine its significance. The problem, of course, is getting individuals to link documents together.   Believe it or not I have had developers argue this point.   Even though they have never read the short paper or long paper written by Brin and Page the founders of Google they just somehow know all the inter-workings of Google.  They think it would be easier to design a complex search engine with artificial intelligence not yet invented by mankind instead of standardizing terminology and using some sort of naming convention for their documents.

Read more at Reboot! Rethinking and Restarting Software Development.


What gets rewarded gets repeated!

A behavior followed by a reinforcing stimulus results in an increased probability of that behavior occurring in the future. – B.F. Skinner, Psychologist

The idea of “what gets measured gets done” is only partially correct.  It would be more correct to say, “what gets rewarded gets repeated.”  One of the most thoroughly accepted notions in psychology is the principle that behavior eventually extinguishes if it is not followed by reward.

I was on an airplane and in front of me was a father and his young daughter (she was about 3 years old).  When it came time to take off the father worked and forced his daughter to sit down in her seat with her safety belt fastened.  The child screamed and yelled.  The child wanted out of her seat. As an outside observer, I understood if he let his child out of her seat, the child would learn that the way to get out of her seat is by yelling and screaming.  The father gave in and let the child out of her seat and stand up.  The flight attendant had other ideas and told the father his child had to be seated.

Let’s look at the behavior and the reward.  The father’s behavior of allowing his child to stand in her seat was immediately squelched, and he was forced to follow the rules.   Can you imagine what happened the second time the father tried to fasten his child in the seat?  The child was more resistant and screamed even louder.  The father made the situation worse by giving into the behavior the first time.  In other words, the father rewarded the behavior of screaming and yelling.

Non-negotiable estimates
I was working with a manager, and I watched him interact with his client (an internal client). The manager estimated it would take 600 hours to complete a project.  His client pushed and challenged him, so the manager reduced the project by 100 hours.  This is a 20% reduction in time.  Then his client pushed a bit more and got the estimate reduced another 50 hours down to 450 hours.  This is a 25% reduction!  Whoa!  What behavior did the manager exhibit and reward?

Estimates need to be non-negotiable.   An estimate should be created using a quantified method.  That means there is some method to creating estimates.  Put some data into a formula and derive the result.  The only thing you should be willing to budge on is the inputs.   There are several inputs with an estimate including size of the project, deadlines, staff, so on and so forth.  Hence, if the estimate is too high, then one of the inputs needs to be changed.
Unfortunately, what traditionally happens is that an estimate is nothing more than a guess.  The estimate has no substance at all; in other words, it is not based upon historical performance or statistical modeling.  Often when working on a contract I ask the question, “How did you come up with your estimate?” More often than not the person actually admits it was a guess.   Another common answer begins, “based upon my vast experience as a software professional.”  In other words, questioning the estimate is the same as questioning their integrity.    It is important for an estimator to be able to quantitatively explain how they derived their estimate.
Unfortunately software development is full of bad behaviors that get rewarded and get repeated.
Read more at Reboot! Rethinking and Restarting Software Development

Welcome to Club Chaos!

The secret of all victory lies in the organization of the non-obvious.
– Marcus Aurelius (Roman Emperor AD 121-180)

Things Just Seem Complex

Software applications and documentation seem more complex when they are not organized. It does not matter if it is software applications, books, collections, or lives anything that is not organized seems more complex than it really is.   As John Maeda writes in The Laws of Simplicity, “Organization makes a system of many appear fewer.”   If applications and associated artifacts are not organized, then there is only a sense the application is complex.  By organizing application artifacts complexity of applications is reduced.

I have been to a lot of grocery stores around the world, including Rome, and it seems like they are all organized in a similar manner.  In any grocery store there is the fruits & vegetable section, meat & cheese, bread, canned goods, frozen foods, snack foods, liquor, so on and so forth.

Organized documentation and functionality is important for everyone especially the novice.  A grocery store is a complex system. Since the grocery store is organized, it is easy for the novice clerk to re-shelf the contents of the grocery store and the novice customer to find what they are looking for.   A novice is a person who is trying to understand something for the first time.  Think about all the different manufacturers of products or the number of countries where products originated to stock the grocery store shelves.   The goods only represent one layer of complexity.  Another layer is the scheduling of employees to clean the store, to shelve, and to run the cash register.  The beauty is all the complexity has been hidden from the customer.   The grocery store is an organized complex operation; yet, our interactions seem simple.

When software applications evolve without a comprehensive design, functions get scattered around and it becomes hard to navigate all the confusion.  Since there is no clear functional decomposition of the application, the user of the application and developer have a hard time finding a specific piece of functionality.   The application is difficult to navigate and users become frustrated.

Developers have a hard time finding functionality or understanding the functionality that already exists.   This causes redundant functionality to be created.  This extra functionality is not needed and it becomes additional expense to create and to maintain.

The perceived complexity is worse when multiple applications share information and communicate via undocumented interfaces.  If documentation is not organized which describes all the complex interrelationships it is going to be next to impossible to grasp what is actually happening.

It’s a small world after all – Global Software Development I

When I was a young boy my parents told me to eat all of my food because there are people in China and India who are starving.  I tell my kids and those college students I speak to study hard and to work hard because there are people in India and China that want your job.  They are smarter than you, and they are willing to work harder than you too.

Everyone who has had an economics course knows that success attracts competition and software development is no exception.   Many countries are predicting large growth in employment in software development.   Country after country expects to utilize software development as a key industry in hopes of accelerating job growth and positively impacting the entire economy.

It is not just a few countries making software anymore. The idea of software development begins simultaneously in several countries.   In the middle of the 1990’s most software was created in only seven countries Canada, France, Germany, Italy, Japan, United Kingdom and the United States.  These seven countries are known as the G7.  Based upon published government reports I estimate employment in software development in the G7 was about 2 million in 1990.  The current number employed in software development in G7 countries is around 5 million.

No one knows the exact number of those employed in software development worldwide. The current employment levels in software development in India are around 1.5 million, and in China, they are around1.3 million. The remaining world (Eastern Europe and South America) about 2.2 million employed in software development. In 2008, there were about 5 million employed in software development outside of the original G7 countries and about 5 million in the G7 countries.  Based upon all these official, government reports I estimate it to be around 10 million.

The G7 countries went from 100 percent of the software development industry in 1990 to 50% by 2006.   Right now it is a tie game, but most of the G7 countries are predicting steady 5 to 7 percent growth rates while India, China, and Eastern Europe are predicting 20 percent growth rates.  In less than twenty years 70 percent of software will be developed outside G7 countries.  Only about 15 percent of software will be developed in the USA.

We have seen these same patterns before.  A few countries dominated the auto industry, the wine industry, and steel industry.  There was rapid growth, a period of no growth, then decline.
Software does not have to be put onto a ship and transported across an ocean.  Countries have established import fees (duties) on many goods and services.  This will not be possible with software.   Software can be moved from one side of the world to the other in a blink of an eye.   Software does not have to stop at shore’s edge and be counted.    Even if it did stop at shore’s edge what would inspectors count?  How would they size the amount of software crossing a border?  It is easy to count the number of cars, tons of steel, or bottles of wine.
Tomorrow I will shed some light on software development country by country.

Changes in Lattitudes, Changes in Attitudes

As Jimmy Buffet sings, “Reading departure signs in airports reminds me off all the places I have been.”  I am on my way to California today and whenever I travel it reminds me of all the places and people I have met before.  After 3 million frequent flyer miles, working in every possible industry, and on every contentent there are somethings that are very predicatble.  I can count on software developers saying or maintaining the following positions.

1. Measurement is not possible in the software industry because it is too complex and unpredictable.

Many in the statistics profession, including myself, do not believe in random events.  Some statisticians and mathematicians go mad trying to find patterns.   There is no need to go mad in the domain of software development because many of the patterns are blatantly obvious.   The process only appears to be random because it has not been studied.  It is common for a clinical psychologist to see patterns in an individual’s behavior while the individual is totally blind to his or her own patterns of behavior.   One of the functions of a clinical psychologist is to get people to recognize these patterns and behaviors.   The most common way for psychologists to get people to recognize their patterns is through measurement and keeping a journal.

Many software organizations write journals which are more commonly known as lessons learned documents.  I often review lessons learned documents, and it is painfully obvious the same patterns repeat over and over again. If all the past lessons learned documents are pointing to the same problems, then it is safe to conclude the next lessons learned document is going to point out the same problems. There are only three possible reasons for software organizations to keep repeating the same behaviors: the first is denial and ignoring the glaring fact that the same bad stuff happens in every project, the second is “not my fault syndrome” or nothing can be done about it, the third and final reason it is just too hard to change.

2.  My intuition works better than systematics study.  A wet finger in the air seems to work best for us.

The software developer prefers to use intuition instead of systematic study.  In a method of intuition, what makes logical sense must be true.   I have heard many software developers say, “I know what I know.”   It reminds me of the lyrics from the song “What I am” by The New Bohemians: “I’m not aware of too many things, but I know what I know if you know what I mean.”   If knowledge cannot be communicated via some type of, measurement it is difficult to understand what is meant.  If any process is “too complex” or “random” to be studied using the scientific method by applying measurements, then it is a safe bet that it cannot be studied using intuition either.

Read more at Reboot! Rethinking and Restarting Software Development

Software development is not random. IT has pattens.

Software development organizations, like people, have patterns.  In other words organizational behavior is not random.  Organizational behavior, like the behavior of a person, has a pattern and therefore can be predicted.   If you study your software development organization I guarantee you will start to see repeating patterns be able to predict patterns and make systematic improvements. The only way to learn about a software development organization is by systematic study.  When studying a software development organization it is important to gather data from samples, make conclusions and then forecast the future.

The entire premise of software metrics is the best way to can improve a software development organization is by systematic study.    Peter Senge points out in his book, The Fifth Discipline organizations must learn if they are going to survive in a global economy.  Software development is a global industry that changes and moves rapidly.

The movie Groundhog Day exemplifies many software development organizations.  The basic premise of the movie is the main character, Bill Murray, never learns.   Too many software development organizations never learn.  They just keep making the same mistakes over and over again.

The best learning comes from systematic study of past projects.  Some organizations, like people, need to learn from their mistakes and their successes.  Unfortunately many software development organizations, again like people, never learn from their mistakes.   These organizations keep making the same mistakes over and over again.


The fancy or academic word for what you are about to do is, “Ethnography.” In ethnography a researcher examines the group’s observable and learned patterns of behavior, customs, rituals, and ways of life.

The lengthy study of Gorilla’s by Dian Fossey is an example of an ethnographical study.   If Fossey can successfully study and learn about gorilla’s in the wild,  then I am pretty sure anyone can  study and learn about their software development organization in an office environment.

Read more at Reboot! Rethinking and Restarting Software Development.

White Men Can’t Jump and Software Developer’s Can’t Estimate.

There are few things as predictable as software developers can’t estimate.  Nonetheless it software projects follow the same patterns.  Most small projects are underestimated by about 10%, medium size projects by 40% and very large projects by more than 200%.  As project size increases the error rate of the estimate increases with near mathematical precision.

Estimating problems are not to software development.  Gustav Theordor Fechner wrote a book Elemente de Pscyhophysik in 1860.   He coined the term “psychophysics.”   Over the years all the senses have been studied: vision, hearing, touch, taste and smell and the sense of time.   There are three main topics in psychophysical classification scheme: absolute thresholds, discrimination thresholds and scaling.  I am writing about scaling.  It is clear that the untrained are not good at scaling. On the other hand, those individuals that are good at scaling have been trained and they have adopted some sort of strategy to help them measure and then scale.   The point here is that estimating is a skill that can be learned.

The same pattern with estimating is seen over and over again.   A person is pretty good at estimating the height of a 20-foot tree and horrible at estimating the height of a 200-foot building.  The same would be true when they try to estimate the weight of a rock.  They are good at estimating the weight of a ten pound rook and very poor at estimating a 800 pound rock.  So it does not matter if we are estimating the weight of a rock, volume of a liquid, loudness, the height of tree or the size of a software project the error rate increases with size.  As size gets larger the error rate increases.

Musicians learn the specific vocabulary of dynamics.  Trained musicians can hear if one piece of music is being played louder than another piece of music.   As the music gets relatively louder they are able to distinguish loudness because they have been trained to do so.    Musicians are trained in scaling and they have a specific vocabulary for loudness.    They know that piano p means soft while pianissimo pp means very soft.  On the other hand they know that forte f means loud and ff means very loud.    If they are playing their instrument at p and their sheet music or conductor directs to play at ffff they know that means to turn up the volume about six times.   Musicians also know if they are to gradually get louder (crescendo) or gradually get software (decresendo).

You may be thinking that it is lack of experience that prevents people from estimating the height of a tree or loudness.  Wansink studied bartenders with an average experience of over ten years.  He removed all their devices to estimate the volume of liquor  they pour into a glass. When their strategies and measuring devices were removed they were unable to pour the correct amount of liquor into a glass.

The same would be true with musicians.  If suddenly all the symbols and strategies used by conductors to communicate with musicians were removed, then the musicians would have a difficult time playing at the right volume and right tempo. The musician would be constantly asking, “How much louder or softer?”

One of the values of measurement in software is it provides a vocabulary to communicate.  It allows developers to communicate with the core business and explain how much larger. Software developers are not aware of the importance of scaling.  They do not understand scaling is critical to sizing software projects.  The absence of a measurement of scale or an ability to size a software projects prohibits one software project to be compared to other software projects.

WeightWatchers trains dieters how to estimate their food consumption.   It is not just the amount of calories to consider because the grams of fat  and fiber are a important too.  Calories, fat and fiber are combined to determine points.  The point system is a method of scaling food consumption.  The “points” allow one food item to be compared with other food items.

The reason software development can’t estimate is because they don’t measure.   Those few that are successful at estimating software projects have adopted a strategy to size and scale projects.   The best estimators are those that base their size on functionality.

Read more at Reboot! Rethinking and Restarting Software Development.

Unskilled and Unaware

Many software organizations are left with the mistaken belief they are doing fine when in actuality they are not.  Individuals in these  organizations actually believe they are doing a “quality” job .   These individuals  have this false belief because they do o not have the fundamental skills necessary to realize they are actually doing a poor job.

One of my favorite psychology papers is called Unskilled and Unaware.  The premise of the paper is when people are incompetent they suffer a dual burden.   First, they reach erroneous conclusions and make bad choices.  Second, their incompetence robs them of their ability to realize they are making bad choices and they are left with the mistaken belief they are doing fine.   Since they think they are doing fine, they will never learn from their mistakes.  They will never make an effort to improve until disaster strikes.

Those skills making one competent in a particular domain are often the very same skills necessary to evaluate competence in the domain.  Because of this, incompetent individuals lack what cognitive psychologists call “metacognition” or self-monitoring skills.  In other words they are just too ignorant to realize they have a problem.

Read more at Reboot! Rethinking and Restarting Software Development.