Is documentation still relevant?

There was a great question posted on linkedin by Danielle Vokski.  Her question was, “Do you think that documented working procedures are still relevant in the changing climate of todays market?”

In todays changing and evolving market place there has to be a way to communicate and share ideas.   Documented work procedures do not have to be a written document.  It can be a video, podcast or an animation.

Keep in mind complexity does not necessarily equate to more documentation and a thicker user manual.  Consider the difference between an automatic transmission on a car versus a manual transmission. Clearly the automatic transmission is more complex, but requires much less documentation and user training than a manual transmission.

Another problem is if the environment changes then the procedures needs to change too.  The procedures become dated and useless if they are not changed.  Yet, the change needs to communicated to all those that are impacted by the change.

If the process is intuitive and/or the product is designed well, then it does not require much user documentation.   Don’t be confused because I am not advocating something like the code is the documentation.  What I am suggesting is the process has been studied so much and designed so well that it is intuitive.  If the process or product changes then it is intuitive how to adapt.

The documentation and skill required to repair an automatic transmission is much more than that of a manual transmission.  In the end whose perspective matters.

So let me answer Danielle’s original question with two answers.

1.  User documentation needs to become irrelevant. If the product is designed well and it is intuitive.

2.  Documentation is still relevant for the technical documentation.  The more complexity that is hidden away from the user the greater the skill to understand all the inter-workings.http://www.dotcodedump.com/2009/10/tips-for-effective-documentation/Read more at Reboot! Rethinking and Restarting Software Development

Links for writing better documentation

Tips for effective documentation –http://www.dotcodedump.com/2009/10/tips-for-effective-documentation

/http://techwriter.dk/writing/writing-good-documentation.html

IT the toilet that never stops flushing

I was consulting for a large international firm and one of the executives on the business side made the comment that, “IT is the Toilet that never stops Flushing.” To often this is an accurate phrase to describe how money is spent in IT. Most of us have had a running toilet in our bathrooms and it can be very annoying. In the end a running toilet is just money down the drain and too often spending money on IT is money down the toilet too.

Over the past 20 years a lot of IT spending was done without careful cost benefit analysis.  Too many projects were undertaken because there was a desire to move to the latest and greatest technology.  The current recession is causing IT to really justify its spending.   The business is forcing IT to wiggle the handle to see if they can’t stop the water from running.

Even though the current fade de jour is build as you go or ad hoc software development it is an expensive proposition.  Those IT organizations that embrace, study, and learn  the core business are the same organizations with the highest productivity rates, best quality and lowest costs per unit.

Read more at Reboot! Rethinking and Restarting Software Development

TPS Reports and Cover Sheets

Too often people are asked to record their time or report data and are clueless the reason.   They start thinking, “what’s the

initech

point?”  They think of the ole, “IniTech – T.P.S. report” from the movie Office Space (TPS Time Sheets & Cover Sheet).

When people see how the data is being used, the value, “what’s in it for me”, then they are more likely to input data correctly and sometimes enthusiastically.

When I develop metrics for an organization, I have to rely on the quality of time sheets. Often people tell me the time sheet data is no good. If I am building estimating models based upon past data and the data is no good, then, the estimating model is going to be wrong too! I have to make a lot of adjustments to the historical data. Knowing how people spend their time on past projects is an invaluable management tool when planning for future projects. When projects are planned properly there is less chaos and often less overtime. Overtime is often the result of poor planning and poor estimating.

Read more at Reboot! Rethinking and Restating Software Development.

TPS Initech Time Sheets

Just Google it! Huh?

There is a misconception it is okay to have documentation scattered about individual computers and a “shared drive.”  The myth is documentation can be found via searches and it does not need to be organized.  Some organizations want to implement a Google style search mechanism.  The beauty of the Google search engine is it seems so simple.  Google relies on creators of websites linking information together and some relatively complex algorithms.  The founders of Google wrote a paper entitled, The Anatomy of a Large Scale Hypertextual Web Search Engine outlining how the Google search engine would work.   It is not as simple as it looks.  Most organizations with disorganized documentation cannot get their developers to use consistent terminology nor can they get them to embed hyperlinks in a document.

If a specific document is referenced by many other documents, then the document is significant.  If the document is not be referenced via some linking mechanism, then there is no way to determine its significance. The problem, of course, is getting individuals to link documents together.   Believe it or not I have had developers argue this point.   Even though they have never read the short paper or long paper written by Brin and Page the founders of Google they just somehow know all the inter-workings of Google.  They think it would be easier to design a complex search engine with artificial intelligence not yet invented by mankind instead of standardizing terminology and using some sort of naming convention for their documents.

Read more at Reboot! Rethinking and Restarting Software Development.

http://infolab.stanford.edu/~backrub/google.html

What gets rewarded gets repeated!

A behavior followed by a reinforcing stimulus results in an increased probability of that behavior occurring in the future. – B.F. Skinner, Psychologist

The idea of “what gets measured gets done” is only partially correct.  It would be more correct to say, “what gets rewarded gets repeated.”  One of the most thoroughly accepted notions in psychology is the principle that behavior eventually extinguishes if it is not followed by reward.

I was on an airplane and in front of me was a father and his young daughter (she was about 3 years old).  When it came time to take off the father worked and forced his daughter to sit down in her seat with her safety belt fastened.  The child screamed and yelled.  The child wanted out of her seat. As an outside observer, I understood if he let his child out of her seat, the child would learn that the way to get out of her seat is by yelling and screaming.  The father gave in and let the child out of her seat and stand up.  The flight attendant had other ideas and told the father his child had to be seated.

Let’s look at the behavior and the reward.  The father’s behavior of allowing his child to stand in her seat was immediately squelched, and he was forced to follow the rules.   Can you imagine what happened the second time the father tried to fasten his child in the seat?  The child was more resistant and screamed even louder.  The father made the situation worse by giving into the behavior the first time.  In other words, the father rewarded the behavior of screaming and yelling.

Non-negotiable estimates
I was working with a manager, and I watched him interact with his client (an internal client). The manager estimated it would take 600 hours to complete a project.  His client pushed and challenged him, so the manager reduced the project by 100 hours.  This is a 20% reduction in time.  Then his client pushed a bit more and got the estimate reduced another 50 hours down to 450 hours.  This is a 25% reduction!  Whoa!  What behavior did the manager exhibit and reward?

Estimates need to be non-negotiable.   An estimate should be created using a quantified method.  That means there is some method to creating estimates.  Put some data into a formula and derive the result.  The only thing you should be willing to budge on is the inputs.   There are several inputs with an estimate including size of the project, deadlines, staff, so on and so forth.  Hence, if the estimate is too high, then one of the inputs needs to be changed.

Unfortunately, what traditionally happens is that an estimate is nothing more than a guess.  The estimate has no substance at all; in other words, it is not based upon historical performance or statistical modeling.  Often when working on a contract I ask the question, “How did you come up with your estimate?” More often than not the person actually admits it was a guess.   Another common answer begins, “based upon my vast experience as a software professional.”  In other words, questioning the estimate is the same as questioning their integrity.    It is important for an estimator to be able to quantitatively explain how they derived their estimate.

Unfortunately software development is full of bad behaviors that get rewarded and get repeated.

Read more at Reboot! Rethinking and Restarting Software Development

Measure what is not measurable

It does not matter what is trying to be done, if it can’t be measured, it  can’t be understood , controlled, or predicted.  If it can’t measured it  can’t moved forward.  If the software development process can’t be measured it can’t be understand it.  Measurement has been the basis of all science since the time of Galileo.   The idea of measurement is nothing new; it is only new to the field of software.  If the software development process is not measured it cannot be studied, it cannot be understood, and it is difficult for the organization to move forward. Measurement is one of the most ordinary actions.  We speak its language whenever we exchange goods or information.

Better A Surgeon’s Notes on Performance is a recent book written by Atul Gawande.   The book is an investigation into how medical professionals can progress from good to great.    Many of the lessons outlined in the book apply to everyone including software developers.  Gawande recommends, “Count something. Regardless of what one ultimately does in medicine or outside of medicine – one should be a scientist in this world.”    This applies to software developers too.  To be considered a scientist, something needs to be counted and measured.   It is common for software developers to refer to themselves as a computer scientists or software engineers.  Both of these titles imply utilization of the scientific method, systematic study, and measurement.

Knowing something
The most common and accepted way to know something is through systematic study and utilizing the scientific method.  Another way to know something is through personal experiences.  Through all experiences, each individual constructs a “reality” of the world.  The danger arises when this method is the only method since biases develop or information can be distorted. Often, the events experienced represent a biased sample that in turn can lead to inaccurate conclusions.   Most biased individuals do not see themselves as biased, and often they defend their opinions with a lot of passion. Basically, the scientific method is a set of assumptions and rules about collecting and evaluating data.  The idea is to reduce bias as much as possible, and the proof of the science is the data. “Stripped of fall its glamour, scientific inquiry is nothing more than a way of limiting false conclusions.”

The real conflict in software  is between systematic study and personal opinion.  There are a number of software professionals who rely on personal experience and hold the misconception that systematic study and measurement of the software development process is not possible or not necessary.  Some believe that measuring the software development process is just too complex.

One reason why software professionals do not measure is because they do not have the necessary basic quantitative and qualitative skills.  They do not know how to study a software environment.  Many of them studied computer science where there was an emphasis on learning programming languages instead of learning about the software development process. The emphasis has been on writing code and not on the idea of developing software to solve customer problems.   Since the emphasis has been on programming, it is hard for a programmer to explain how he or she knows what they know.  There is also limited knowledge and experience of what goes on before  code is written and what happens after it has been written.  The reason for this is because there is little interaction with the actual customer or the person who uses the products.  Many developers do not have the desire to know the customers either.

Read more at Reboot! Rethinking and Restarting Software Development

“I will tell you… I don’t know…its Tradition!”

Some friends or mine David and Jo Cowdery operate a vineyard and winery in the south of France called Chateau La Bouscade. They have received several awards for their wines, so I was excited when  I received a couple of bottles wine from them.    What I found interesting is the bottles were sealed with screw caps not corks.   Since I do not know a lot about wine, I asked my friend Jo, “What’s up with the screw caps?”  She laughed and told me that screw caps were far superior than corks.  She told me that all the wines at Chateau La Bouscade have screw caps.   “Say it ain’t so, Jo!”

The scientific evidence is overwhelming that screw caps are better than corks in a bottle of wine.   It turns out a bottle of wine with a cork has at least a 1 in 10 chance of having cork taint.  Cork taint ain’t a good thing either.  It makes wine smell like shoes or a wet dog.

So what does this have to do with software? A lot. I have spent years gathering scientific evidence on the best methods to develop software and the scientific evidence is overwhelming.  I am not alone because organization like SEI and QAI share my findings.  Just like it is hard for get consumers of wine and wine makers to change it is hard to get software developers to abandon traditions.

So why do wine makers still use corks and how did this tradition get started?  Tevye a character in the Fiddler on the Roof sums it up nicely, “I will tell you… I don’t know…its Tradition!” The same can be said about software development.   I don’t know why so many software organizations refuse to adopt best practices.

There is an old proverb saying, “Seldom does Saul become Paul.” It means that seldom do people actually change.  What happens is a new generation has to grow up with the idea from the beginning.   There are pioneers like David Cowderoy who lead the way for a whole new generation of wine makers.    The new generation starts with the idea from the beginning.  A new generation of human factors engineers are going to be the future of software development.   These human factors engineers understand the importance of studying actual users and customers.  One best practice is to study the business and not to be passive in the requirements process.

I was speaking at a conference and I was asked the question,  “if things like metrics are such a good idea, then why don’t more software organization adopt metrics?”  I don’t know why?  I don’t know why people are overweight or why they smoke or why some wine makers still use corks.  I guess it is like Tevye says, “Tradition!”  Because It can’t be explained with logic.

Cheers!

Read more at Reboot! Rethinking and Restarting Software Development.

———————————————————————————-

For my  non American friends
“Say it ain’t so Joe” is a reference to Shoeless Joe Jackson who was accused of fixing the outcome of  the 1919 World Series.  Legend has it that as Jackson was leaving the courthouse during the trial, a young boy begged of him, “Say it ain’t so,  Joe!” and that Joe did not respond.

To learn more about the argument of screw tops v. cork  see http://www.hoguecellars.com/feature/homework.html

Bah, Humbug! A look at IT Past, IT Present and IT Future

Everyone knows the story of the Christmas Carol.  Charles Dickens examines Christmas past, Christmas future, and Christmas present. As perhaps the worlds only Software Economist, I examine IT past, IT present and IT future in this short article.  Like Dickens, I have endeavored in this Ghostly little article, to raise the Ghost of an idea.

1. IT past.
2. IT present.
3. IT future.

All industries have cyclical ups and downs.  There is no reason to believe that the software development industry (IT) will not follow the same up and down cycles like other industries and the overall economy.  When the economy rebounds will so IT.

The more I study, learn and experience software development, the more I am convinced that software development is just like every other industry.   What a Ghostly idea!

The Current State of IT
What has happened in the field of software has happened before and has happened in a lot of other industries too.  The recession in IT is worldwide problem and not isolated to the United States or Western Europe.  An Australian friend who does IT consulting in Southeast Asia wrote, “The few consulting jobs I had planned in 2009 in Malaysia, Singapore, and Vietnam are gone or on hold for the time being.”  Another friend living in Europe wrote, “these are tough economic times and it is a bad time to be looking for a job.”   More than half of IT employees and contractors in the UK express dissatisfaction with their career path.

It seems like there are comments on all the Linkedin groups about the economic woes of IT.

According to a recent CIO survey CIO’s are continuing to put discretionary IT projects on hold, renegotiating IT vendor contracts, and freezing IT hiring.   Nearly 64% of the CIO’s reported they were freezing IT hiring and nearly 43% planned on reducing head count.

Before we get depressed here are some basic economic principles:

1. All industries have cyclical ups and downs. There is no reason to believe that software will not follow the same up and down cycles like other industries.  For those of us with a bit of gray hair we remember the slump in oil industry in the 1970’s and look at the profits of the oil industry today.

2. Recessions are like forest fires because they clear out a lot of the dead and unnecessary wood. Investment in IT needs to be re-examined and carefully scrutinized.   Too much investment was based upon “latest” and “greatest” technology and not on sound cost/benefit analysis.

3. When the economy rebounds so will IT. IT is following the same cyclical pattern as the overall economy.

The unemployment rate for the software development industry rose over the last year.  Overall the unemployment rate for software development rose from 2 percent to 4 percent or it just about doubled.  According to the Bureau of Labor Statistics (bls.gov) during the past year the

– Software Engineers unemployment rate rose 1.9 percent to 4.1 percent

– System Analysts unemployment rate rose 3 percent to 5.7 percent

– Computer and Information Systems Managers unemployment rate rose 2.7 percent to 4 percent

This is just about the same level of unemployment for college educated individuals.  While the overall unemployment rate is just around 9.5 percent,  the unemployment rate for those with a bachelor degree is around 4.8 percent.   The rise in unemployment in the software industry is following the same trend as overall employment too.  The unemployment rate for individuals with a bachelors degree or higher went from 2.4 to  4.8 percent during the same period of time.

IT Past
Not too long ago, there were companies testifying in front of congress saying there was as shortage of IT workers and they need more H1B visas.  During the early 1990’s many proclaimed that the IT industry was recession proof.   Some experts proclaimed there would be a shortage of IT workers by the year 2010.   IT turns out these experts were wrong.   It seems that IT is following the same economic trend as the overall economy.  I feel like I am repeating myself.

Many in the IT industry were fat and happy and did not continue to improve skill sets or learn about the business.  It seems like few in IT actually study the core business.

IT Future
The software industry will return and be healthy again.   Will the software industry be exactly the same as before?  No, this much we know for sure.  It will emerge and become healthy again, but the shape and especially the location is not known for sure.

Those individuals that take the time during a recession to sharpen skills and increase their skill sets will prosper.   There will be many individuals that will leave the software development industry for good and look for career options elsewhere.  This same thing happened in the late 1970’s with the oil industry.

This is the time to polish skills and sharpen your tools.  The internet provides an avenue for individuals to self study without outlaying any cash.  The library offers a lot of online resources and current technology books too.

Specific Trends

Continuation of Offshore Development
Clearly a lot of IT jobs are being exported and this trend will continue. Other industries like textiles moved completely offshore and the design and marketing jobs remained in the USA.    It appears the same is going to be true of software development.

The number of graduates in computer science continues to plummet.  The number of those choosing to major in computer science has fallen about 50% since 1999. BUT the number of PhD’s getting computer science degrees continues to grow.  The percentage of non residents (international students) getting PhD’s in Computer Science rose from 20% in 1985 to nearly 80% in 2008.  Most of these doctoral students are from Asia.   These PhD’s return to their home countries and teach computer science.  This means the instructors for off shore developers are being trained at schools in the US.

The next round of off shore developers will be better trained than the first round of developers.  It will be increasingly difficult for US and western European developers to compete on skills and on price.

Specialization
As industries mature they specialize.  Software development is still an immature industry.   There are a variety of industries that have been around for a few centuries like  engineering, medicine, law, architecture and textiles and they all have specialists along an industry line.

In the early nineteenth century a skilled seamstress performed all the roles necessary to make a dress. All dress making was customized. The industry matured until several specialists performed all the roles once performed by the seamstress. Designing, planning, and marketing became a specialization too.  There are some designers who specialize in men’s clothing and others that specialize in just designing wedding dresses. This trend is already starting to happen in software development.

Less Technical Skills and More Business Skills
The long term trend is away from technical skills and towards business and design skills.  When the only tool was a needle and thread it took a lot of skill to sew something.  With the invention of the sewing machine, less skill was required of the seamstress. Clothes were first worn for very functional purposes and new we use them for both a functional and design purpose. The same was true with software development and especially with developing websites.    The first software programs were for functional purposes only and now design does matter.  Not long ago it took a strong understanding of html to develop a website but as tools improved the necessary technical skills diminished.  (a nice graph here).

While the number of computer science degrees have plummeted the number of degrees in human factors design continues to grow.  This is going to be a driving force toward understanding the business and understanding the user.

Summary
I have a heard a couple of themes from software developers for a number of years.  The first is software development is somehow different from all other industries.  No one can really give me a reason beyond “just because it is different.”  As things shape up it appears that software development is just like every other industry.

The second thing and more to the point here is I have heard from many software developers they are better than their counterparts in India, China or eastern Europe.  When I push and keep asking the question, “What are you better at?” They answer, “I am better at requirements, design and knowing the business.”  This is true!   Americans and western Europeans need to focus on what they are better at and move away from technical skills because there is no long term future in it.

The number of developers in Asia and eastern Europe continues to grow.  They are becoming better educated too.  It is going to be harder and harder for an American or western European to compete on skill and especially on price.

Government IT Dashboard

For those companies interested in contracting with the Federal Government there is a new site that lets citizens follow the Government’s IT money trail.  It will not surprise anyone that Department of Defense (DOD) dwarfs all other IT government spending.

Check it out at http://it.usaspending.gov/

A pint between friends!

Measure what is measurable, and make measurable what is not so.– Galileo Galilei (1564 – 1642)

I hope my British friends do not take offense to this, but the current measurement system in England is confusing.   Milk brought to the house by the infamous milkman is in pints, but the milk sold in stores is in liters.  “Petrol” stations sell gas in liters and advertise the price in gallons.  The distance between cities is in miles, but other road signs are in meters.   Of course, beer is sold in pints!  Everyone just knows when you walk into British pub you ask for a pint.

It is not uncommon to have local rules that dictate measurement.  Most people do not know that the volume of a UK pint is not the same volume of a USA pint.  One UK pint is equal to 1.20 USA pints.    A pint in the US is about twenty percent smaller by volume than a pint in the UK.  If a “pint” of beer is ordered in London, it is going to be about twenty percent larger than a pint of beer in the USA.  When my British friends order a pint of beer in the USA, they typically spend some time examining the size of the glass.  They often comment, “This doesn’t look right.”  They know something is wrong, but they just can’t identify it; they just can’t put their finger on it.

I have had CEOs tell me the same thing about their software organizations.  They tell me, “Things just don’t look right.”  Malcom Galdwell points out in his book Blink that intuition is often right, especially for those with a lot of experience.   The problem is that it is difficult to persuade someone based just upon intuition.   In the end, everyone should be able to explain and answer the question, “How do you know what you know?”   Without measurement, many people get attached to their beliefs and embrace all the emotions that go with their opinions.  In the absence of measurement and data, the person with the highest pay grade wins the argument that is based upon opinion.

I tell my British friends the reason a pint is smaller in the States is due to King George.  I tell them that King George taxed the colonist in America and shorted them on beer.  History tells us about taxation without representation, but fails to tell the story about the beer.

Read more at Reboot! Rethinking and Restarting Software Development (www.RebootRethink.Com)

Cheers!

David Longstreet

Software Economist.