Tips For Software Developers on Seeking Help and/or Reporting Problems with Code They are Writing

Throughout courses I have taught, students have asked for help because some code they are writing isn't working, or they have received error messages (Java, Umple, shell scripts, etc.) that they don't understand.

Throughout any software developer's career they will encounter coding problems and need to either seek help or report problems with technology they are re-using. They might be reporting a bug in a code library their company uses, or seeking help on Stack Overflow. The principles are the same when they are when reporting problems or asking for help from a technical mentor, Stack Overflow, a TA or professor in a course.

Too often, beginners don't bother to report problems or don't do so in an effective way. I have distilled the following from dealing with way to many incomplete, unhelpful, or unneeded reports or help requests.

Things to do when you have a coding problem and feel you need help or think it might be a bug:

1. Boil the problem down to the smallest case that still has the issue. So if you have 200 lines of code with an issue, subtract  parts of the code bit by bit until the problem disappears, then add the last bit back. Keep trying to subtract other bits. See if you can submit a question or issue just related to perhaps 5 lines of code, or the smallest reasonable number that manifests the probem. To determine the bits to subtract, focus on just the changes you have made to the previous working version of the code by doing a 'diff' with the previous version.

2. Try hard to ensure you cannot solve the problem yourself first. For coding problems, clean up the remaining code after step 1 code so it is nicely indented. Check for balanced brackets and quotes. Add some comments to it (this can help you understand it and often solve your own problems). Make sure you are using the latest version of the technology (the problem may have been fixed very recently -- maybe even in the last days -- so update to the latest possible version). Check on Stack Overflow or other places in case others have asked the same problem (e.g. in the chat tool your group or company uses). Search the relevant parts of the user manual (usually online) to see if you have misunderstood something.

3. If at this point it is a defect or failure you want to report, then look in the bug-reporting system to see if it has already been reported. A Google search for <technology> bugs will normally take you to the relevant system. And a Google search for <technology> bug <description of bug> will often locate other reports. If you find another report, you might find help for working around the problem, or you might be able to add additional information that will help developers. If you don't find a report, then you can add your own issue report. Give it a very descriptive and precise title and a good initial paragraph describing the issue.

4. Submit actual code that you are having problem with, rather than (just) a screen shot. Actual code that can be copied and pasted allows the person helping you, or the person reproducing the problem to very quickly run it themselves. Eyeballing the problem on a screenshot might not be enough: Sometimes the root of the problem is somewhere else in the code, or it is a problem with 'invisible' characters. And you cannot expect the person who has the problem to type in your code. You can also send a screenshot if it somehow sheds additional light on the problem (e.g. shows the error in context), but the actual code is critical. If you are sending error messages that refer to a line of code, make sure the lines numbers are visible in any screenshots.

5. If the problem involves a user interface, send at least one screenshot of that. Remember that usability difficulties are legitimate problems to be reported, or to seek help about.

6. If the problem is on the web, check if it occurs in at least 3 browsers (Chrome, Firefox, Edge, Safari, Opera), and report any differences. It maybe a browser incompatibility.

7. Make sure you carefully describe relevant aspects of your setup: Operating system and version, programming language or library version installed (e.g. result of java --version), browser version(s), IDE used, environment variables set.

8. Describe any inputs or steps that need to be done to reproduce the problem. This includes command line arguments, compilation steps, etc.

9. If it is not obviously an error you are reporting or asking about (e.g. if it is something you are trying to achieve), then describe in sufficient detail what you expect to happen or are trying to make happen,


Above all! Report. Don't just let problems lie there. Be proactive in reporting problems. It is your professional responsibility. So many times I have heard reports like, "All 100 of us had that problem, and we spent ages trying to work around it". The problem may have had a 10-minute fix and the developers may be very greatful to get the report.

Should you change all your passwords due to HeartBleed? I say no!

Huge number of companies and experts are saying 'change all your passwords' or 'change ours'.

Here's my take on it. I say don't blindly change your password on all sites:

A. Change your passwords on individual sites when all four of the following are true:

• you used password-protected areas of the site between April 1, 2014 and the date the site announced it has patched the bug (or someone logged into using your account)

AND

• the site reports it was vulnerable or was reported by others as vulnerable. See here for the status of some sites; consider a site vulnerable if you can't find out any information about its vulnerability and are worried because it contains sensitive data.

AND

• The site contains information that could cause harm if it was exploited, or your password is similar to a password on another site that you would care about.

AND

• the site does not use two-factor authentication (e.g. sending you a text containing a one time special code when you log in) or similar backup security mechanisms.

B. Also, change your passwords on other sites where:

• you use the same or similar password to those you had to change in item A (but now try to make the passwords reasonably strong and different -- see my guidelines below).

OR

• the site stores particularly risky information and recommends a change. This would apply to banks and taxation agencies that were affected, perhaps even if you haven't logged on for a longer time. Note that most banks report they were not affected. 

So I guess most people might end up changing 20% of their key passwords based on the above, but certainly not all of them. Why do I not say 'change all your passwords' to be safe? It is because there is significant risk and this is a classic 'lets overdo it' panic situation:

1. Some sites are just not affected. Many important sites like most banks, Apple, and Microsoft are just not vulnerable. Other sites have secondary mechanisms in place and have determined that users are safe.

2. There may be residual sites that still have the vulnerability; if you use one of these with your new password(s), then you are compromised when you weren't before.

3. The HeartBleed bug works by looking at transmitted data or data nearby where transmitted data is stored; if you or someone you know have not logged on (and your computer has not automatically logged you on) to a vulnerable site there is highly unlikely to have been data accessible to the bug that contains your password.

4. The password reset process itself has risks: Many people actually don't know many of their passwords, and rely on a tool to remember it for them, or have remained logged on essentially forever. In such cases, sites typically send a reset link; if a hacker truly wants to get you they may have ways to intercept that link, or generate fake links anticipating that people are in the middle or resetting their password. Some sites even send the original password back to you unencrypted, which is dreadful.

5. Many people now have hundreds of accounts, and several dozen they use regularly. It is essentially impossible to change all passwords and remember them all, so likely you will end up resetting passwords again in the future, or be forced to write then down or use an easily-guessed pattern. These add extra risk.

For unaffected and low impact sites (i.e. ones not dealing in financial and personal data) the risk of an attack on you is very small. In my Opinion, the risk posed (items 2, 4 and 5 above) by everyone changing their password, when multiplied by the low probability in most cases (items 1 and 3 above) outweighs the benefits of the blanked 'change all of them' advice.

For ongoing security with passwords. Here's what to do as a consumer:

• Use passwords that are at least 6 characters, are not just letters or numbers; use special characters in passwords if the site allows.
• For financial institutions, governments and other agencies processing sensitive information use completely distinct passwords from all others.
• For other sites, make sure there are several characters of difference even if you follow a password pattern.
• Only change your password based on my guidance at the top of this email, or if you think someone may have a  way to guess your password, or have specific reason to want to hack you.
• Never click on a link that says to change a password unless you have requested such a link in the last few minutes. In other circumstances, go to the website by typing the URL or using a bookmark you have used before.

Here's what to do as a site administrator or programmer

• Allow passwords to have any combination of letters, numbers and special characters and be of very long length. Don't restrict password content other than for minimal length, or requiring at least two of the above types of characters. So many people run into sites that have complicated rules (short password, no special characters, etc, that they have to make up a password they will inevitably forget).
• Implement two-factor authentication if there is a high risk of compromised information.
• If your site his risky information such as substantial personal or financial data, implement some other forms of extra security, such as challenge questions when a computer at a different IP address range is used, and gradually slowing-down of response as more and more password attempts are entered.
• Don't block people from using password managers without good cause. Password managers likely result in a net increase of security. 
• Put in place a robust reset process that uses multiple factors. Force people to phone if some of the factors are not present. Factors might include emailing their stored email address first, without a reset link initially, and verifying some other known personal information first.
• Allow people to save multiple email addresses, so if people change service provider you still have a way to contact them to verify identity.
• After a password is changed, email people at their email addresses of record, to alert them that the password has been changed.
• Never put a link to any password-protected website in any email you sent to people; the only exception might be a link sent in a reset operation that follows the above guidelines, is sent instantly on request, and is only valid for a very short time.
• Always think about usability as well as security; low usability of a security setup will force people to use simple passwords, write them down, or abandon your site.

Some other sites of interest include this and this. My opinion above contradicts these sites to some extent.

What's Important in Software Engineering Education articles still having an impact

Over a decade ago I conducted research in which I determined that certain areas of software engineering education are under-taught and over-taught, based on their relevance to the careers of software professionals.

The three articles published from this research continue to be cited and discussed. Just recently Robert Glass and Johann Rost summarized some of my findings in their Dark Side of Software Engineering blog. They invited me to add some of my thoughts, which became part of their blog post. I invite you to read their blog entry by clicking here.

In my original papers, I found:

• The top topics needed in the workplace are specific programming languages, data structures, software design, software architecture, requirements gathering and analysis, and human-computer interaction.
• Most of those same topics are also those for which employees need additional training, and which are under-taught relative to their importance.
• The most over-taught topics relative to their importance are differential equations, calculus and chemistry. These are also among the topics most forgotten.

The three papers from my original research are below; please cite them:

• What Knowledge Is Important to a Software Professional? (IEEE Computer, vol. 33, no. 5, pp. 44-50, 2000). 100 citations according to Microsoft Academic Search. Interestingly, the first submission of this article (to IEEE Software) was rejected, as being uninteresting to the IEEE software readership. That's good, because IEEE computer has a much larger readership.
• Priorities for the education and training of software engineers, Journal of Systems and Software - JSS , vol. 53, no. 1, pp. 53-71, 2000 - A more in-depth article about this. (24 citations)
• A survey of the relevance of computer science and software engineering education, Conference on Software Engineering Education and Training - CSEE&T , 1998 - The original article with preliminary data. (35 citations)

Note that the raw survey data is still available online, and has had over 10,600 hits, suggesting the data is being used quite a bit.

I am planning on repeating the study and comparing the new data with the original. I will add a few topics that have become important in recent years such as web development, mobile development, agile methods, and game development. I will also examine recent curricula for additional items.

If you are interested in helping either as a participant or to help distribute the survey in your company, please email me with the subject line Education Relevance. In the original survey it was extremely useful to receive the assistance of senior managers and HR personnel from several companies. I was able to give those companies sub-analyses relevant to their employees.

In my comments in Glass and Rost's blog, I added a few additional points worth reiterating (these are my opinions, not based on any survey):

• Central to becoming a productive software developer, in my opinion is practice, practice, practice, but structured so the practice is spread out over diverse technologies, problem types and application domains.
• There are different sub-contexts for education. If you want to be a quality-assurance expert, you would want to focus on material that would be of much less interest to the 'average' developer; and clearly many of the less-important topics such as calculus will be critical to certain application domains. The survey results should not be interpreted as meaning that all software engineering or computer science educational programs should give uniform emphasis to the same set of topics.
• The notion of agility is becoming not only important in the work force, but also in education: Educational programs need to become more agile at responding to the needs of students (actually I think the reverse is happening in some contexts where fear of losing accreditation drives decisions). But, even more importantly, having students work on agile projects can, I think, help them learn the most important material they will need. I have found UCOSP student projects very successful in this regard.

Top degrees: Computer Science PhD and Software Engineering Bachelors

An analysis by the thebestdegrees.org website presents interesting data that should encourage high-school students to enter the computing profession.

They rank PhD in Computer Science as the second-best degree, and Bachelors in Software Engineering as the third-best degree. Bachelors in Computer Science is not far behind at 7th.

Their criteria combine job demand and salary, and are based on data from the US Bureau of Labor Statistics. This is not the only study to come to similar conclusions; every year fresh studies say the same thing. For example CareerCast rated software engineering the top career earlier this year, and another study used BLS data to draw interesting charts showing that software engineering is expected to dominate new job creation among all technology jobs.

I always tell people that software engineering and computer science degrees are among the few that essentially guarantee good jobs in the student's field of study. Note that most computer scientists end up practicing software engineering when they are actually hired (for my discussion of the differences, written a decade ago but still valid, see here) and I consider a CS graduate in an SE job to be, broadly speaking, practicing 'in their discipline'. Students who obtain most other degree types have a much higher chance of having to settle for a career outside the discipline of their degree. This includes degrees in the ever-popular biosciences or social sciences.

However a decade after the famous supposed tech-crash, a lot of members of the public remain incredulous when I tell them the above. They still have in their mind the total fallacy that computing is a field with high unemployment, largely because they have heard media reports of layoffs at prominent companies. But other hiring companies abound and there have been huge numbers of high-tech startups in many areas of the world.

At the University of Ottawa, where I am a professor, our first-year enrolments in these fields have only just this year begun to approach the levels of the mid-to-late 1990's, after being dramatically down for 7-8 years. Enrolments are still well behind their tech-boom peak of a decade ago. Meanwhile we are bombarded by companies desperately trying to hire people, but unable to find them. Co-op students complain of having too many interviews (which takes too much time away from their studies). By their fourth year, many of our top software engineering students have found jobs in at companies like Google, Adobe and Oracle at their head offices in the US – sucking fresh talent out of the local Ottawa market. A quick plug: UOttawa pioneered software engineering education in Canada and internationally; when our students go to these companies they report that colleagues from very prestigious US universities admire their abilities.

My message is this (and it is a message I have been telling people for 10 years): Tell anyone you know who is in high school or has kids of that age, and who has any interest in technology, to go into software engineering or computer science. There never was a crash of employment in the field, just minor blips that have long since passed. There are plenty of jobs as the above links show. In my own city, here are some links from job-posting aggregator Jooble to companies hiring now using 'software' and 'computer programmer' keywords.

As for doing your PhD in Computer Science (second-top job). I am glad to be one of those people! It is overall a great career. And most of my PhD graduates are doing well. However, new PhD's in Computer Science shouldn't count on becoming a professor any time soon, there are very few openings precisely because of the lack of undergraduate students over the last decade. This will hopeful rectify itself over the next decade, but consistent undergraduate enrolment increases are needed first. One of my five PhD graduates is a professor; two are independent consultants and two are working for high-tech startups. The challenge for those who are not actually professors is to maintain their research skills and publication record while waiting for the faculty-position job market to loosen up a bit.

Steve Jobs: A Personal Inspiration

Here's are my personal thoughts about what Steve Jobs has meant to me, and how I hope many aspects of his like will continue to serve as an inspiration to me for many years to come.

I have been an Apple user since the earliest days, and have exclusively used Macs as my personal computers since 1984.

It was partly the stark simplicity of the early Macs, and of many later Apple products, that led me to focus my career on usability and simplicity in software engineering.

But perhaps, above all, it is Steve Job's rejection of many technology dogmas, and his advice to not be afraid of changing course, and even quitting a job that one doesn't like, that I think it is most important to take as an inspiration.

Steve Jobs quit college to pursue his dreams. While I don't recommend that people quit their studies or jobs on a whim, I certainly believe that if somebody doesn't feel they are following a life path or career path that leads to fulfilment, then changing course is critical, even if it is risky and frowned upon by others.

This encourages me to question the norms of my profession, and to try new approaches, even in the face of discouragement by others. My career passions are writing, software design and teaching. Yet a professor of computer science is boxed in by narrow criteria of how they should publish. A former Dean even told me that I should not be developing software since, that did not constitute research.

I have spent a lot of time on educational research, even though that is looked at by many others as a second-tier research topic.

Recently I have made a decision change the way I do research, even though this means fewer papers and may cost me future grants. I have a passion for what I believe is the future of software development – model-oriented programming. And I want to develop Umple as an example of this and a vehicle for exploring ideas in that domain. To do that I have to put personal effort into actually doing software engineering (including model-oriented programming itself), which takes time away from writing papers. I also have to require the same of my graduate students and have to be fastidious about software qualities such as usability, and other matters that many researchers would say should not be my concern.

So may Jobs rest in peace, and may he serve as an inspiration to help people like me who don't want to have to slavishly follow the norms of their profession and instead to follow their passions.

Surveys regarding CS and SE knowledge: Please participate

CIPS CBOK Survey: I am chair of the Body of Knowledge committee of CIPS; we are in the middle of conducting a survey to ask which topics should every computing and IT professional know, and at what level. This would include computer scientists, software engineers, business technology managers (BTM), project managers, data centre managers, database administrators, CIO's etc. If you are interested in participating in the CIPS CBOK survey, please click this link to access the survey. You may skip the questions on each page regarding references if you want to save time; if you do that, the remainder of the questions take about 35 minutes. The deadline is July 15th, 2011.

CS2013 Survey: Also the ACM and IEEE are looking to revise their recommendations on computer science education. They would be interested in having you fill out their survey of the characteristics that computer science graduates require.

SE2004 Review Task Force survey: Similarly, another ACM/IEEE group is reviewing the recommendations for software engineering education that were published in 2004. If you are interested in helping them assess the state of SE education, please fill out their survey.

Call for balance in computer science reviewing: Evaluation and ideas papers are both important

I was sparked to create this post by reading a tweet that led me to this Letter to PC Chairs. In it, a group of eminent computer scientists express how they want the field "to become a better empirical science by promoting both observation and reproduction studies."

I applaud this initiative. Much computer science research needs better empirical validation.

However I have some criticisms of the open letter as it stands. Firstly, the push to improve empirical validation has been around for my entire career, especially in software engineering. The cited call to action suggests that what they are seeking is something new.

My main criticism, however, is that I have seen the pendulum swing too far the other way. In some conferences it is now common to have papers with good ideas rejected because they do not have a rigorous empirical evaluation. This causes just as many problems.

From my perspective, we are rejecting far too many papers of all kinds in computer science conferences. If a paper has a rigorous evaluation, or is a well-executed replication of an existing study, it should be published. If a paper has good ideas with thoughtful analysis, it should also be published, even if it doesn't have much of an evaluation. And if a paper has a healthy mix of these, in other words, an incremental idea idea with a moderate evaluation, then it also deserves to be published.

We shouldn't be rejecting a paper purely because "it has insufficient evaluation" as long it contains an interesting or novel idea. And we shouldn't be rejecting papers that are "pure empirical studies" or "mere replications". We need to be balanced.

Many of the good computer science conferences have very low acceptance rates (between 12 and 30 percent) and remain fairly fixed in their attendance numbers over the years. In fact, there is often stagnation, where a high proportion of conference attendees are graduate students presenting papers. Their supervisors and many other people in the field don't go because they have nothing to present. This is not going to promote debate and development of the field.

I agree that papers should be rejected if they are badly written, have too much wrong reasoning, have bad statistical analysis, have ideas that are "half-baked", don't say anything new, etc. But  more papers with either decent ideas and/or empirical studies should be accepted, such that the conferences grow in size over time.

The authors of the "Letter to PC Chairs" point out that fields such as biology and medicine have a tradition of rigorous empirical evaluation. True. We can certainly learn from them. But there are also papers in these fields that are case studies, that express new ideas, or that simply describe a single sample of a new species, syndrome or medical procedure.

I have published many papers with empirical evaluation. I actually find that it is the "ideas" papers that are harder to get published. I would like to have discussion of the ideas at a conference before I embark on the years-long process of performing rigorous experiments.

I have probably been guilty of rejecting too many papers as a program committee member. My tendency is to want to be "fair and consistent" with how other papers will be treated, and if papers are being rejected for insufficient empirical evaluation, I tend to do the same. I have therefore contributed to some ideas papers being rejected that perhaps should have been accepted. I think they would have been accepted if there had been a carefully written set of crieria sent to all reviewers that encourages acceptance of a wide variety of types of paper.

A few years ago I co-authored the criteria for CSEE&T that describe the kinds of papers that should be acceptable in that conference. I actually think those criteria may have erred on the side of demanding too much rigorous empirical evaluation, at the expense of interesting, ideas-oriented papers.