Sunday, April 5, 2020

Active Covid-19 cases visualized on a map per million - update April 4th with USA and Canada

On March 17 and March 22 I showed charts of the number of active cases of Covid-19 around the world, colour-coded to make trends more visible. Here is an update as of April 4th.

The purpose of these posts is to present information that I haven't seen elsewhere. Active cases indicate how badly each country is currently experiencing the disease. Most existing charts show either total cases or total cases per capita, whereas active cases per capita I think is more useful. The choice of colours is designed to differentiate countries with different experinces. Clearly the data has weaknesses due to delayed reporting and differences in testing rates, but I think the information is still useful.

The first chart uses the same colour-coding I used in the previous posts, where my top colour was red, indicating 100 or more active cases per million.



The above is still good at showing which countries are experiencing the pandemic severely (now including Canada). But many countries have gone well beyond 100 active cases per million since my earlier charts, so I have created a new colour scheme below to show the more extreme cases. Red still anchors the 100 cases per million, but now I have added light blue to show the most extreme activity rates (Italy, Iceland, Spain), and purple to show the bad, but near-extreme rates (USA, Germany, France, Portugal, Ireland and so on). I have stopped using green for the very lowest cases because it looked like 'all clear'.



Here is a detailed breakdown of Canada and the USA using the above scale (active cases per million)



This shows New York in the extreme category, as has been widely reported in the news. But it also shows New Jersey, Connecticut, Massachusetts,  Michigan and Louisiana in that category. It shows Quebec as having the biggest problems in Canada, being in a near-extreme situation. But it also shows several US States in a near-extreme situation.

Data source for Canada was CTV news. Data source for world and USA was Worldometer. As before, these were created using Microsoft Excel's built-in mapping capability.


Tuesday, March 31, 2020

Don't cancel summer plans except foreign travel: Thoughts about when we might be able to restart the economy.

A lot of people have been thinking about the massive wave of Covid-19 to come in the next month in North America. It is sure to be as horrific as it currently is in Italy and Spain in the USA (New York is already there).

But it is not too early to start thinking about the longer term and the decisions we would need to take to start moving back towards 'normal'.

Note: I have read a lot and know how to do data analysis. But I do not have epidemiological or medical expertise so what I say below should just be considered food for thought.

I am not thinking in Trumpian terms here and putting the economy ahead of lives. But there is, I think a risk of over-reaction. It may be that we need to socially distance until September, but we hopefully do not. And I think we need to prepare to push on the accelerator at the earliest time when epidemiologists give the go-ahead to avoid massive economic and social destruction, and excess deaths due to the fight against Covid-19 (rather than due to the disease itself).

I am reading about organizations cancelling activities in the summer and even well into the fall. My thought is DON'T CANCEL TOO SOON unless it involves overseas travel. I have, for example been urging my own faculty at the University of Ottawa not to cancel summer camps. They may eventually need to be cancelled, but we should wait to the absolute latest possible date before making that decision (even if this puts extreme pressure on the teams organizing them). The argument given for cancelling summer activities now is that it takes 'planning' time to ramp up, hire people and train them. This is reasonable. But it would be much better cancel on short notice (despite the pain of that), than to pre-emptively cancel. I have suggested only cancelling summer camps 30 days before they are due to start (so July ones would be cancelled progressively during June if necessary). If we and other organizations can pull off even half a summer of camps, it will allow parents to get back to work and kids to have less stressful lives. And the employment of staff would not be lost to the economy.

I have heard of whole summer theatre seasons being cancelled, even for August (see https://www.osfashland.org/en/tickets-and-calendar/covid-prep.aspx). Yes, it takes months to ramp up for a large production, but all of these artists are going to be potentially destitute as a result. It might have been better to suspend and then pick up after a 70-day hiatus.

For activities involving international travel, such as conferences, it does seem reasonable to make no plans within a 4-to-5 month time horizon, since nobody will want to commit to flying until some certainty returns, too many people have lost money on travel, and insurance won't cover cancellation any more. Besides, the airlines are going bankrupt. This, of course, poses incredible risks for the September semester at Universities that have a lot of foreign students, or even students from other provinces and states. Hopefully epidemiologists will be able to give good advice near the end of May so overseas travel can resume to some extent in September.

The following scientific paper by Glea et al, entitled "Estimated Deaths Attributable to Social Factors in the United States". Discusses death rates in the US in NORMAL times:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3134519/

The paper calculates that in the US the death rate among people with low social support has been calculated at about 33% higher than among people with adequate social support), with about 21% of the US population aged 25-65 suffering from this. That means that about 65000 extra deaths arise from this cause in this age group each year. Another excess 97,000 deaths are caused by excessively low social support among seniors. So the total amounts to about 160,000 excess deaths in normal times. The paper also shows 130,000 excess deaths among people in poverty, compared with what would be the case if poverty could be eliminated. Clearly many people in poverty also have low social support, so we can't add these figures, but it seems fair to say that these social factors account for something like 200,000 excess deaths a year in the US.

If the poor social support rate or poverty rate go up substantially, say by 33% due to massive job destruction and other factors flowing from how we handle the Covid-19 crisis, that perhaps might cause many tens of thousands of excess deaths.

Total Covid-19 deaths in the US are projected to be something between 36,000 and 150,000 according to this site: https://covid19.healthdata.org/projections
I think we have to look at these figures and realize that after the peak passes, we ought to put our feet on the economic accelerator.

The above figures are for the US, but divide by 10 for Canada perhaps (or maybe by a somewhat larger number since we have a better health care system, better political system, and better social safety nets).

So what policy decisions can be made so that the sum total of deaths from both Covid-19 and the fight against it is minimized. We obviously need to keep up strict social distancing until the peak is well past. But then what?

The longer people are not paid and business cannot function, the greater the destruction of the economy. A year of this and I suspect we would be in a deep depression that would take a decade or more to dig out of. Many societal systems would collapse. Millions of people would die due to causes stemming from poverty and other social issues. Even people in countries with socialized medicine and other safety nets would be massively affected. So a year is out of the question, no matter how bad the epidemic. That leaves three possibilities:

Option 1: Get people back to work and kids back in daycare and school at the earliest point that would avoid overwhelming the health care system, perhaps in the last week of May if current peak projections are accurate. This would likely cause more Covid-19 to spread than in the other options, and some people would be extremely angry when they suffer from Covid-19 after this date or have their relatives die. Many people would have lost two months of income, but we might get away just with a severe recession. Government bailouts might be able to save the majority of businesses, so most people would get their jobs back. On the other hand there might be another wave of social distancing needed for a month or so later in the year or in 2021.

Option 2: Wait a bit longer until case numbers dwindle much lower, perhaps near the end of June when active Covid-19 case rates might go back down to where they were in early March in North America. Kids wouldn't be back in school this academic year, but summer camps could be run so people could get back to work, and some forms of low-travel tourism could resume (camping, local attractions) in the summer. There would likely be some outbreaks here and there and a steady flow of Covid-19 cases everywhere, but many businesses may be able to pull themselves back from the brink. But hardship would be very deep for many people. Even with government support, there would be a lot of economic destruction.

Option 3: Be very cautious, trying to completely eliminate nearly all Covid-19 cases, maintaining most forms of social distancing until late in the summer. I think this would be a disaster. A high percentage of companies, large and small would be bankrupt, with impacts lasting many years. We would be in a depression, not a recession. There would be extensive social unrest and mass migrations; many governments would be insolvent too. Real estate prices would crash. Most people's life savings would deeply depleted if not entirely lost.

For each option, the exact dates would depend on epidemiology, and the possibility of being saved by anti-viral drugs is also something we could hope for. In all three cases, social distancing for the elderly and other vulnerable groups would need to continue for longer.

My thinking is this: Governments should plan for Option 1. i.e. release social distancing and shutdown limitations after about 65 days (or when we get back to the level of infections seen in early March in North America) as is happening in China. Hopefully by this time massive-scale testing will be working well so we can track remaining cases, and also maybe start testing to find out who is immune due to having silently had Covid-19. If this happens in late May, that means that people can ramp up for normal economic activity for the summer months (minus international tourists).

Bottom line: Keep summer plans on hold without cancelling (other than foreign travel) at least until early May, when the situation might become clearer.

Sunday, March 22, 2020

Maps of Covid-19 cases - March 22, 2020

On March 17, I published a visualization of Covid-19 cases per million people. We are now 5 days later, so below is an update using the same scale. This map shows the total number of cases accumulated over time, per million. One can now see that the US situation has rapidly worsened, and is similar to Europe (red) having surpassed China.

Australia, Canada and a few South American countries are now in orange, meaning that they are catching up.

Grey cells are those lacking data (e.g. North Korea).




This is not 'new news'. The importance of this visualization is that it shows trends that are less visible in other visualizations. I have been criticized by some people for publishing data that is already well known. The point, however, is that I am not publishing raw data, or even surprising results, but rather making certain aspects of the data more visible.

I have also been criticized for generating visualizations from data that is inevitably inaccurate. In particular, the source data does not account for 'community spread' cases where no tests have been done. Countries that have been able to do more testing of potential cases will appear to have more cases. But this is the best data we have, and I think it is still useful to see these visualizations.

Next I would like to show a visualization of Active cases per million. This is designed to highlight the forward-looking danger to the population and the health care system. Active cases don't include the cases that have been resolved (by people recovering or dying), and suggests the potential for spread. I have used a different colour coding from the above so readers do not think it is the same type of data.

The very dark countries are seemingly out of control with 500 or more cases per million and, presumably, the likelihood of rampant spreading unless there is very strict quarantine and extensive testing to find all new cases. We see this particularly in Iceland (1574), Switzerland (856), Italy (770), Spain (529), and Norway (422) . Again, this is not new data; we know about Italy and Spain from the news, but the situation in the other black or very dark orange countries is not so widely reported (at least in the Canadian media).

Orange cases are at 100 cases per million, and include the US (117). These countries are in real danger.

The fainter the orange the more likely that, just perhaps, the country could keep this under control with decent isolation methods. Canada is at 38.

It is notable that China has now dropped far down on this scale (as has been well reported in the news). It is at 4. Not everybody is confident that the Chinese situation is accurate, but we can only do the best we can with reported data.



Finally, below is a visualization of Active cases per doctor. This shows the current pressure on the medical system. This is very similar to the above, but a few countries have more risk because of fewer doctors per capita.

Very dark countries have one case or more per doctor. They are in a desperate situation.

Bright red countries have 0.1 case per doctor. This includes Canada. Once again, the US is doing worse than Canada, although it has a way to go before catching up to many European Countries.



The pandemic data for these visualizations comes from the Worldometer website. The population data and number of doctors comes from the built-in Excel data sources, as compiled by Microsoft.

I will periodically update these charts so people can see the progression.

Last week I was super-busy working my Vice-Dean role, helping work on policies and processes for the conduct of final assessments to be undertaken by University of Ottawa students who are working remotely now.

Tuesday, March 17, 2020

Map of Covid-19 cases per million - March 17, 2020

The following chart shows the number of cases per million people in the various countries of world as of March 17th. (For followup showing the same visualization method, see my post on March 22)



This shows the number of cases since the first outbreak (not current cases). It clearly shows that the EU is in fact doing much worse than China. It also shows the UK and US are the are the 'big' countries next in line, closely followed by Australia and Canada.

I will do more posts in the days to come showing how this is evolving and also showing current cases.

This map was generated as follows:

1. I took data of current cases from Bing https://www.bing.com/covid by simply cutting and pasting into Excel. Update: It turns out that Bing was getting most of its data from Wikipedia, which was in turn getting most of its data from here: https://www.worldometers.info/coronavirus/#countries. In future I will use the latter (although Worldometers does not give the 'cases per million' in its table for countries with less than a million).

2. I used Excel's Map charting capability to create the above map. Information about how to do that is here: https://support.office.com/en-us/article/format-a-map-chart-2c744937-a1cc-48f7-bc5e-776497343a29. Population of countries is generated in Excel directly from Bing sources.

Note that Diamond Princess data was attributed to Japan.

Below is a table of data for the countries with 10 or more per million.

CountryCases per millionCasesPopulation
San Marino3443.1111533,400
Vatican City1000.0011,000
Iceland616.16220357,050
Italy462.092798060,551,416
Switzerland323.8827428,466,017
Norway271.0114445,328,212
Spain242.061131246,733,038
Monaco232.59938,695
Luxembourg228.05140613,900
Iran199.221616981,162,788
Liechtenstein184.59737,922
Andorra183.781476,177
Denmark177.0310255,789,957
Estonia169.832251,324,820
Qatar166.344392,639,211
South Korea161.66832051,466,201
Bahrain158.792371,492,584
Austria151.2113328,809,212
Slovenia132.082732,066,880
Brunei130.6356428,697
Sweden115.60119110,302,984
Belgium107.94124311,515,793
Germany103.69860482,979,100
Netherlands99.59171117,181,084
France98.82663367,118,648
Malta81.6738465,292
Finland58.433225,511,303
China58.34808811,386,395,000
Singapore47.402665,612,253
Ireland46.332234,813,608
Portugal42.2644810,600,000
Cyprus39.00461,179,551
Czech Republic37.1839610,649,800
Israel36.443248,891,800
Greece35.9738710,760,421
Kuwait31.431304,136,528
Seychelles31.30395,843
Latvia31.09601,929,900
Maldives29.7913436,330
United Kingdom29.54195066,022,273
Armenia24.57722,930,450
Hong Kong21.651627,482,500
Malaysia21.2867331,624,264
Macau20.8813622,567
Lebanon19.671206,100,075
Albania19.14552,873,457
Australia18.3445224,641,662
Panama16.84694,098,587
United States16.005213325,719,178
Croatia15.83654,105,493
Slovakia15.44845,439,892
Republic of North Macedonia13.01272,075,301
Japan12.611574124,776,364
Canada12.2344936,708,083
Bulgaria11.45817,075,991
Romania11.3322219,586,539
Saint Lucia11.182178,844
Chile11.1320118,054,726
United Arab Emirates10.43989,400,145
Palestinian National Authority10.20414019000

Tuesday, September 20, 2016

Tasks that are keeping me busy as a professor this semester

Here are the tasks that are keeping me busy this semester (Sept-Dec 2016).

I am writing this  so that when people ask me if I wouldn't mind helping them out, I have somewhere to point them to when I say either 'sorry, no', or 'I will try to fit it in, but please don't expect fast response'. Members of the public often don't understand the heavy workload of a professor. In the following, I am sure I have forgotten some stuff. I probably shouldn't have 'wasted' time making this list, but I have done it to reduce my stress/guilt levels when I have to say 'no' or 'slow'!

Teaching and research: (It is not always easy to separate these areas, since graduate teaching and undergraduate project supervision blends into research)

  • Supervising ten 4th year University of Ottawa capstone software engineering projects. (each team has 2-5 students) Meetings many weeks with cohorts (sets of groups) to discuss progress. Meeting with individual groups as needed. Meetings with individual students when there are issues. Liaising with the 'customers' of the students. Constant monitoring of Github pages to ensure there is progress.
  • Preparing for the next cohort of capstone projects by helping find projects.
  • Co-mentoring, along with one of my PhD students, 4 undergraduates from other universities working on Umple as part of their 4th year capstone project through UCOSP. Meetings every week, along with time spent finding issues for them to work on, discussing design options,  reviewing design and code, as well as giving/reviewing formal feedback to them
  • Supervising a student in a directed studies course that is related to my research
  • Supervising 7 PhD students, 5 of whom are in the final thesis-writing stage (topics relate to Umple, user interface evaluation using machine learning and vision, and enterprise architecture). Includes finding and liaising with committees, guiding research, discussing research and design options, editing papers and theses, and so on. I meet almost every week with each student.
  • Supervising one masters student, in the thesis-writing stage.
  • Supervising/assisting 3 postdocs/visiting researchers (on topics of reverse engineering, software engineering education and deep learning for robotics)
  • Sitting on the committees of various students supervised by other professors (includes reading theses, preparing comprehensive exams,  etc.)
  • Travelling to present several papers that have been accepted at conferences (expected 2 weeks of travel this semester). This also includes attending sessions at these conferences, networking, and so on. This semester I am going to Models, Isola and hopefully Cascon.
  • Travelling to a meeting of a research consortium I am part of.
  • Planning travel and filling out paperwork before and after travel (sometimes it seems as though doing the paperwork can take as long as the travel).
  • Working on at least 7 scientific papers at various stages of preparation for journals and conferences, related to the above. Most papers involve collaboration of multiple grad students and/or external colleagues.
  • Investigating and working on one or more grant proposals
  • Responding to almost daily requests from potential future graduate students. These days I am saying 'no' until some of my existing students graduate, to lighten my load, and until I have new sources of funding.
  • Writing letters of reference for many former undergraduate and graduate students.
  • Actually conducting some of the research! This includes doing a certain amount of active work on Umple (e.g. fixing an issue or two) in order to maintain my personal software engineering skills
  • Responding to other researchers' requests about my research. I receive inquiries for help, requests for papers, and have to deal sometimes with people who make mistakes when writing about my research ... and I need to set the record straight.
  • Organizing meetings of my research group
  • Managing research infrastructure (servers etc.)
  • Keeping track of my research finances including setting up contracts for those graduate students that I pay. The finance system is quite hard to work with; I have to manage my own spreadsheets so I can be 'forward-looking' and reconcile these with the 'backward looking' university accounting system.
  • Filling out paperwork required by granting agencies regarding the progress of each research project.
  • Keeping up to date by reading literature, researching the latest software engineering techniques, etc.
  • Preparing for my graduate course in Software Usability to be taught next semester.
  • Applying for 'Ethics approval' for certain kinds of research, and reporting on ongoing projects. The forms are extremely complex, so this is an unduly time-consuming task.
  • Skimming/reading/replying to large numbers of emails relating to all of the above tasks
  • Keeping up-to-date my membership in the IEEE, ACM, CIPS, PEO etc.
  • Writing blog posts (it seems only about once a year now). Helping to raise public awareness.


Administration (I am Vice-Dean Governance)

  • Attending meetings of Faculty Executive and Faculty Council; helping to prepare agendas, preparing minutes, running special votes, and so on.
  • Attending Senate, Senate Executive, Senate Undergraduate Council (includes reading large volumes of material in preparation for these meetings).
  • Working on negotiations with the TA/RA Union. Multiple meetings most weeks.
  • Doing whatever other research is needed for the above roles, and any tasks assigned by the Dean (I am 'excluded' from the professor union so I can help with personnel tasks).
  • Applying for an academic leave  (sabbatical) next year, including writing a proposal, documenting progress, and so on. Hard deadline at end of September. I am overdue for this. The focus will be on Software Engineering Education.
  • Consulting with professors who are seeking advice (e.g. about tenure and promotion)
  • Assisting in preparation for accreditation at UOttawa (Computer Science and Software Engineering)
  • Participating in the Software Engineering curriculum committee.
  • Sitting on various ad-hoc committees (e.g. a committee on research IT infrastructure)
  • Other minor tasks: e.g. managing citation data for the faculty, liaising with other vice-deans, making active suggestions for improvements in various areas, such as faculty management
  • Skimming/reading/replying to large numbers of emails from the Dean, Executive, and so on
  • Attending certain 'social' events where my attendance is required/desirable (e.g. celebrations of retirements, awards ceremonies, announcements, welcoming of new students and staff, representing the Dean if he is not available)
  • Attending Council of the School of Electrical Engineering and Computer Science
  • Attending convocation ceremonies
  • Attending (if I can fit it in my schedule) certain training activities.


Service (beyond my formal role as Vice-Dean Governance)

  • Peer-reviewing for many journals and conferences (several papers to review formally every month). Publons lists my recent journal reviews, but not conference reviews.
  • Serving on the editorial team of SoSym Journal (finding reviewers, helping the editor in chief make decisions)
  • Serving as CIPS visitor and team lead to the Seoul Accord (I have a report to write that is overdue)
  • Serving as evaluator for tenure and promotion, or grant proposals, for professors from other universities
  • (perhaps) attending accreditation visits at other universities
  • Skimming/reading/replying to large numbers of emails relating to all of the above tasks
  • Responding periodically to requests from journalists for my expertise
  • Periodically helping out student groups (if I have the time)

Wednesday, February 4, 2015

Oil prices will continue on their wild swings for years to come

"Oil prices drop by half in just a few months". A headline from early 2015? No from 2008, as this article attests. Prices had been up in the $140's before that, and between 2008 and mid 2014 they went back up again well above $100.

So in fact, we are currently in the midst of a series of wild multi-year swings in the price of oil. This pattern of wild swings in the price of an important resource as it was being depleted was also experienced in the 1800's in the price of whale oil.

It's really pretty simple: High prices encourage investment in improved extraction (in the case of whales, better boats, better hunting techniques, longer voyages). After an important time lag these investments pay off in terms of increasing supplies; for a while the investment keeps pouring in because there are high prices and lots of product to sell. The resource producers do well, those dependent on the resource suffer from the high prices. 

But then a glut takes hold. Too much resource. In 2008 it was exacerbated by a recession that dropped demand, but whether or not there is a recession, the hyper-investment in supply will cause a glut, even in a dwindling resource. The temporary glut causes prices to start to fall, fast. It takes time to turn off the investment, there is still plenty of supply for a while, pushing the price way down. Some suppliers (with lowest production costs) are not to bothered by the situation as they know it will push other suppliers (with high production costs) out. It works. Suppliers shut down, Investment stalls. OPEC is right now pushing Russia and US out.

Then supply slows just as demand is soaring due to people loving the low prices. Smash ... the price soars again. But for some time there are fewer suppliers, so the price overshoots at the top end. Even now OPEC is predicting $200 oil in the not-too-distant future. There will be a lot fewer US wells producing by then, so OPEC will make a lot more money than they would have if this crash in prices had not happened. The price swoon in fact might jeopardize the dreams of the US being independent of OPEC supplies.

The last thing businesses and the modern economy need is price swings. Business can't plan; investments are too uncertain. But laissez-fair economics will guarantee swings of this nature. That is, unless a new disruptive technology takes hold. In the 1800 crude oil took over from whale oil, whose price eventually dropped off as it was no longer needed.

So there are two possible futures:

  1. Solar + nuclear + fusion + other technologies eventually save the day.
  2. We will be stuck with swinging oil prices. 

My guess, for the next few years is the latter. $200 oil in late 2016 or 2017? Another crash to $50 in the early 2020's; back unto $250 oil shortly afterwards?

Investors who can play long-term markets and can wait out these swings could make fortunes. But most futures plays are only for the short to medium term.

Theoretically governments could intervene in the market to smooth out prices. For example taxes on gasoline and other petroleum prices could be much, much higher when the crude price is low, with funds going into a trust to be saved for the next peak and to support investment in continued supply. When prices peak again, the taxes would drop, supporting a slow and steady rise in real prices paid by consumers and business. Unfortunately conservative thinkers currently in power would never stomach this.

Tuesday, April 15, 2014

Should you change all your passwords due to HeartBleed? I say no!

Huge number of companies and experts are saying 'change all your passwords' or 'change ours'.

Here's my take on it. I say don't blindly change your password on all sites:

A. Change your passwords on individual sites when all four of the following are true:
  • you used password-protected areas of the site between April 1, 2014 and the date the site announced it has patched the bug (or someone logged into using your account)
AND
  • the site reports it was vulnerable or was reported by others as vulnerable. See here for the status of some sites; consider a site vulnerable if you can't find out any information about its vulnerability and are worried because it contains sensitive data.
AND
  • The site contains information that could cause harm if it was exploited, or your password is similar to a password on another site that you would care about.
AND
  • the site does not use two-factor authentication (e.g. sending you a text containing a one time special code when you log in) or similar backup security mechanisms.

B. Also, change your passwords on other sites where:
  • you use the same or similar password to those you had to change in item A (but now try to make the passwords reasonably strong and different -- see my guidelines below).
OR
  • the site stores particularly risky information and recommends a change. This would apply to banks and taxation agencies that were affected, perhaps even if you haven't logged on for a longer time. Note that most banks report they were not affected. 

So I guess most people might end up changing 20% of their key passwords based on the above, but certainly not all of them. Why do I not say 'change all your passwords' to be safe? It is because there is significant risk and this is a classic 'lets overdo it' panic situation:

1. Some sites are just not affected. Many important sites like most banks, Apple, and Microsoft are just not vulnerable. Other sites have secondary mechanisms in place and have determined that users are safe.

2. There may be residual sites that still have the vulnerability; if you use one of these with your new password(s), then you are compromised when you weren't before.

3. The HeartBleed bug works by looking at transmitted data or data nearby where transmitted data is stored; if you or someone you know have not logged on (and your computer has not automatically logged you on) to a vulnerable site there is highly unlikely to have been data accessible to the bug that contains your password.

4. The password reset process itself has risks: Many people actually don't know many of their passwords, and rely on a tool to remember it for them, or have remained logged on essentially forever. In such cases, sites typically send a reset link; if a hacker truly wants to get you they may have ways to intercept that link, or generate fake links anticipating that people are in the middle or resetting their password. Some sites even send the original password back to you unencrypted, which is dreadful.

5. Many people now have hundreds of accounts, and several dozen they use regularly. It is essentially impossible to change all passwords and remember them all, so likely you will end up resetting passwords again in the future, or be forced to write then down or use an easily-guessed pattern. These add extra risk.

For unaffected and low impact sites (i.e. ones not dealing in financial and personal data) the risk of an attack on you is very small. In my Opinion, the risk posed (items 2, 4 and 5 above) by everyone changing their password, when multiplied by the low probability in most cases (items 1 and 3 above) outweighs the benefits of the blanked 'change all of them' advice.

For ongoing security with passwords. Here's what to do as a consumer:
  • Use passwords that are at least 6 characters, are not just letters or numbers; use special characters in passwords if the site allows.
  • For financial institutions, governments and other agencies processing sensitive information use completely distinct passwords from all others.
  • For other sites, make sure there are several characters of difference even if you follow a password pattern.
  • Only change your password based on my guidance at the top of this email, or if you think someone may have a  way to guess your password, or have specific reason to want to hack you.
  • Never click on a link that says to change a password unless you have requested such a link in the last few minutes. In other circumstances, go to the website by typing the URL or using a bookmark you have used before.
Here's what to do as a site administrator or programmer
  • Allow passwords to have any combination of letters, numbers and special characters and be of very long length. Don't restrict password content other than for minimal length, or requiring at least two of the above types of characters. So many people run into sites that have complicated rules (short password, no special characters, etc, that they have to make up a password they will inevitably forget).
  • Implement two-factor authentication if there is a high risk of compromised information.
  • If your site his risky information such as substantial personal or financial data, implement some other forms of extra security, such as challenge questions when a computer at a different IP address range is used, and gradually slowing-down of response as more and more password attempts are entered.
  • Don't block people from using password managers without good cause. Password managers likely result in a net increase of security. 
  • Put in place a robust reset process that uses multiple factors. Force people to phone if some of the factors are not present. Factors might include emailing their stored email address first, without a reset link initially, and verifying some other known personal information first.
  • Allow people to save multiple email addresses, so if people change service provider you still have a way to contact them to verify identity.
  • After a password is changed, email people at their email addresses of record, to alert them that the password has been changed.
  • Never put a link to any password-protected website in any email you sent to people; the only exception might be a link sent in a reset operation that follows the above guidelines, is sent instantly on request, and is only valid for a very short time.
  • Always think about usability as well as security; low usability of a security setup will force people to use simple passwords, write them down, or abandon your site.
Some other sites of interest include this and this. My opinion above contradicts these sites to some extent.

Monday, September 23, 2013

Just because fingerprints can be hacked doesn't make them useless in the iPhone 5S

As this article states, the fingerprint reader of the new iPhone 5S has been hacked by the Chaos Computer Club.

But does that mean Apple is "stupid" as they say, and that fingerprint authentication is unwise?

No, for the following reasons:

  • Right now, many people avoid using passcode locking because it is slow. This method will encourage them to lock their phones because it is faster to unlock them.
  • Passcode locking is almost certainly less secure than hackable-fingerprints due to the possibility of people looking over one's shoulder.
  • The average thief who decides to keep a lost phone they found or mugs someone and runs off with their phone generally won't have time to perform sophisticated fingerprint forging before the owner of the iPhone locks or wipes their device remotely.
  • It improves accessibility for the blind.

The lesson is that we should approach security from several directions. Avoid keeping critical information in plaintext on any computer or phone, protected by just one method. Use two-factor authentication, obfuscation, and passwords/passcodes in addition to fingerprints for such data. Also arrange for remote wiping in advance.

I have other suggestions for Apple (and others thinking of using this technology).

  1. Use geofencing. As an option, allow fingerprint-only access when in the home or other places that the phone recognizes it spends a lot of time; it could 'learn' the users workplace geographic coordinates, but require the passcode when elsewhere.
  2. Allow longer time intervals for passcode-required access. Currently the passcode can be required immediately, or after an interval has passed, with settings p to 15 minutes. The only other alternative is 'no passcode'. However, an interval of half an hour or an hour or even a day could be very useful too, to deter theft, especially in conjunction with geofencing and entry of an Apple ID for changing the passcode.
  3. Keep developing biometrics: Fingerprint recognition combined with facial recognition and/or voice recognition could double the difficulty of hacking. For example, with both fingerprint and facial recognition (both instant) a hacker couldn't just lift a fingerprint without also obtaining a photo of the user. That would require knowing whose phone it is. 

The idea is that someone reluctant to enter their passcode very often might be more willing if it was required only once in a while.