Text File

RISKS-LIST: RISKS-FORUM Digest Friday 10 November 1989 Volume 9 : Issue 40

FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Contents: "Computer Error" in Durham N.C. election results (J. Dean Brock, Ronnie W. Smith, John A. Board) Glitch in Virginia election totals (Paul Ammann) Rome: Operator error causes publication of wrong election results (Lorenzo Strigini) Delayed Stock Exchange Opening (Brian M. Clapper) Electronic Warfare Systems not working--Congress () Computer used to find scoflaws in Boston (Peter Jones) Computer errors and computer risks (Randall Davis) Equinox program on Airbus (Lindsay F. Marshall)

The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, and nonrepetitious. Diversity is welcome. CONTRIBUTIONS to RISKS@CSL.SRI.COM, with relevant, substantive "Subject:" line (otherwise they may be ignored). REQUESTS to RISKS-Request@CSL.SRI.COM. TO FTP VOL i ISSUE j: ftp CRVAX.sri.com<CR>login anonymous<CR>AnyNonNullPW<CR> cd sys:[risks]<CR>get risks-i.j . Vol summaries now in risks-i.0 (j=0)

----------------------------------------------------------------------

Date: Thu, 9 Nov 89 14:00:12 EST From: J. Dean Brock <brock@cs.unc.edu> Subject: "Computer Error" in Durham N.C. election results

The headline on the November 9, 1989, Durham Morning Herald is Computer Twists Election Results According to the article a "computer error" caused eight precincts to be counted twice. The correction actually changed the result of one city council race twelve hours after it was assumed settled.

It's difficult to determine the nature of this computer error from the newspaper article. Another front-page article entitled: "Haywire Machine Counted Precinct Vote Totals Twice" quotes Jo Overman, the chairman of the County Board of elections, as saying:

One terminal used Tuesday apparently counted twice each precinct entered into it.... What was called in was correct, the computer just added it twice.... It was not added by an operator, it was a glitch in the program.

Ms. Overman, also added that the "errant terminal was an extra unit put on election duty as part of a last-minute effort to process returns faster."

Interestingly, the precinct-by-precinct breakdown given to the media was correct, even though they did not match the totals. The mistake was discovered in a later hand check of the results by the Board of Elections. Apparently, no one else bothered to check the totals.

The director of the county's Management Information Services department, which would be responsible for any programming errors, was instructed by the elections supervisor not to say anything about the election.

------------------------------

Date: Fri, 10 Nov 89 08:55:29 EST From: Ronnie W. Smith <rws@cs.duke.edu> Subject: "Computer Error" in Durham N.C. election results

The only information I have to add is that the local TV media kept referring to it as a "computer error" without ever mentioning that the original source of the error was a person. The newspaper never explicitly made this link, but at least mentioned it was a programming error. Interestingly enough, the man who became the winner after he had been declared the loser did refer to it as a "human error". The number of votes that had been double added was slightly more than 6000. Ronnie

------------------------------

Date: Fri, 10 Nov 89 12:22:16 est From: John A. Board <jab@dukee.egr.duke.edu> Subject: "Computer Error" in Durham N.C. election results

[...] I find it most fascinating and troubling that it took over a day for anyone to notice that the correctly reported precinct votes duly tabulated in the paper the morning after the election did not add up to the numbers reported as totals at the bottom of the columns, and the errors were not small - the Mayor's race vote, for example, had been reported as 19,381 to 17,118 when in fact the real totals of the votes as listed were 16,136 to 13,356! To the credit of the elections board, the errors were apparently found during manual verification of the automatically reported "unofficial" results.

[With a "Duke" as Governor of both Massachusetts and California, I wonder if any Duke Univ. folks were governing this election? PGN]

------------------------------

Date: Thu, 9 Nov 89 14:25:22 -0500 From: pammann@gmuvax2.gmu.edu (Paul Ammann, George Mason University) Subject: Glitch in Virginia election totals

In the Nov. 7 Virginia gubernatorial race, Doug Wilder (D) appears to have defeated Marshall Coleman (R) in a close race. Currently out of a total of 1.7 million votes, AP reports a difference of 5,533 votes and UPI reports a 7,755 vote gap. The Post article referenced below discusses the reasons for the discrepancies and the mechanism for official vote tallies. Buried within the article was the following gem: (Washington Post, Thursday, Nov 9, 1989, pp. A37, A40.)

Vote Counting Methods, Race Factor in Polls Leave Plenty of Room For Error Disparities Remain in Va. Governor's Race Tallies By Stephen C. Fehr, Washington Post Staff Writer

[discussion of discrepancies between AP and UPI vote tallies] ... For an hour on Tuesday, [AP's director of planning Evans] Witt said, a computer glitch caused some of Wilder's votes in predominantly black precincts to be counted twice; the error was fixed and the vote total was adjusted. ... AP's Witt said that "there's almost always a variation between the official and the unofficial count," but said he could not think of an instance in which the results of an election had been reversed because of a mistake by the wire services. ...

Comment: There were many surprises and mistakes in the projections and reports of election results; the type of problem cited above is minor, but, left undiscovered, potentially quite serious. The decision to call for a recount, as well who bears the cost of a recount, depends upon the closeness of the election (according to the official tally, of course, which is due on the fourth Monday of November).

------------------------------

Date: Fri, 10 Nov 89 09:37:51 SET From: Lorenzo Strigini <STRIGINI@ICNUCEVM.CNUCE.CNR.IT> Subject: Rome: Operator error causes publication of wrong election results

On October 29-30, elections were held in Rome for a new city administration. Unofficial results published at first gave an important victory to the Christian Democrats, but at the end of the tallying this victory almost vanished. The publication of wrong results was attributed to a data-entry operator error. Since then, the political parties have been exchanging accusations of intentionally manipulating the data for political advantage (the supposed advantage would be a short-term boost in popularity for the Christian Democrats, or casting suspicions on the Christian Democrats, for the Communist). To set things in context: besides deciding who will manage the capital city of Italy, these elections were regarded as an important indicator for national policy, and the major parties had put much effort in a combative, venomous campaign.

Now the details. (Disclaimer: this is my interpretation, checked with a few colleagues, of very imprecise press and radio reports. I'll look for more precise reports, and send in corrections if I can)

Voting and vote counting are by hand, with paper ballots. After the count started, and as partial results were transmitted from the individual "electoral sections", an EDP center of the City of Rome added them to obtain partial accrued results (without official value) and transmitted them to the press, radio and TV.

Very soon in this process, the published results showed a marked gain for the Christian Democrats. Later, it turned out that a few tens of thousands of extra votes had been erroneously given to them. The error became evident because the sum of the votes was greater than the number of voters. In an interview, the director of the EDP center stated that he had received from the computer program warnings about the discrepancy, but had ordered the publication of results to continue, assuming the problem was temporary and it would disappear later on.

Two days ago, the operator was found that allegedly caused the problem. He had to type in a screenful of data, send them to the computer and wait for it to clear the screen and prompt for new data (or to unlock the keyboard?). He found that pressing a certain combination of keys allowed him to clear the screen and restart input sooner, so speeding up his work. But by this trick he sent wrong data ("this affected the votes for 4 parties, and in particular the number of votes for the Christian Democrats -line 18 on the screen - was substituted with the number of the electoral section"). The program would complain about receiving inconsistent data, but give him an override option, which he used.

Now my comments. Funny: everybody is complaining about evil intentions (of which there's no proof), not about incompetence. From the news stories, some technical/organizational flaws are evident:

- the input routines checked for transmission overruns, or the application program ran consistency checks on each individual transaction (the entering of the results from a given number of ballots) but allowed the operator to override them (there was a log of the override requests, though: all inputs were logged to tape; but the log of part of the session was lost because tapes were scarce, and some were used twice).

- the director of the EDP center ignored the warnings (it is unclear whether these were from a global auditing of the data base or were the same error messages sent to the operator) about inconsistent data.

But, most important: in their greed for early results, both the press and the politicians trusted a non-trustworthy system. It appears that the only checks applied to this unofficial counting procedure were the consistency checks mentioned. If one were to bribe the operators to shift votes _consistently_ from one party to another, this could go undetected until the official tally was available, several days later. The vulnerability so created is great: news reports of, say, an 80 % victory of the Communist Party would certainly hit the Stock Exchange hard; the resulting allegations of fraud would cause a political earthquake (in the '50s, they might well cause a civil war). As things are, the effect on the public appears quite serious: according to an opinion poll, some 30 % of the voters interviewed said that, if the election were held again after the news of the mix-up was known, they would refuse to vote.

Lorenzo Strigini Istituto di Elaborazione dell'Informazione, Pisa, Italy strigini@icnucevm.cnuce.cnr.it , strigini@icnucevm.bitnet IEI-CNR Via Santa Maria 46 I-56100 Pisa ITALY

[Regarding greed for early results, it was interesting to note that the advance polls in the New York City mayoral race were off by roughly 11%, and the exit polls were off by 10%. PGN]

------------------------------

Date: Fri, 10 Nov 89 11:53:05 EST From: Brian M. Clapper <bmc@SEI.CMU.EDU> Subject: Delayed Stock Exchange Opening

I received the following information from a friend of mine, William Power, who works as a reporter for the Wall Street Journal.

The New York Stock Exchange (NYSE) and the American Stock Exchange (AMEX) opened for trading approximately one hour late this morning (November 10) due to an inability to receive information from or transmit information to the Securities Industry Automation Corporation (SIAC), the jointly owned computer processing subsidiary of the two exchanges. SIAC suffered equipment damage due to a fire in its building at 55 Water Street in lower Manhattan. The fire apparently damaged equipment in a basement electrical vault, resulting in power outages to some areas of the building.

The initial fire alarm was posted at 8 am; the NYSE and the AMEX officially opened for trading at 10:30 am, one hour later than usual. The delayed opening resulted in a "domino effect," including the partial shutdown of the Chicago Mercantile Exchange.

Brian Clapper, Software Engineering Institute, Pittsburgh, PA 15213

------------------------------

Date: 8 Nov 89 14:19:36 GMT From: news@linus.mitre.org (USENET NEWS) Subject: Electronic Warfare Systems not working--Congress

The Nov. 7 issue of the *Washington Post* carries a front page article on the failure of long-term EW development projects to deliver on their goals and to adequately counter 20-year old threat techniques.

The article describes a Congressional study, to be published soon, that looked at the B-2 bomber, and a service-wide EW system, now in its thirteenth year of development. Of particular interest is the criticism of the test methods, described as not keeping up with teh technology to be tested.

Although the article doesn't mention software specifically, the B2 software has been a significant issue.

My own experience in EW systems is that black projects seem to engender the attitude that since the project is not as visible, we can get away with less formal control and more ad hoc technical approaches.

Disclaimer: the truer it is, the stronger the denial.

------------------------------

Date: Tue, 7 Nov 89 18:05:59 EST From: Peter Jones <MAINT@UQAM.bitnet> Subject: Computer used to find scoflaws in Boston

On Sun, 5 Nov 89 13:14:43 EST, "Barry C. Nelson" <bnelson@ccb.bbn.com>, in RISKS Volume 9 : Issue 39 said: > >When five out of six hits are human errors, imagine the complaints!

It goes to show the importance of considering the total effect of a system change, not just the project at hand. It was a serious design error to assume that licence numbers, even if they could be read accurately from a TV camera, could be used to positively identify wanted vehicles, if the database that indicates which numbers are "hot" is unreliable.

Peter Jones MAINT@UQAM (514)-987-3542 "Life's too short to try and fill up every minute of it" :-)

------------------------------

Date: Thu, 9 Nov 89 15:40:17 est From: davis@ai.mit.edu (Randall Davis) Subject: Computer errors and computer risks (e.g., RISKS-9.39)

Numerous stories have been reported on this list under the title "computer error" and "computer risk," that seem to me to have nothing essential to do with computers, and a great deal to do with very different issues.

Consider this story, for instance, from 9.39:

>Subject: new computer risk: child abuse data base proposed > According to a news release heard a day or two ago, MI is now considering >legislation permitting local communities to establish and maintain data bases >of "suspected" child abusers, or those meeting another of the nebulous >"profiles" used to identify all sorts of persons and ethnic groups in our >society. Aside from permitting hearsay from neighbors, teachers, co-workers, >associates and assorted third parties to be entered and disseminated, >the framers of this legislation are also attempting to gain back-door access >to medical records. One profile criteria disclosed for "identifying" child >abusers is use of multiple doctors/hospitals by the same family.... > >Obviously, the privacy considerations and potential for misuse and/or >malicious use, such as slanderous reports by neighbors against an unpopular >neighborhood resident, inherent in this legislation are enormous.

If this is essentially a computer risk, there is an easy solution: get rid of the computer and we get rid of the risk. Modify the legislation to require that all database records must be kept manually on paper. If this story is really about computer risk, then all the problems noted above will disappear when the source of risk is removed. But do they?

Of course not. Because the problems are privacy, vague definitions, hearsay, backdoor entry, and interference in our lives. The technology used to accomplish those things is of some consequence (typically it changes the economics), but it is not of the essence. The real problems existed long before this particular technology and are largely independent of it.

It matters how we describe these things because descriptions implicitly set the agenda for discussion. To call it a "computer risk" is to set an agenda for discussing computers. This is particularly misguided when the questions that ought to be asked are: Should we collect such information at all? HOW we collect and store it will eventually matter, but the first and fundamental question is, shall we do it at all? What rights to privacy do we have? What modifications are we willing to make to those rights in pursuit of other, clearing conflicting goals in society?

In Mass., for example, (and perhaps elsewhere) doctors are required to report to a state agency evidence of child abuse (not just obvious cases, evidence). This is clearly a risky violation of the privacy of the doctor/patient relation, one that includes most of the problems noted above. The risks are reduced here because the information required is a professional opinion based on physical evidence. In this case it is a risk we accept, presumably because we believe the tradeoff is worth it. And *that's* what the discussion ought to be about: the risks and benefits of what we are doing, not what technology is used. The risks and benefits are often magnified by the technology, but the essential question is the risks/benefits of various sorts of privacy and the character of the information collected, not the technology that happens to be employed.

Of all groups, this list ought to get this right. Let me thereby enter a plea to use the term "computer risk" and "computer error" with considerable technical discretion. I suggest the simple test above: Ask, can the identical problem can arise in the absence of computers?

In some cases the answer is no (eg, instant, large-scale access to data from arbitrary distances), and these are essential, computer-related risks.

But if the same problem can arise, it is quite likely the technology is fundamentally irrelevant and that the risk involves something else. In that circumstance ask what the problem is normally called and use that name. The story above, for example, is about risks to privacy, the dangers of using inaccurate information from questionable sources, and requiring people to report one another's activities. Removing the computer from the picture does not change those problems in any fundamental way. And the problems are serious enough that they ought to be debated on their own terms, without muddying the waters with technology.

------------------------------

Date: Thu, 9 Nov 89 12:50:03 BST From: "Lindsay F. Marshall" <Lindsay.Marshall@newcastle.ac.uk> Subject: Equinox program on Airbus

I managed to get round to watching the program last night and found it very interesting. The program was very smooth except for one sound glitch - which occured right in the middle of the word "reliability" when the narrator was discussing the multiple processor architecture...

Lindsay Marshall, Computing Laboratory, The University, Newcastle upon Tyne, UK NE1 7RU

------------------------------

End of RISKS-FORUM Digest 9.40 ************************