Text File

8-Oct-85 17:47:28-PDT,13893;000000000001 Mail-From: NEUMANN created at 8-Oct-85 17:45:26 Date: Tue 8 Oct 85 17:45:26-PDT From: RISKS FORUM (Peter G. Neumann, Coordinator) <RISKS@SRI-CSL.ARPA> Subject: RISKS-1.20 Sender: NEUMANN@SRI-CSL.ARPA To: RISKS-LIST@SRI-CSL.ARPA

RISKS-LIST: RISKS-FORUM Digest Wednesday, 9 Oct 1985 Volume 1 : Issue 20

FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS Peter G. Neumann, moderator

Contents: Risks using robots in industry (Bill Keefe) Re: Computer databases (Matt Bishop) Registrar's databases; Database risks - census data (Hal Murray, 2 messages) The winners of evolution... (William McKeeman)

Summary of Groundrules: The RISKS Forum is a moderated digest. To be distributed, submissions should be relevant to the topic, technically sound, objective, in good taste, and coherent. Others will be rejected. Diversity of viewpoints is welcome. Please try to avoid repetition of earlier discussions.

Warning: Dave Poole will be at it again tonight -- risk of multiple copies!

(Contributions to RISKS@SRI-CSL.ARPA, Requests to RISKS-Request@SRI-CSL.ARPA) (FTP Vol 1 : Issue n from SRI-CSL:<RISKS>RISKS-1.n)

----------------------------------------------------------------------

Date: Tuesday, 8 Oct 1985 07:26:03-PDT From: keefe%milrat.DEC@decwrl.ARPA (Bill Keefe) To: risks@sri-csl.ARPA, keefe%milrat.DEC@decwrl.ARPA Subject: Risks using robots in industry

I don't know who to credit with typing this in. I was going to summarize, but it's too easy to take some points out of context. It brings up many questions as to who bears the responsibility (liability?) to protect people from such occurrences.

--------------------------------

In The Lion's Cage" [Forbes Oct. 7, 1985]

On July 21, 1984, at about 1 p.m., a worker at Diecast Corp. in Jackson, Mich. found Harry Allen, 34, a diecast operator pinned between a factory pole and the back of an industrial robot. But Allen's co-worker couldn't come to his aid. Using the robot's controller, the company's director of manufacturing finally unpinned Allen, who was alive but in cardiac arrest. He died in a hospital five days later.

Allen had entered a restricted area, presumably to clean up scrap metal from the floor. While there, he got in the way of the robot's work, and thus became the first - and so far only - U.S. victim of an industrial robot-related accident.

That's not a bad safety record, considering that 17,000 robots are now installed in the U.S. But the bet is he won't be the last. The Japanese, who lead the world in robot installations, also lead in robot-related fatalities: There have been reports of at least 5, and possibly as many as 20, such deaths in Japan.

That's only fatalities. In this country, companies are not required to report injuries related to specific equipment, so no reliable data are available. But in Sweden, a pioneer in the use of industrial robots, one study estimates that there is 1 accident a year for every 45 robots. By 1990, when the number of robots installed in American Industry could climb as high as 90,000, the number of injuries could climb accordingly. That's because robots move quickly and are programmed to go through a series of motions without stopping. A worker who gets in the way can be struck, pushed aside, crushed or pinned to a pole as Allen was.

How will industry minimize the risk to its workers? Probably with difficulty. Robots don't easily accommodate safeguards. Whereas most machinery operates within a fixed set of boundaries, robots have a large "striking distance" - the reach of their mobile arms within three dimensions. In automotive assembly plants, maintenance workers often collide with robots adjacent to the ones they're servicing because they don't realize they are in another robot's work area. A robot may perform one task five times and then start on a completely different activity, and with it a different set of motions. Also, a robot can sit idly for a time and then come to alive again, threatening injury to a worker who mistakenly thought it was shut down.

What's being done to make robots safer? Right now, not much. "The extent of most safety precautions are signs saying, 'Restricted Area: Keep Out,' or maybe a guardrail," says Howard Gadberry of the Midwest Research Institute in Kansas City, Mo. Indeed, the most common safeguards - perimeter barriers such as guardrails and electric interlocked gates, which automatically shut down the robot when opened - don't protect those maintenance workers and programmers who must enter the lion's cage. Presence-sensing devices, such as pressure-sensitive mats and light curtains, both of which automatically cut off a robot's power, also don't seem to offer as much protection as is needed, if only because workers are even more unpredictable in their movements than robots. They may not step on the mat when feeding parts to a robot, or they may not break a light curtain's beam.

That's not to say that robots can't be made safer. Researchers at the Renssalaer Polytechnic Institute, for example, recently completed a research prototype for several large U.S. companies of a four-sensor safety system that continuously monitors the area around a robot. Using ultrasonic, infrared, capacitance and microwave sensors, the RPI system is designed to stop a robot in its tracks if a worker gets too close. Cost? Five thousand dollars in production, according to Jack Meagher, a senior project manager at RPI.

The National Bureau of Standards has also been working with ultrasonic sensors on robot arms similar to the system at RPI. They both have developed a secondary, or watchdog, computer to monitor the actions of the robot and its microprocessor. After all, if the robot's computer goes berserk, how can it monitor itself? That's more important than you might think, 30% of robot accidents seem to be caused by runaways, according to John Moran, director of research at the National Institute for Occupational Safety & Health.

While such systems slowly make the transition form research to the factory floor, industry is trying to put basic safety standards into practice. Recently, the Robotic Industries Association proposed a set of national safety standards for robots that could go into effect as early as next summer.

Would such standards have prevented Harry Allen's death? Maybe not. The robot at the Diecast plant was surrounded by a safety rail with an electric interlocked gate that automatically shut down the robot when the gate was opened. However, there were two gaps in the rail that allowed workers to easily bypass the safeguard; that has since been corrected by the company.

Says Allan Harvie, deputy director of the Michigan Department of Labor's bureau of safety and regulation, "I could only presume Harry Allen thought he could go in and do what he intended to do without having to shut the robot down."

------------------------------

Date: 8 Oct 1985 1123-PDT (Tuesday) From: Matt Bishop <mab@riacs.ARPA> Organization: Research Institute for Advanced Computer Science Address: Mail Stop 230-5, NASA Ames Research Center, Moffett Field, CA 94035 Phone: (415) 694-6363 [main office], (415) 694-6921 [my office] To: risks@sri-csl.ARPA Subject: Re: Computer databases

I guess I'll start the ball rolling on this discussion.

I think the greatest risk is not from the technological end but the human end. For instance, there was a case a couple of weeks back where someone got stopped for a traffic ticket. Call this gentleman John Lee Jones (I've forgotten his real name.) A routine computer check showed James Lee Jones was a fugitive from an LA warrent, and the description of James Lee Jones was pretty close to what John Lee Jones looked like. So the SFPD hauled him downtown, and ran a fingerprint check to see if there was anything else they could find out about John Lee Jones. Turned out he had used several aliases in the past -- so the SFPD notified the LAPD they had arrested James Lee Jones, and would the LAPD please come up and get him? The LAPD obliged, took him down to LA, and notified the prosecutors.

Throughout all this, Mr. Jones was (vehemently) denying he was James Lee Jones. About a week after he had first been locked up, his public defender persuaded the judge to order the police to compare John Lee Jones' fingerprints with James Lee Jones' fingerprints. They didn't match. End of case.

What's so surprising is that the people throughout the whole proceeding did not question whether the data the computer gave them was relevant. True, it was accurate (so far as I know.) But it was used incorrectly. In other words, in this case the technology didn't fail; the human safeguards did. (Incidentally, in defense of the police, when this came out an investigation was begun to see why the fingerprint comparison was not made immediately; according to police procedure it should have been.) And no amount of database security can guard against this type of breach of security.

[Caveat -- I read the newspaper story I outlined above a couple of weeks ago in the S.F. Chronicle. I have undoubtedly misremembered some of the details, but the thrust of it is correct.]

Matt Bishop

[Add that to the database-related cases of false arrest reported in RISKS-1.5. PGN]

------------------------------

Date: Tue, 8 Oct 85 06:59:25 PDT From: Murray.pa@Xerox.ARPA Subject: Registrar's databases To: RISKS@sri-csl.arpa

Just mentioning grades, computers and risks, all in the same paragraph instantly brings to my mind visions of hackers who are flunking freshman English smiling anyway, knowing that they have figured out how to get an A.

I've always assumed that everybody "knew" that students and grades couldn't really coexist on the same machine. Does anybody know of a school brave/silly enough to do it?

It seems like a great opportunity for somebody who makes a secure system to get a LOT of publicity, one way or the other. Has anybody ever been confident enough to try it? What happened?

Changing the topic slightly... Security on an ethernet is clearly non-existent unless you encrypt everything you care about. Our personnel people upstairs take the problem seriously. The solution is simple. They have their own section of coax. It's not even gatewayed to the rest of our network.

------------------------------

Date: Tue, 8 Oct 85 07:15:28 PDT From: Murray.pa@Xerox.ARPA Subject: Database risks - census data To: RISKS@sri-csl.arpa

The census bureau distributes their data broken down to quite small areas. I don't know the details, but I'm pretty sure it gets down to "neighborhood" sized regions, and it may even get down to the block. When the sample size gets small enough, there are obviously opportunities for gleaning non-statistical information by using carefully crafted querys to read between the lines.

I remember somebody telling me that they worked pretty hard to make sure this couldn't happen. Anybody know what they actually do? Is it written up someplace? Does it work well enough? Any war stories? Are the techniques simple once you know the trick? ....

[As I noted in RISKS-1.19, Dorothy Denning's book is a good source. The Census Bureau tries to add phony data that preserves all of the overall statistics but that prevents inferences... PGN]

------------------------------

Date: Tue 8 Oct 1985 10:00:15 EST From: William McKeeman <mckeeman%wang-inst.csnet@CSNET-RELAY.ARPA> Subject: The winners of evolution... To: risks@sri-csl

A recent submission included the following paragraphs on evolution of morals...

Now I think it fairly easy to see that the capacity to put group survival ahead of self-interest is an important genetic trait and that tribes of people that had this trait would be more likely to survive that tribes that didn't. That is not to say that this moral capacity doesn't vary greatly from one person to the next or that even that it may not be more fully realized in one person than another because of upbringing. It is even possible that, because of some genetic error, some people may be born without a moral capacity, just like they might be born without arms or legs.

Moral progress means the evolution of survival customs more appropriate to the current context. The trouble in recent centuries has been that our ability to evolve new technology has outstripped our capacity to evolve the appropriate morality for it. There is a strong tendency to stick to the morality that one learns as a child, even if it [is] not appropriate to the current situation.

Evolution is being used with its Darwinian meaning but with an interpretation that includes the more ordinary progress of mankind. The central mechanism of evolution is the failure of the less successful forms to reproduce -- often for failing to live long enough. Evolution is never fast enough to avoid bloodshed -- it is bloodshed that activates it. Until disaster strikes the adapted and unadapted survive undifferentiated.

My point is that if we treat the present sad state of affairs as a problem in evolution rather than politics or technology, we are implicitly planning on rebuilding a world with the (apparently) adapted survivors of WWIII.

W. M. McKeeman mckeeman@WangInst Wang Institute decvax!wanginst!mckeeman Tyngsboro MA 01879

------------------------------

End of RISKS-FORUM Digest ************************ -------