Text File

1-Jan-86 22:49:04-PST,10989;000000000001 Mail-From: NEUMANN created at 1-Jan-86 22:47:42 Date: Wed 1 Jan 86 22:47:42-PST From: RISKS FORUM (Peter G. Neumann, Coordinator) <RISKS@SRI-CSL.ARPA> Subject: RISKS-1.33 Sender: NEUMANN@SRI-CSL.ARPA To: RISKS-LIST@SRI-CSL.ARPA

RISKS-LIST: RISKS-FORUM Digest Wednesday, 1 Jan 1986 Volume 1 : Issue 33

FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS Peter G. Neumann, moderator

A VERY HAPPY AND LOWER-RISK NEW YEAR TO ALL OF YOU

Contents: Star Wars and Bank of NY (Brint Cooper, Chris Hibbert, Jim Horning) Lipton and SDI (Herb Lin) The robot sentry (Martin Minow) Murphy is watching YOU (Rob Austein) Re: Failure probabilities in decision chains (Stephen Wolff)

Summary of Groundrules: The RISKS Forum is a moderated digest. To be distributed, submissions should be relevant to the topic, technically sound, objective, in good taste, and coherent. Others will be rejected. Diversity of viewpoints is welcome. Please try to avoid repetition of earlier discussions.

(Contributions to RISKS@SRI-CSL.ARPA, Requests to RISKS-Request@SRI-CSL.ARPA) (FTP Vol 1 : Issue n from SRI-CSL:<RISKS>RISKS-1.n)

----------------------------------------------------------------------

Date: Mon, 23 Dec 85 17:38:18 EST From: Brint Cooper <abc@BRL.ARPA> To: horning@decwrl.dec.com cc: RISKS@sri-csl.arpa Subject: Re: Can Bank of New York Bank on Star Wars? [PGN's retitling]

The idea of independent, non-communicating "battle groups" for an SDI system sounds great. But what about the "fratricide" problem?

Brint

------------------------------

Date: Mon, 30 Dec 85 12:00:25 PST From: Hibbert.pa@Xerox.ARPA Subject: Re: Can Bank of New York Bank on Star Wars? [PGN's retitling] To: RISKS FORUM (Peter G. Neumann, Coordinator) <RISKS@SRI-CSL.ARPA> cc: horning@decwrl.DEC.COM (Jim Horning)

------------------------------------- From: horning@decwrl.DEC.COM (Jim Horning) Date: 20 Dec 1985 1413-PST (Friday) To: RISKS@SRI-CSL.ARPA Subject: Can Bank of New York Bank on Star Wars? [PGN's retitling]

Last night in the debate at Stanford on the technical feasibility of the SDI, Richard Lipton chose the financial network as an example of the advantages of a distributed system (such as he is proposing for SDI) over a centralized one. "There have been no catastrophes." [What about the ARPANET collapse? PGN] -------------------------------------

The ARPANET collapse is a good contrasting case to show what Lipton was talking about. His point about the financial "network" is that it isn't a monolithic system, but a set of many (dozens?, scores?, hundreds?) independant systems. Any one of the systems could fail (even catastrophically) and it wouldn't be much of a problem for the whole system. ARPANet is a single monolithic system, centrally designed and administered. Most bugs manifest at exactly the same provocations at widely separated parts of the net.

There are at least a couple of separate national networks of automatic teller machines, and if any one of them dies, it shouldn't have any effect on the others, or on any of the banks with only local networks, or no networks at all. It would take a collapse of the phone system to put them all out of commission.

ARPANet on the other hand is a monolithic system. There is one protocol that all parts of the system must share, a common medium is used, and in there are only a few implementations of the protocols. It doesn't take much to blow the whole system out of the water. (For the most part it's as reliable as it is is only because it gets constant use, and new parts aren't put in until they are shown to work most of the time.)

What Lipton was proposing at the Stanford debate was that we make an anti-missile shield from many separately designed and implemented parts so that their failure modes are more independant. This is a good idea, and if it were done, I would have plenty of faith in the system. However, that's not the way government gets things done. Since the DOD is running the program there's no way there would be more than three "separate" designs, and they would all go through the same approval process, removing many of the differences they started with.

Back to the ARPANet example, if you look at a larger system than just ARPA, including UUCP, DECNet, IBM's internal network, as well as the SOURCE, TYMNet, Compuserve, etc., you find the same robustness. ARPANet may die and be out of commission for a long time, and most people will still be able to get work done through some other medium, since only a fraction of the people using computer networks depend on any one of them.

Chris

------------------------------

From: horning@decwrl.DEC.COM (Jim Horning) Date: 30 Dec 1985 1419-PST (Monday) To: Hibbert.pa@Xerox.ARPA Cc: Neumann@sri-csl, horning@decwrl.DEC.COM (Jim Horning) Subject: Re: Can Bank of New York Bank on Star Wars? [PGN's retitling]

Chris,

I agree with many of your comments, and feel that the $38 billion problem at Bank of New York is much more typical of how problems in nominally "independent" systems can propagate because of the intrinsic need to communicate. (As an example of a non-obvious interaction, recall its effects on the platinum futures market.)

In addition to the problems you cite, Lipton's scheme suffers from a few other flaws, including:

- The "simulation" that indicates that "only 5-10% extra bullets" would be needed apparently makes two dubious assumptions: 1) Independent "battle groups" (with sufficient "teraflops") can pinpoint targets as accurately as a cooperating distributed system. 2) Each "battle group" is able to recognize all "kills" by any battle group. I.e., the "extra bullets" counted are only those that are fired simultaneously at a target. With many of the proposed weapons, targets would be disabled, rather than disintegrated; with kinetic weapons, a single target could disperse to form a threat crowd. (Note that observation of kills is a form of communication intrinsic to the problem.)

- There were good systems reasons (completely outside of the computing requirements) that led the Fletcher commision to propose cradle-to-grave tracking (especially for RV vs. decoy discrimination) and a layered defense. Lipton gave no evidence of understanding those reasons, let alone making credible alternate proposals.

- The systems that you cite, and that he cited, are all ones where each component is in routine use under the exact circumstances that they must be reliable for. No matter how many independent subsystems the Lipton SDI is divided into, NONE of them will get this kind of routine use under conditions of saturation attack where reliability will be most critical. Thus there is a high probability that each of them would fail (perhaps in independent ways!).

Jim H.

------------------------------

Date: Mon, 23 Dec 85 18:09:59 EST From: Herb Lin <LIN@MC.LCS.MIT.EDU> Subject: Lipton and SDI To: horning@DECWRL.DEC.COM cc: LIN@MC.LCS.MIT.EDU, RISKS@SRI-CSL.ARPA

From: horning at decwrl.DEC.COM (Jim Horning)

More generally, I am interested in reactions to Lipton's proposal that SDI reliability would be improved by having hundreds or thousands of "independent" orbiting "battle groups," with no communication between separate groups (to prevent failures from propagating), and separately designed and implemented hardware and software for each group (to prevent common design flaws from affecting multiple groups).

That is absurd on the face of it. To prevent propagation of failures, systems must be truly independent. To see the nonsense involved, assume layer #1 can kill 90% of the incoming threat, and layer #2 is sized to handle a maximum threat that is 10% of the originally launched threat. If layer 1 fails catastrophically, you're screwed in layer #2. Even if Layers 1 and 2 don't talk to each other, they're not truly independent.

------------------------------

Date: Friday, 27 Dec 1985 13:11:28-PST From: minow%rex.DEC@decwrl.DEC.COM (Martin Minow, DECtalk Engineering ML3-1/U47 223-9922) To: risks@sri-csla.ARPA Subject: The robot sentry

The following appeared on USENET net.general today (Dec 27). Martin.

--------

"A much more sinister arrival on the robot scene is named Prowler. Created by Robot Defense Systems in Colorado, Prowler has been designed for use as a sentry to guard military installations, warehouses and other sites where security is important. When made available in the near future, this squat, sturdy, mobile device will carry microcomputers, software and sensors capable of locating intruders. Chillingly, buyers will be able to arm Prowler with machine guns and grenade launchers; they'll also be able to program the robot to fire at will. The manufacturer claims that interest in Prowler has been high, both among domestic companies who see it as a comparatively low-cost replacement for 24-hour human security, and certain foreign countries where government officials might prefer guards that will never revolt."

-- US Air magazine

-- JP Massar, Thinking Machines Corporation, Cambridge, MA -- ihnp4!godot!massar, massar@think.com.arpa -- 617-876-1111

Posted Fri 27-Dec-1985 16:08 Maynard Time. Martin Minow MLO3-3/U8, DTN 223-9922

------------------------------

To: RHEA::DECWRL::"risks@sri-csla.arpa" Date: Mon, 23 Dec 1985 16:45 EST From: Rob Austein <SRA@XX.LCS.MIT.EDU> To: neumann@SRI-CSL.ARPA Subject: Murphy is watching YOU

About six hours after sending that message about mailers [RISKS-1.32], I found myself with the pleasant task of doing bit level reconstruction of XX's MAILQ: directory with DDT, because the system had crashed while MMAILR was in the middle of a disk transfer. Talk about ironic postscripts....

Cheers, Rob

------------------------------

Date: Mon, 23 Dec 85 17:33:15 EST From: Stephen Wolff <steve@BRL.ARPA> To: wmartin@almsa-1.ARPA cc: risks@sri-csl.ARPA Subject: Re: Failure probabilities in decision chains

* IF the overall decision is correct if and only if all five sub-decisions are correct, and * IF the sub-decisions are statistically independent, and * IF the probability that each sub-decision is correct is 0.9, * THEN the probability that the overall decission is correct is 0.9^5 = .59049 (vide any textbook in probability)

which is *suspiciously* close to "59%". But when Bill Walsh mentioned this problem to me in LA he was adamant that this was NOT the explanation he wanted. -s

------------------------------

End of RISKS-FORUM Digest ************************ -------