Text File

9-Jan-86 23:48:56-PST,14544;000000000001 Mail-From: NEUMANN created at 9-Jan-86 23:46:33 Date: Thu 9 Jan 86 23:46:32-PST From: RISKS FORUM (Peter G. Neumann, Coordinator) <RISKS@SRI-CSL.ARPA> Subject: RISKS-1.38 Sender: NEUMANN@SRI-CSL.ARPA To: RISKS-LIST@SRI-CSL.ARPA

RISKS-LIST: RISKS-FORUM Digest, Friday, 10 Jan 1986 Volume 1 : Issue 38

FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS sponsored by the ACM, Peter G. Neumann, moderator Contents: Ad-hominem SDI discussion (Mike McLaughlin [and Peter Neumann]) Men in the loop (Martin J. Moore) Failure probabilities in decision chains (Jim Miller) [also in SOFT-ENG] Testing SDI (Karl Kluge, Robert Goldman) Summing Up on SDI (Jim McGrath) NOTE: More Lin-McGrath discussions were submitted to RISKS, but are not included here. They appear in ARMS-D, and are also available in SRI-CSL:<RISKS>RISKS-1.38x.

The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, nonrepetitious. Diversity is welcome. (Contributions to RISKS@SRI-CSL.ARPA, Requests to RISKS-Request@SRI-CSL.ARPA.) (Back issues Vol 1 Issue n stored in SRI-CSL:<RISKS>RISKS-1.n.)

----------------------------------------------------------------------

Date: Wed, 8 Jan 86 05:48:49 est From: mikemcl@nrl-csr (Mike McLaughlin) To: risks@sri-csl.ARPA Subject: Ad-hominem SDI discussion

I note with dismay the ad-hominem phraseology that is appearing in the SDI discussions. It is understandable but unacceptable. The issue is too important. I request that the participants review their inputs prior to submission, and edit out phraseology that is not relevant to the issue. It is not fair to expect the forum's moderator to be a censor. - Mike McLaughlin [Thanks. PGN]

[I have received various complaints about some of the recent SDI verbiage -- its quality, accuracy, relevance to RISKS, what right has RISKS to distribute SRI discussion when it should be in ARMS-D, etc. In general, I have to hope that our contributors exert some good sense. Some of the nit-picking should clearly be resolved privately. In some cases I might request two antagonists get together and write a single position statement to which they both agree -- but that seldom works.

So, at this point I would like to elevate the quality of SDI discussion -- but not to stifle meaningful discussion that is really RISKS-related. SDI remains one of the most important issues confronting us, and open <but informed> discussion of the computer risks therein is clearly vital.

This issue omits a bunch of further McGrath-Lin discussions, but I point out above they can be found in ARMS-D or in the <RISKS> directory. PGN]

----------------------------------------------------------------------

Date: 0 0 00:00:00 CDT From: "MARTIN J. MOORE" <mooremj@eglin-vax> Subject: Men in the loop To: "risks" <risks@sri-csl>

Talking about men in the loop for SDI:

> From: scott@rochester.arpa > I find it remarkable that this suggestion is taken seriously. Time scales > simply do not permit it.

There are at least several minutes in which to respond. While this is certainly too short for any chain-of-command processing, I believe (for reasons which are detailed below) that it is plenty of time for a trained individual, responsible for a single battlestation (or a jointly commanded small group of battlestations) to respond.

> As has been pointed out by previous contributors, human intervention during > boost phase is out of the question, since it's all over in a minute or two.

For SLBMs, or missiles aimed at Europe from the western Soviet Union, I'll grant the point. But for land-launched missiles aimed at the US, I estimate at least a five-minute boost. If this is wrong, someone please correct me!

> But even for later phases of missile flight, it is ridiculous to expect > competent, cool-headed behavior from human operators who 1) have presumably > been watching their consoles for years with no action, ...

No real action...but plenty of simulated action designed to train that operator to make the correct response almost automatically. For several years I had the privilege(?) of observing the training of Range Safety Officers at Cape Canaveral. These officers monitor the launch of a missile in real time, and if the missile endangers a populated area, they destroy it. Of course, if that missile is a Space Shuttle, they would either destroy the Shuttle and its flight crew...or they would fail to destroy it and possibly kill several thousand bystanders. I would not like to have that job! One of the RSOs is selected as the Training Officer on a rotating basis. He sets up launch simulations, realistic in every detail, to train his fellow officers. In addition to the missile going awry, various equipment failures, communication outages, etc., are thrown at the officers during some simulations; other simulations are completely nominal. Some of the simulations I've seen were much, much rougher than the worst actual launch. On the very rare occasions when a flight termination has been necessary, it has been handled cooly and correctly.

I'll save someone the trouble of pointing out that there has never yet been a termination of a manned launch. This is true. But I believe, based on my observations, that even in this case the Range Safety Officers would make the correct decision -- and make it fast.

> ... 2) have the fate of the world in their hands, ...

This will slow them down? I think it would make them most conscious of the need for timely action. If you are suggesting that the responsibility would be too great, and the operator would freeze up...I suppose it's possible, but I doubt such a person would be in the job in the first place.

> ... and 3) have only a few minutes in which to make their decisions.

Ten minutes is a long time. In the early stages of a missile launch from the Cape, the missile is perhaps ten seconds from impact in Cocoa Beach. There, the men in the loop have worked very well; when they've had to throw the switches, they've done it well before those seconds ticked away. No missiles in Cocoa Beach yet! (Lest someone mention the famous incident early in the Cape's history when a missile impacted in the Banana River just offshore from a trailer park -- that missile did not carry a destruct device...it was the last missile launched at the Cape not to carry one.)

Generally speaking, until the day when a computer system can be made 100% reliable (not just 99.999...9%), I strongly believe that *any* system capable of causing harm to human life must have a man in the loop. He may do no more than press a YES/NO switch when the computer selects an action, but he *must* be there. This transcends SDI or any other weapons system...it applies generally!

These views are strictly my own and do not necessarily agree with those of my employer or any of its customers.

Martin J. Moore mooremj@eglin-vax.arpa

----------------------------------------------------------------------

Date: Wed 8 Jan 86 15:37:24-CST From: Jim Miller <HI.JMILLER@MCC.ARPA> Subject: Re: Failure probabilities in decision chains To: wmartin@BRL.ARPA cc: Soft-Eng@MIT-XX.ARPA, Risks@SRI-CSL.ARPA, walsh@ALMSA-1.ARPA US-Mail: MCC, Human Interface Program, 9430 Research, Austin TX 78759 Phone: 512-834-3342

Well, if you assume that each decision has a 90% likelihood of being correct (or, a 10% chance of being wrong), and that all of the five decisions are independent of each other, and all that other statistical stuff, then the probability that all five decisions will be correct is .9^5, or .59. However, how you would go about determining in a real-life situation what the actual probabilities of those decisions being correct really are, or whether the decisions are really independent, is beyond me. This sounds like one of those statistics that people love to quote, regardless of whether it has any validity.

Jim Miller MCC Human Interface

------------------------------

Date: 7 Jan 1986 22:41-EST From: Karl.Kluge@G.CS.CMU.EDU To: Risks@sri-csl.arpa Subject: Testing: Differences between SDI and other systems. Message-Id: <505539701/kck@G.CS.CMU.EDU>

> Date: Mon 6 Jan 86 18:24:44-PST > From: Jim McGrath <J.JPM@LOTS-A> > > You are, of course, correct. The problem is that your points could > also be (and are) made about any complex weapons systems (or indeed, > any complex system at all). It is NEVER possible to fully test ANY > system until it is actually used in battle (and even then it can fail > in future battles).

1) The consequences of failure of an SDI system are orders of magnitude greater than the consequences of failure of a "normal" weapons system. We damn well should be orders of magnitude more confident in it.

2) Which is really irrelevant, since the function of the SDI is to enhance deterrence, not replace it. The SDI system doesn't have to work, it just has to create reasonable worry that it might work in the mind of a potent- ial attacker. This makes it different from most systems, which are built to be used. If the SDI system ever has to be used then it has failed, which means...

(* rest of message on ARMS-D. Karl *) [THANKS... PGN]

My opinion, of course, in no way reflects the opinion of anyone I'm associated with. Karl

------------------------------

Date: 8 Jan 86 (Wed) 15:26:18 EST From: Robert Goldman <rpg%brown.csnet@CSNET-RELAY.ARPA> To: risks@sri-csl.arpa Subject: Testing SDI

In RISKS-1.34 Jim McGrath discusses testing the SDI. A common objection to the SDI is that it could not realistically be tested.

McGrath correctly points out that this is true of any modern weapon system. I agree with this basic point, but of course, some weapons systems are easier to test than others. For example, Mr. McGrath points out

it took WWI for the Germans to realize that the lessons of the 1880's concerning rapid infantry fire (and thus the rise of infantry over calvary) did not take artillery development adequately into account This is correct, but dodges the point that at that time, modern artillery, rifles and machine guns HAD all been fired in anger. The almost universal inability to profit from the experience was due to institutional failures, rather than lack of raw data.

I need hardly point out that ICBMs and SLBMs have NOT been tested in wartime conditions, and that we have reason to believe that there won't be a second chance to correct any mistakes.

Mr. McGrath goes on to suggest that the SDI might be tested against meteor storms. I question this for three reasons: 1. As I understand it, the particle-beam weapons of the SDI are not intended to destroy warheads outright, but rather prevent them from reaching their targets and detonating. How would one judge that a meteorite's fusing and guidance mechanisms had been destroyed? 2. Meteorites are not human-made objects which are designed with an eye to penetrating enemy defenses: they do not drop chaff, employ electronic counter-measures, etc. 3. Meteorites have a wholly different flight pattern. As I understand it, ICBMs have a boost phase, a cruising phase and a re-entry phase. Doesn't a missile's detectability and vulnerability depend on which of these phases it is in? Robert Goldman

[I deleted a paragraph on what might happen "if the SDI accidentally (or on purpose) shot down a Soviet reconnaissance satellite?" PGN]

------------------------------

Date: Thu 9 Jan 86 22:45:57-PST From: Jim McGrath <J.JPM@Epic> Subject: Summing Up on SDI To: "risks@sri-csl"@Sushi Reply-to: mcgrath%mit-oz@mit-mc.arpa

[Reminder: I have omitted a bunch of McGrath-Lin messages. If you wish to read them, see aove. PGN]

From all these messages, I've come to two conclusions. First, that on the whole, I tend to feel that SDI is a more complex system (mainly because of its mission size) than existing ones, such as Aegis. But I could be wrong (either way). And I do not believe that it is "orders of magnitude" more complex than existing systems. Thus it would seem that it could be built to existing system performance standards.

The second question is whether these standards are "adequate," and if not can we improve upon them? It is clear that existing systems are not being tested as fully as possible (Herb Lin's report on the Aegis tests makes them appear to be a joke). I've already pointed out that substantial testing of mid-course and terminal phases, far more extensive testing than any of our other systems have received, can be carried out. Thus, provided we have a proper commitment, even our current testing technology can be better applied (with better results) than we have done up until now.

To a large extent the standard you require depends upon your definition of the mission of the system. Clearly the system was never designed to make OUR nuclear weapons obsolete (just the Soviet's). So under any mission we would probably retain a force sufficient to destroy the USSR, and so can always fall back on MAD. Any reasonable performance level would protect our weapons to a significant extent (and if not, you can always keep a sub force). So the real question is whether it can protect cities.

I would tend to doubt it, under a full and unimpeded Soviet attack. But there are many scenarios (from accidental launch to a limited (decapitation or counterforce) strike to a second strike (the first perhaps going to Europe and/or China)) where it quite possibly could.

In any event, I am certainly not sure enough to either commit to SDI deployment nor to terminate research. Since all SDI is at the moment is research, I have no problem with the existing program. Still, knowing how programs have a tendency to outlive their usefulness, I think strong scrutiny is appropriate. But not mindless opposition.

Jim

------------------------------

End of RISKS-FORUM Digest ************************ -------