Full report

Original page 1

• . -, TECHNICALMEMORANDUMNO. 40 b . DISCUSSION OF TELEMETRY PERFORMANCE STANDARDS - (ACCESSION NUM (TURUI ( N A S A CR OR TMX O H A D NUMBER1 (CATEGORY) 2 - I ' Leonard S. Schwartz I I V Prepared for i NATIONAL AERONAUTICS AND SPACE ADMINISTRATION I GPO PRICE S GODDARD SPACE FUGHT CENTER CFSTl PRICE(S) S E 7 Hard copy (HC) ,// 80 Microfiche (MF) ff 653 July 65 1A BORA T 0 RY GREENBELT, MARYLAND CONTRACT NAS 53508 FOR ELECTROSCIENCE RESEARCH DEPARTMENT OF ELECTRICAL ENGINEERING SCHOOL OF ENGINEERING A N D SCIENCE N E W YORK UNIVERSITY N e w York 5 3 , N e w York

Original page 2

• . . e b c I In addition to published papers, the School of Engineering and Science reports the results of i t s research i n the form of reports to sponsors of research projects, Technical Reoorts. and Technical Notes. The latter are normally limited to distribution within the School. Information regarding the availability of reprints of journal articles and Technical Reports may be obtained by writing to the Director of the Research Division, School of Engineering and Science, New York University, New York 53, N.Y.

Original page 3

• • , TECHNICAL MEM0RAM)UM NO. 40 DISCUSSION OF TELEME;TRY PERFORE’lANCE STANDARDS Leonard S. Schwartz June 1964 Contract No. NAS 5-3508 Prepared bY NEW YORK UNIVERSITY SCHOOL OF EI?GINEERING AND SCIENCE DEPARTMEXT OF ELECTRICAL ENGIJ!IEZRING Laboratory for Electroscience Research University Heights N e w York, New York 10453 f o r NATIONAL AEBONAUTICS AND SPACE ADMINISTRATION GODDARD SPACE F L I G H T CENTER GREXNBELT, MARYLAND

Original page 4

ii ABSTRACT /4-992 Standards f o r operation of telemetry systems a r e presented. It is shown t h a t suitable performance standards a r e given i n terms of error and rejection probabilities. The trade-off between these proba b i l i t i e s with signal-to-noise ratio as parameter i s discussed, and a method of testing different equipments f o r a given policy o r of testing different policies on the same equipment i s outlined. Possible application of these ideas on the S - 3 vehicle data is examined. .

Original page 5

iii TABLE OF CONTENTS 1. INTRODUCTION 2. CRITERIA OF PERFORMANCE 3. TEST PROCEDUIW Page- 1 4 6 3.1 Comparison of Epipments for the Same Decision Policy 6 3.2 Comparison of Decision Policies on the Same Quipment 8

Original page 6

1 DISCUSSION OFY-T PERFORMANCE STANDART)ci 1. INTRODUCTION I n looking f o r standards of operation f o r telemetry o r cammuwe m u s t recognize two things: first, the nature of nication systems, the standards must be appropriate t o the operational performance required of the system, and second, having agreed on the standards, the specified performance indices a r e a t best subjective. I n any case, they can be assigned only by operational people. For example, i n tracking radar, operational performance is i n teras of the tracking error, which determines the killprobability, the quantity of ultimate interest. Should the k i l l probability be 50 percent, 75 percent o r 90 percent? The specified k i l l probability w i l l be the r e s u l t of an estimate of the damage that attacking planes o r missiles can i n f l i c t on friendly targets. If 25 percent of the attacking force is presumed t o penetrate the defenses, the estimates of possible i n f l i c t e d damage may be unacceptably higho It w i l l then be necessary t o attempt t o increase t h e kill probability, but notice that the subjective aspect enters i n t h e statement of what i s considered an unacceptable loss. In communication systems the operational performance is given i n terms of error and rejection probabilities, but the assigned values of these probabilities depend ultimately on t h e i r r e l a t i v e subjective t importance. For example, a maximum allowable error probability of one

Original page 7

2 percent, reflects the belief that an average of more than one error in a hundred has bad consequences. The subjective element here is evident A possible operational performance measure for communication or telemetry systems is information rate or per-unit equivocation. It w i l l be recalled that the error-free information rate I is given by I = H(x) - H(x~Y) where H(x) is the transmitted information in bits per symbol and is the per-symbol equivocation. A measure of the information- H(x1y) rate efficiency of a system is obtained by normalizing I with respect to the transmitted rate H(x). Thus, the normalized error-free rate is The ratio H(xly)/H(x) is the normalized equivocation, called the perunit equivocation and designated by the symbol E. Thus Hence, Inor is maximized by minimizing E, and either one of these parameters can serve as a performance measure when information rate is the criterion of performance. The objection to E and therefore Inor as a performance measure is that they are computed from the set of tra,nsition probabilities, all of which must be known before the calculations can be made, and their employment is strictly valid only for infinitely long messages

Original page 8

. 3 Question has arisen concerning the possibility of using the quantity Tt as a performance measure. Tt is the ratio of the total number of digits transmitted to the total number accepted. It is given by Tt = 1 1 - u where U is the rejection probability. Since Tt and U are functionally related, either parameter may be used as a performance measure; but since probability of error is clearly one measure that should be used, it is reasonable that both measures should be probabilities.

Original page 9

4 2. CRITERIA OF PERFORMANCE Error probability and rejection probability are regarded as reasonable performance measures, but signal-to-noiseratio is not. The reason is that signal-to-noiseratio is not the final objective of communications or telemetry. The final objective is performance with minimum error axld rejection probabilities, Actually, in practical cases we necessarily and, in fact, reasonably settle for less. The reasonable objective is performance with maximum acceptable rejection probability. Now this result is achieved error and minimum by m e a n s of a certain signal-to-noiseratio and a certain threshold setting or decision system. If we change the threshold setting or the decision system, for the same signal-to-noiseratio, the error probability, the rejection probability, or both will be different. For this reason, signal-to-noiseratio should not be regarded as a performance measure. It is, however, the means by which two systems may be compared. Thus, if two systems perform with the same error and rejection probabilities for different signal-to-noise Patios, the better system is the one requiring the smaller signal-to-noiseratio, As we may wish to specify different operational values of emor and rejection probabilities for different applications, a tradeoff curve between them be useful. Such a curve is obtained by w i l l adjusting the decision threshold for a given signal-to-noiseratio as parameter. setting of the decision threshold results in l o w A high error probability and high rejection probability; conversely, a low

Original page 10

5 setting of the decision threshold results in high error probability and low rejection probability. If the signal-to-noiseratio is changed, a new trade-off curve results. Hence, a family of curves is described for different signal-to-noise ratios, as sham in Fig. 1. The hatched rectangle is the area of acceptability, with dimensions determined by specified maximum allowable error and rejection probabilities. The U I l Rectangle of /Acceptability e Fig. 1 - Trade-Off Curves f o r Rejection Probability U Versus Error Probability Peo "he p ' s Represent Signal-to- Noise Ratios in Order of Decreasing Magnitude. corner point labelled P is the point of meximum acceptable error and rejection probabilities. It is a unique point and, therefore, one logically to be used in a performance test. adjusting the decision threshold and the signal-to-noiseratio, one can always reach the corner point

Original page 11

6 3. TEST PROCElluRES To devise meaningful test procedures, we must consider the goal of a data reduction system. It is to maximize the amount of useful data that can be recovered from the data storage tapes. Useful data are those data that meet some criterion of acceptability, specifically, some minimum reliability, which means that the error probability cannot exceed a specified maximum. According to this point of view the object in devising test procedures is twofold: (1)To devise test formats that will permit us to compare different equipments for a given decision policy and ( 2 ) To devise test formats that w i l l enable us to compare the effectiveness of different policies on a given equipment. Let us examine each of these objectives in turn. 3.1 Comparison of Equipments for the Same Decision Policy A basic step, common to all testing procedures, is to prepare simulated data tapes for a variety of signal-to-noiseratios, including one obtained for the noise-free condition, to be used as a reference for the others. In accordance with the signal-to-noise ratio, errors and rejections occur and the corresponding probabilities can be estimated by reference to the noise-free data. Before the testing procedure can properly begin, we must assume that the maximum acceptable values of the error and rejection probabilities have been specified. This defines the corner point P in Fig. 1, the point at which the test begins. Each equipment to be evaluated can

Original page 12

7 be brought to tine corner point by suitable adjustment of signal-tonoise ratio and decision threshold. Nar suspose that for each equipment we increase the signal-to-noiseratio in steps and at each step readjust the decision threshold to minimize the rejection probability, subject to the requirement that Pe 5 P-. With each step of increasing signal-to-noiseratio we increasingly penetrate the rectengle of acceptability, but ideally we would like to proceed along the edge of the rectangle fram the corner point P to P-. That this is the desired course is seen with the aid of a diagram such 8,s Fig. 2, illustrating an artificial satellite in a highly eccentric orbit about the earth. When the satellite is close to the earth, the signal-to-noiseratio is large; when it is at Fig. 2 maximum range, the signal-to-noiseratio is small. In comparing one data reduction system w i t h another, we would like to h a w which of the two is capable of reducing the larger percentage of data with specified reliability. Ideally, we would of course like to recover a l l the data I with specified reliability. This would be achieved most nearly by going along the edge of the rectangle of acceptability between P and P-.

Original page 13

, Hence, i n the light of Fig. 2, i f we were t o plot rejection probability U as a function of signal-to-noise ratio, the parameter of the curve would be a given equipment. The goal i s clearly t o select the equipment for which the area under the curve is less. & I 1 0 I I I I I I I I I I I I I I Fig. 3 - Rejection Probability Versus Signal-to-Noise Ratio f o r Given EQuipment 3 . 2 Comparison of Decision Policies on the Same Quipment performance measure of Fig. 1is useful not The trade-off only for comparing t h e performance of different systems with the same decision policy but also for evaluating the performance of a given system with different policies. Consider, f o r example, PFM channel supercammutation i n which a correct datum is not obtained unless a number of PFM channels are correctly received, as in the S - 3 data. In the experiment, gamm ray counts are recorded. When the number of counts goes up or remains the same on successive readings, the data are accepted as correct. When it goes down, the data are

Original page 14

9 considered i n error. When decisions are withheld because of internal are printed. The established decision policy threshold settings, 9's is t o reject the data from one or more channels on either side of a o r incorrect data points. This i s a decision policy series of 9's designed t o minimize the error probability; but as a consequence the rejection r a t e is likely t o be too high. In comparing different decision policies on the same equipment we proceed f o r each policy as we did above f o r each equipment, so that now the curve corresponding t o Fig. 3, would have a given policy as parameter, and the object would be t o choose the policy that would minimize the area under the curve. We close w i t h a word on the determination of error probabili t y . Emor probability is the weighted sum of the miss and false alarm probabilities. I n the established method of processing S - 3 data, the procedure of recognizing an error only when the count goes dawn is misleading because account is not taken of errors that as a result of noise. If a test may occur when the count increases tape of simulated data f r e e of noise is used as a reference, so that t h e error and rejection probabilities can be correctly estimated f o r varying threshold settings and signal-to-noise ratios, we cas establ i s h the correct family of trade-off curves f o r any rejection policy. The estimates of error a.nd rejection probabilities can be obtained from the observed data by means of formulae (as i n New York University Technical Memo No. 3l), knaring the sample size,

Original page 15

10 A f i a a l point is that the t e s t tape format of the kind just described will permit the effects of noise with the synchronizing waveform and w i t h the data bits t o be separately investigated, so that appropriate decision thresholds can be established f o r each separately.