Information Theory – Chapter X Lempel-Ziv coding
This chaper is to talk about LZ78 and its asympotic optimaling.
Information Theory – Chapter 4: Asympotic Equipartition Property(AEP)
Informal statement of AEP: Let X_1,\cdots,X_n be i.i.d. Then almost all realizations (x_1, \cdots,x_…
Information Theory – Chapter 3: Important Inequality
Data Processing Inequality Let X\rightarrow Y\rightarrow Z form a Markov chain if P_{XYZ}(x,y,z)=P_X…
Information Theory – Chapter 1: Entropy
This chapter introduce the definition of entropy, joint entropy, conditional entropy, relative entro…