Subject

Information Theory with Digital Communications

Course Title

Information Theory with Digital Communications
Information theory and digital communication

Code

F23L2S164

Study Programme

—

Organizer of the study programme (unit, institute, department or division)

Faculty of Computer Science and Engineering

Degree level (first, second, third cycle)

First Cycle

Academic year / semester

4 / Summer

Number of ECTS credits

Teacher

Alexandra Popovska Mitrović, Verica Bakeva

Prerequisites for enrolling in the course

Probability and Statistics or Mathematics 3 or Business and Statistics

10.

Objectives of the course programme (competences)

Students will be introduced to quantitative information theory and its application in reliable and efficient communication systems. They will also be introduced to the mathematical model of the communication system.

11.

Course content

1. Definition of random processes. Characteristics: mathematical expectation, first moment of order 1-1. Correlation function.
2. Properties of the correlation function. Types of stationarity of random processes.
3. Entropy of a random variable. Entropy of a random vector. Relative entropy. Information. Markov chains.
4. Properties of Entropy and Information.
5. Markov chains: definition, properties. stationarity.
6. The War on Entropy
7. Data Compression: code definition, non-singular code, prefix code, Knuth's inequality.
8. Optimal Codes. Huffman Code. Shannon-Fano-Elias Code. Arithmetic Codes.
9. Channel Capacity. Definition. Determination of the capacity of known channels.
10. Differential entropy. Gaussian channel.
11. Ideal decoding scheme for a binary symmetric channel.
12. Linear codes. Codes that detect and correct errors.

12.

Learning methods

Lectures supported by slide presentations, interactive lectures, practical classes (using equipment and software packages), teamwork, case studies, guest lecturers, independent preparation and defence of a project assignment and seminar paper, and learning in an electronic environment (forums and consultations).

13.

Total available time

6 ECTS x 30 hours = 180 hours

14.

Distribution of available time

30 + 45 + 15 + 15 + 75 = 180 hours

15.

Forms of teaching activities

15.1.	Lectures - theoretical instruction	30 hours
15.2.	Exercises (laboratory, auditory), seminars, teamwork	45 hours

16.

Other forms of activities

16.1.	Project assignments	15 hours
16.2.	Independent assignments	15 hours
16.3.	Home study	75 hours

17.

Assessment method

17.1.	Tests	0 points
17.2.	Seminar paper / project (presentation: written and oral)	15 points
17.3.	Activities and learning	10 points
17.4.	Final exam	90 points

18.

Grading criteria (points / grade)

up to 50 points	5 (five) (F)
from 51 to 60 points	6 (six) (E)
from 61 to 70 points	7 (seven) (D)
from 71 to 80 points	8 (eight) (C)
from 81 to 90 points	9 (nine) (B)
from 91 to 100 points	10 (ten) (A)

19.

Requirement for obtaining a signature and taking the final exam

15.1 and 15.2

20.

Language of instruction

Macedonian and English

21.

Method for monitoring the quality of teaching

internal evaluation and survey mechanism

22.

Literature

22.1.	Required literature 1. T.M. Cover \| Elements of Information Theory \| John Wiley & Sons, Inc. \| 1991 2. Ž. Pauše \| Introduction to Information Theory \| Školska knjiga, Zagreb 3. D.J.C. MacKay \| Information Theory, Inference, and Learning Algorithms \| Cambridge University Press \| 2003
22.2.	Additional literature No. Author Title Publisher Year