Information theory and digital communication

1.

Subject title

Information theory and digital communication

Теорија на информации со дигитални комуникации

2.

Code

F23L2S164

3.

Study program

Примена на информациски технологии, Софтверско инженерство и информациски системи, Компјутерски науки, Примена на информациски технологии, Софтверско инженерство и информациски системи, Компјутерски науки, Компјутерско инженерство, Интернет, мрежи и безбедност, Software engineering and information systems, Компјутерско инженерство, Интернет, мрежи и безбедност, Информатичка едукација, Software engineering and information systems, Стручни студии за програмирање, Стручни студии за програмирање,

4.

Organizer of the study program (unit, institute, department, division)

Faculty of Information Sciences and Computer Engineering

5.

Study cycle (first, second, third)

Прв циклус

6.

Academic year / semester

2 / Летен

7. Number of ECTS credits

6.0

8.

Instructor

проф. д-р Александра Поповска Митровиќ проф. д-р Верица Бакева

9.

Prerequisites for enrollment

Веројатност и статистика или Математика 3 или Бизнис и статистика

10.

Subject goals and competencies:

Students will be introduced to quantitative information theory and its application in reliable and efficient communication systems. Additionally, they will be introduced to the mathematical model of the communication system.

11.

Subject content:

1. Definition of random processes. Properties: mathematical expectation, initial moment of order 1-1. Correlation function. 2. Properties of correlation function. Types of stationarity of random processes. 3. Entropy of a random variable. Entropy of a random vector. relative entropy. Information. Chain rules. 4. Properties of entropy and information. 5. Markov chains: definition, properties. stationarity. 6. Rate of entropy 7. Data compression: definition of code, non-singular code, code with prefix property, Kraft`s inequality. 8. Optimal codes. Huffman code. Shannon-Fano-Elias code. Arithmetic codes. 9. Channel capacity. Definition. Determination of capacity of known channels. 10. Differential entropy. Gaussian channel. 11. Ideal decoding scheme for a binary symmetric channel. 12. Linear codes. Codes that detect and correct errors.

12.

Learning methods:

Предавања поддржани со презентации преку слајдови, интерактивни предавања, вежби (користење на опрема и софтверски пакети), тимска работа, пример случаи, поканети гости предавачи, самостојна изработка и одбрана на проектна задача и семинарска работа, учење во електронско опкружување (форуми, консултации).

13.

Total available time fund

6.0 ECTS x 30 hours = 180 hours

14.

Time distribution

30 + 45 + 15 + 15 + 75 = 180 hours

15.

Forms of teaching activities

15.1.

Lectures - theoretical teaching

30 hours

15.2.

Exercises (laboratory, classroom), seminars, team work

45 hours

16.

Other forms of activities

16.1.

Project tasks

15 hours

16.2.

Independent tasks

15 hours

16.3.

Homework

75 hours

17.

Grading method

17.1.

Tests

0 points

17.2.

Seminar work / project (presentation: written and oral)

15 points

17.3.

Activities and learning

10 points

17.4.

Final exam

90 points

18.

Grading criteria (points / grade)

up to 50 points

5 (five) (F)

from 51 to 60 points

6 (six) (E)

from 61 to 70 points

7 (seven) (D)

from 71 to 80 points

8 (eight) (C)

from 81 to 90 points

9 (nine) (B)

from 91 to 100 points

10 (ten) (A)

19.

Condition for signature and taking final exam

15.1 и 15.2

20.

Language of instruction

македонски и англиски

21.

Quality assurance method

механизам на интерна евалуација и анкети

22.

Literature

22.1.

Mandatory literature

22.2.

Additional literature