Subject

Foundations of Information Theory

Course Title

Foundations of Information Theory
Foundations of Information Theory

Code

F23L2W067

Study Programme

Internet, networks and security

Organizer of the study programme (unit, institute, department or division)

Faculty of Computer Science and Engineering

Degree level (first, second, third cycle)

First Cycle

Academic year / semester

3 / Winter

Number of ECTS credits

Teacher

Alexandra Popovska Mitrović, Verica Bakeva

Prerequisites for enrolling in the course

Calculus 1 or Mathematics 1

10.

Objectives of the course programme (competences)

Students will be introduced to the fundamentals of probability theory, and then to the basic concepts of information theory and its application in real communication systems.

11.

Course content

(1) Elements of combinatorics. Probability of random events. Properties of probabilities. (1) Discrete probability space. Classical definition. Conditional probability. Bayes' rule. (1) Independence of random events. Bernoulli scheme.
(2) Discrete and continuous distributions. (1) Random vectors: marginal and conditional distributions. Functions of random variables. (1) Numerical characteristics of random variables: mathematical expectation, variance of a random variable, correlation coefficient between two random variables. Central limit theorem. (2) Entropy and information
(3) Data compression: optimal prefix codes. Huffman algorithm. Shannon–Fano–Elias algorithm. Arithmetic codes. (1) Communication channel.

12.

Learning methods

Lectures using presentations, interactive lectures, exercises (using equipment and software packages), teamwork, case studies, guest lectures, independent preparation and defense of a project assignment and a seminar paper.

13.

Total available time

6 ECTS x 30 hours = 180 hours

14.

Distribution of available time

30 + 45 + 15 + 15 + 75 = 180 hours

15.

Forms of teaching activities

15.1.	Lectures - theoretical instruction	30 hours
15.2.	Exercises (laboratory, auditory), seminars, teamwork	45 hours

16.

Other forms of activities

16.1.	Project assignments	15 hours
16.2.	Independent assignments	15 hours
16.3.	Home study	75 hours

17.

Assessment method

17.1.	Tests	0 points
17.2.	Seminar paper / project (presentation: written and oral)	15 points
17.3.	Activities and learning	10 points
17.4.	Final exam	90 points

18.

Grading criteria (points / grade)

up to 50 points	5 (five) (F)
from 51 to 60 points	6 (six) (E)
from 61 to 70 points	7 (seven) (D)
from 71 to 80 points	8 (eight) (C)
from 81 to 90 points	9 (nine) (B)
from 91 to 100 points	10 (ten) (A)

19.

Requirement for obtaining a signature and taking the final exam

Activities carried out

20.

Language of instruction

Macedonian and English

21.

Method for monitoring the quality of teaching

internal evaluation and survey mechanism

22.

Literature

22.1.	Required literature 1. Veritsa Bakeva \| Probability \| UKIМ \| 2015 2. Thomas M. Cover, Joy A. Thomas \| Elements of Information Theory \| John Wiley & Sons \| 2006 3. D.J.C. MacKay \| Information Theory, Inference, and Learning Algorithms \| Cambridge University Press \| 2003
22.2.	Additional literature No. Author Title Publisher Year