Information theory

Postgraduate course

Course description

Objectives and Content

What is information? The course deals with measures of information content of digital symbols and sequences of symbols. Based on Shannon's information theoretical framework, an introduction to the fundamental principles behind data compression, error-correcting codes, and cryptography is given. In the course we also consider how information theory can be applied to computer networks, and we study information-theoretic wiretap channels in order to introduce the connection between security and information theory. We also investigate applications to gambling and horse betting.

Learning Outcomes


The student should be able to:

  • Explain the concepts of entropy and mutual information.
  • Understand how information can be represented in an efficient manner.
  • Explain how information theory can be applied to multiple-access channels, broadcast channels, and to general networks.
  • Explain information-theoretic aspects of information security.



The student should be able to:

  • Determine the channel capacity of a discrete memoryless point-to-point channel with given parameters.
  • Implement source coding algorithms for simple sources and sources with memory, and other algorithms covered by the course.

General competence

The student

  • has a good understanding of the field of information theory.

ECTS Credits


Level of Study


Semester of Instruction

Required Previous Knowledge
For incoming exchange students: At least 60 ECTS in Computer Science and at least 10 ECTS in mathematics
Recommended Previous Knowledge

INF100 or equivalent, MAT111 or equivalent, MNF130, STAT110, INF240 are highly recommended. In addition are INF101, INF142, INF240 recommended.

(For autumn 2020, the teaching of the course will be adapted to students who do not have STAT110)

Credit Reduction due to Course Overlap
I144, INF144: 10 ECTS
Access to the Course
Access to the course requires admission to a programme of study at The Faculty of Mathematics and Natural Sciences
Teaching and learning methods

Lectures 4 hours/week for 13 weeks.

Exercises 2 hours/week

Compulsory Assignments and Attendance
Compulsory assignments are valid for one subsequent semester .
Forms of Assessment

The forms of assessment are:

Written examination or Digital written examination (3 hours). Mandatory assignments may be graded and included in the final grade.

Grading Scale
The grading scale used is A to F. Grade A is the highest passing grade in the grading scale, grade F is a fail.
Assessment Semester
Examination both spring semester and autumn semester. In semesters without teaching the examination will be arranged at the beginning of the semester.
Reading List
The reading list will be available within July 1st for the autumn semester and December 1st for the spring semester.
Course Evaluation
The course will be evaluated by the students in accordance with the quality assurance system at UiB and the department.
Examination Support Material
Non-programmable calculator, according to the faculty regulations
Programme Committee
The Programme Committee is responsible for the content, structure and quality of the study programme and courses.
Course Coordinator
Course coordinator and administrative contact person can be found on Mitt UiB, or contact Student adviser
Course Administrator
The Faculty of Mathematics and Natural Sciences represented by the Department of Informatics is the course administrator for the course and study programme.