Information Theory

Our fundamental (or “basic”) research is in the science of Information Theory, and specifically in the subfield of Coding Techniques, as defined by the IEEE Information Theory Society. Coding Techniques is somewhat more empirical and intuitive than the  mathematically rigorous field of Algebraic Coding Theory (sometimes implied with the term “Coding Theory”). Coding Techniques is in our opinion also the most creative subfield in Information Theory. On occasion it makes recourse to a wide and interesting spectrum of mathematical tools. Being an engineering group, the starting point of our research is often the development of our own special purpose software for mathematical modelling and coding investigations. In terms of Telecommunications, our research emphasizes Channel Coding. We are especially interested in combined codes, i.e. channel codes constructed to have several desirable attributes combined into one code construction. We work on both code constructions and decoding algorithms.

Information Theory is a very young science, which started with Claude E Shannon’s seminal 1948 paper “A Mathematical Theory of Communications”. In his paper Shannon created the term “coding” to describe the processes performed on information before transmission over a communications channel. It begins with the information source, whose output may have to be compressed, using source coding before transmission, in order to save bandwidth. At this point encryption for information security or secrecy may be employed. This is usually followed by error control coding in order to protect the information against the noise and perturbations on any practical channel. Some channels have input constraints, and at this point constrained coding may be applied. The encoded information is next transmitted via a communications channel which in practice always has a limited capacity and sometimes also input constraints. At the output of the channel is a receiver which performs decoding, i.e. the inverses of the encoding processes, and the original information is retrieved and passed on to the information destination, i.e. the user. Information theory had a profound impact on the modern digital era. It is fundamental to all modern communication systems, consumer electronics and computers. Source Coding has been employed for example in ZIP files, JPEG and MPEG compression, etc. Constrained Coding is implemented in all digital storage devices, including computer hard drives and floppy/stiffy drives, flash memories – in fact all magnetic recording devices, and also in optical storage devices such as CD’s, DVD’s and BluRay discs. Encryption is used for internet security, electronic banking, diplomatic and military security. Error control coding is applied to all of the above and in fact to all modern  telecommunications and information storage systems, including satellite communications, cell phones and wireless communications, computer modems and deep space exploration crafts.

Our current research topics include:

  • Information Theoretic Coding Techniques
  • Channel Coding, especially:
    • Combined error correcting/constrained coding,
    • Insertion/deletion error correction for synchronization errors,
    • Permutation codes,
    • Coding for spectral shaping,
    • Constrained codes, i.e. line codes or modulation codes.