Information Theory and Big Data
Typical or non-typical, that is the question
Prof AJ Han Vinck
Summary of the Presentation
Big-data can be seen as the collection/generation, storage/communication, processing and interpretation of big volumes of data. The collection, storage, processing of data is part of the problems described in field of information theory. However, in the famous paper by Shannon, in 1948, “a mathematical theory of communications” the interpretation or semantics are Shannon states: semantic aspects of communication are irrelevant to the engineering problem. Furthermore, messages are to be selected from a set of possible messages. Shannon clearly tackled the communication problem from an engineering point of view. We will discuss several techniques from information theory that are expected to play an important role in future “big” data systems. Examples are memories, data compression and reduction, information retrieval and biometrics.