✦ScienexySCIENCE KNOWLEDGE LIBRARY
reference · SCI-REF-006-100

Information Theory, Networks and Communication Symbols

Information theory, networking, and communication notation represents information content, uncertainty, data rates, coding, signals, network structures, protocols, transmission errors, and channel performance. It combines probability, logarithms, graph theory, signal notation, and communication-system parameters.

Self-information: I(x) = −log₂P(x) bits. Shannon entropy: H(X) = −∑ₓP(x)log₂P(x). Mutual information: I(X;Y) = H(X) − H(X|Y). Shannon capacity for an ideal band-limited additive white Gaussian noise channel: C = B log₂(1 + S/N) bits·s⁻¹, where B is bandwidth and S/N is the linear signal-to-noise ratio.

Explanation

Information theory provides mathematical tools for quantifying information, uncertainty, compression, and reliable communication. A random variable's information content is measured using logarithms of probability, while entropy describes the average uncertainty of a probability distribution. Joint entropy, conditional entropy, mutual information, and relative entropy describe relationships between random variables and probability models. Communication systems transmit information through channels that may introduce noise, distortion, delay, and errors. Channel capacity represents the theoretical maximum information rate under a specified channel model and assumptions. Coding theory introduces symbols and notation for source codes, channel codes, codewords, block lengths, code rates, and error-correction capability. Networking notation describes nodes, links, paths, topologies, addresses, ports, packets, frames, and protocol layers. A network may be modeled as a graph G = (V, E), where vertices represent devices or network entities and edges represent communication links. Network performance is described through bandwidth, throughput, latency, jitter, packet loss, and reliability. Signal notation includes amplitude, frequency, phase, wavelength, modulation, and signal-to-noise ratio. Digital communication uses bit rate, symbol rate, modulation order, and coding rate. A bit represents a binary information unit, while a symbol represents one signal state or waveform selected from a modulation alphabet. The number of bits represented by each symbol depends on the modulation scheme and coding assumptions. Communication standards define protocol behavior, framing, addressing, timing, and error detection. Symbols such as R, C, B, and SNR may have different meanings in different subfields, so their definitions and units should be stated explicitly.

Variables

SymbolMeaningUnit

I(x)

Self-information associated with outcome x.

bit when the logarithm is base 2.

H(X)

Entropy of random variable X.

bit when the logarithm is base 2.

H(X|Y)

Conditional entropy of X given Y.

bit.

I(X;Y)

Mutual information between X and Y.

bit.

P(x)

Probability of outcome x.

Dimensionless.

C

Channel capacity.

bit·s⁻¹.

B

Channel bandwidth.

Hz.

S

Signal power.

W.

N

Noise power over the specified bandwidth.

W.

S/N

Linear signal-to-noise ratio.

Dimensionless.

R_b

Bit rate.

bit·s⁻¹.

R_s

Symbol rate.

baud.

L

Packet or codeword length, according to context.

Bits or symbols.

p

Probability of bit or symbol error, according to context.

Dimensionless.

Example or application

A worked example has not yet been added.

Definition

Information theory, network, and communication notation is the mathematical and technical language used to describe information content, uncertainty, data encoding, transmission channels, network structure, protocol behavior, and communication performance.

Purpose

To quantify information and uncertainty, describe the limits of communication channels, represent encoded data, model network structures, and measure the performance and reliability of communication systems.

Importance

Information and communication notation supports data compression, error correction, digital communication, network design, protocol engineering, wireless systems, and performance analysis. It provides a mathematical basis for determining how efficiently information can be represented and transmitted despite noise and limited resources.

Information Theory Symbols

SymbolMeaningDescription

H(X)

Entropy of random variable X.

Measures average uncertainty or information content of a probability distribution.

H(X|Y)

Conditional entropy.

Measures remaining uncertainty about X after Y is known.

I(X;Y)

Mutual information.

Measures the information shared between X and Y.

D(P‖Q)

Kullback–Leibler divergence between probability distributions P and Q.

Measures a directed divergence between distributions; it is not generally a symmetric distance.

H_b(p)

Binary entropy function.

H_b(p) = −p log₂p − (1−p)log₂(1−p), with endpoint values defined by continuity.

C

Channel capacity.

The maximum reliable information rate under a specified channel model and coding assumptions.

R

Information rate or code rate in a specified coding context.

Must be defined because R has multiple meanings across information theory and communication engineering.

ε

A small error probability or approximation parameter in some theoretical contexts.

The exact meaning must be defined in the relevant theorem or model.

Coding And Digital Communication Notation

SymbolMeaning

bit

A binary information unit with two possible values, conventionally 0 and 1.

byte

A group of eight bits in the widely used modern convention.

M

Number of possible modulation symbols in an M-ary modulation scheme.

log₂M

Number of bits per symbol when M is a power of two and each symbol represents an equal number of uncoded bits.

R_b

Bit rate, measured in bits per second.

R_s

Symbol rate, measured in baud.

r_c

Code rate, commonly information bits divided by coded bits for a specified block or coding convention.

BER

Bit error rate or estimated bit error ratio, depending on the measurement convention.

SER

Symbol error rate.

SNR

Signal-to-noise ratio.

E_b/N₀

Energy per information bit divided by noise spectral density under the specified convention.

QAM

Quadrature amplitude modulation.

PSK

Phase-shift keying.

FSK

Frequency-shift keying.

Network And Protocol Notation

SymbolMeaning

G = (V, E)

A graph representation of a network, with vertices V and edges E.

V

Set of network nodes or vertices.

E

Set of network links or edges.

d(u, v)

Distance or path cost between nodes u and v, as defined by the network model.

IP

Internet Protocol.

TCP

Transmission Control Protocol.

UDP

User Datagram Protocol.

DNS

Domain Name System.

HTTP

Hypertext Transfer Protocol.

HTTPS

HTTP over a secure transport using TLS.

MAC

Media Access Control in networking contexts.

MTU

Maximum Transmission Unit.

RTT

Round-trip time.

TTL

Time to Live in relevant packet and protocol contexts.

QoS

Quality of Service.

Network Performance Notation

MetricNotationUnit

Bandwidth

B

Hz for frequency bandwidth; bit·s⁻¹ when referring to a data-rate capacity or link rate.

Throughput

R or measured throughput

bit·s⁻¹.

Latency

t or delay

s, ms, or μs.

Jitter

Variation in packet delay

s, ms, or μs.

Packet loss

PLR or packet-loss ratio

Dimensionless or percentage.

Availability

A

Dimensionless or percentage.

Signal-to-noise ratio

SNR

Dimensionless; often expressed in dB.

Bit error rate

BER

Dimensionless.

Signal And Modulation Notation

SymbolMeaning

A

Signal amplitude.

f_c

Carrier frequency.

f_m

Modulating-signal frequency.

φ

Signal phase.

λ

Wavelength.

c

Propagation speed, often the speed of light in a vacuum when the electromagnetic context applies.

ω

Angular frequency.

x(t)

Continuous-time signal.

x[n]

Discrete-time signal sample at integer index n.

δ(t)

Continuous-time Dirac delta distribution.

u(t)

Unit-step function in a continuous-time signal model.

X(f)

Frequency-domain representation of a signal under a specified Fourier-transform convention.

Worked Examples

ExampleGivenCalculationResult

Self-information

An outcome has probability P(x) = 1/8.

I(x) = −log₂(1/8) = −log₂(2⁻³)

Self-information = 3 bits.

Binary entropy

A binary source produces 0 and 1 with equal probability, p = 0.5.

H_b(0.5) = −0.5log₂(0.5) − 0.5log₂(0.5)

Entropy = 1 bit per source symbol.

Ideal channel capacity

A channel has bandwidth B = 1 MHz and linear SNR = 15.

C = 10⁶log₂(1 + 15) = 10⁶log₂(16)

The Shannon capacity is 4 Mbit/s under the ideal AWGN channel assumptions.

Bits per modulation symbol

A modulation scheme has M = 16 distinct symbols.

Bits per symbol = log₂16

Each symbol can represent 4 uncoded bits when the mapping is one-to-one and all symbols are used.

Network packet-loss ratio

A system transmits 10,000 packets and 20 are lost.

PLR = 20/10,000

Packet-loss ratio = 0.002 or 0.2%.

Interpretation Rules

  • Specify the logarithm base when defining information and entropy; base 2 gives bits and base e gives nats.

  • Distinguish a bit from a transmitted symbol and a byte from a packet.

  • Do not confuse symbol rate in baud with bit rate in bits per second.

  • State whether SNR is expressed as a linear ratio or in decibels.

  • Use the correct noise spectral-density convention when interpreting E_b/N₀.

  • Distinguish theoretical channel capacity from practical achieved throughput.

  • Define graph direction, edge weights, and node meanings when modeling networks.

  • Interpret latency, jitter, packet loss, and throughput as different performance metrics.

  • Check protocol specifications before assuming the meaning of a field, flag, or header symbol.

Reference

  • Shannon, A Mathematical Theory of Communication

  • Cover and Thomas, Elements of Information Theory

  • Proakis and Salehi, Digital Communications

  • Tanenbaum and Wetherall, Computer Networks

  • Kurose and Ross, Computer Networking: A Top-Down Approach

  • IEEE 802 series: Local and Metropolitan Area Networks

Related Topic

  • SCI-REF-006-091

  • SCI-REF-006-094

  • SCI-REF-006-095

  • SCI-REF-006-097

  • SCI-REF-006-099

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Source verification pending; check an authoritative textbook, university, standards body, or peer-reviewed source.

Suggest a correction or source · More in Scientific Symbols & Notation