Dynamic Programming for Sequential Deterministic Quantization of Discrete Memoryless Channels

In this paper, under a general cost function, we present a dynamic programming (DP) method to obtain an optimal sequential deterministic quantizer (SDQ) for $q$-ary input discrete memoryless channel (DMC). The DP method has complexity $O(q (N-M)^2 M)$, where $N$ and $M$ are the alphabet sizes of the DMC output and quantizer output, respectively. Then, starting from the quadrangle inequality (QI), two techniques are applied to reduce the DP method's complexity. One technique makes use of the SMAWK algorithm and achieves complexity $O(q (N-M) M)$. The other technique is much easier to be implemented and achieves complexity $O(q (N^2 - M^2))$. We further derive a sufficient condition under which the optimal SDQ is optimal among all quantizers and the two techniques are also applicable. This generalizes the results in the literature for binary-input DMC. Next, we show that the cost function of $\alpha$-mutual information ($\alpha$-MI)-maximizing quantizer belongs to the general cost function we adopt earlier. We further prove that under a weaker condition than the sufficient condition we derived, the aforementioned two techniques are applicable to the design of $\alpha$-MI-maximizing quantizer. Finally, we propose a new algorithm called iterative DP (IDP). Theoretical analysis and simulation results demonstrate that IDP can improve the quantizer design over the state-of-the-art methods in the literature.