1

DesignCon 2019

Accelerating 56G PAM4 Link Equalization Optimization Using Machine Learning-based Analysis Ting Zhu, Hewlett Packard Enterprise (ting.zhu@hpe.com) Yongjin Choi, Hewlett Packard Enterprise (yongjin.choi@hpe.com) Jacky Chang, Hewlett Packard Enterprise (jacky.chang@hpe.com) Chris Cheng, Hewlett Packard Enterprise (chris.cheng@hpe.com)

2

Abstract Adaptive equalizers are widely applied to improve signal integrity in high-speed communication systems. When multiple equalization schemes are used, it is challenging for the tuning algorithms to handle high-dimensional adaptive parameters and to converge within the limited training time. One method to accelerate the convergence is to reduce the tuning dimensions. In this paper, we proposed a new method to reduce the tuning dimensions through machine learning based principal component analysis (PCA). It uses the link bit-error-rate (BER) for analysis and generates principle tuning vectors. The method is demonstrated with 8G PCIe Gen3 and 56G PAM4 link examples. Author(s) Biography Ting Zhu is a Senior Hardware Engineer, working at Storage Group in Hewlett Packard Enterprise. She is responsible for SI/PI design for data center storage products. Her work has been focusing on high speed serial interfaces, DDR design, and IO modeling. She is also experienced in machine-learning based methodologies for high-speed link design and optimization. She received Ph.D. degree in Electrical Engineering from North Carolina State University. B.S. and M.S. in Electrical Engineering from Zhejiang University, China. She was a Senior Engineer with Intel. Yongjin Choi is a Master Technologist at the Storage Division of Hewlett-Packard Enterprise, where he leads the system signal integrity, modeling, SERDES analysis, and material characterization. He received the Ph.D. degree in electrical engineering from North Carolina State University in 2010. His technical interest includes the machine-learning based SERDES modeling and hardware failure prediction. Jacky Chang is a Distinguished Technologist at Aruba Networks Business Unit of Hewlett-Packard Enterprise. He is a senior system architect responsible for system design of multi-layer Ethernet switches and distinguished engineer for physical layer technologies and signal integrity engineering. Chris Cheng is a Distinguished Technologist at the Storage Division of Hewlett-Packard Enterprise. He is responsible for managing all high speed and electrical designs within the Storage Division. He also held senior engineering positions in Sun Microsystems where he developed the original GTL system bus with Bill Gunning. He was a Principal Engineer in Intel where he led high speed processor bus design team. He was the first hardware engineer in 3PAR and guide their high speed design effort until it was acquired by Hewlett Packard.

3

1 Introduction Adaptive equalizers are widely applied to improve signal integrity in high-speed communication systems. Various equalization schemes such as transmitter pre-emphasis/de-emphasis, receiver CTLE (continuous time linear equalizer), FFE (feed-forward equalizer), and DFE (decision feed-back equalizer), can introduce high-dimensional adaptive parameters. Therefore, tuning algorithms face the challenge to handle those parameters and to converge to an optimal solution within a limited time frame. If it fails, some links are forced to up with unexpected bad settings, while some links have to downgrade to a lower speed. One method to accelerate the convergence is to reduce the tuning dimensionality. In this work, we proposed a new method to reduce the tuning dimensions through Principal Component Analysis (PCA). The method samples the equalizer settings and the link BER data under various channel conditions for PCA and generates the principal tuning vectors. The generated tuning vectors are used for link BER optimization.

In this paper, we investigate the theory and implementation of the methodology and demonstrate the application of the method in cases including 8G PCIe Gen3 and 56G PAM4 links. The paper is organized as follows. Section 2 overviews the adaptive equalization and the convergence challenges. Section 3 discusses the basics of PCA for dimensionality reduction and introduces the proposed method for link equalization optimization based on PCA. Section 4 demonstrates the application of the proposed method in 8G PCIe Gen3 and 56G PAM4 link cases, to show PCA is able to greatly reduce the tuning dimensionality and generate tuning vectors for accelerating link BER optimization. A case combining PCA with genetic algorithm for 56G PAM4 link BER optimization is also presented. It shows that the new method is able to accelerate optimization.

2 Overview of Adaptive Equalization Various types of equalization schemes are adopted in today’s high-speed serial links in order to recover the link performance. The equalizers are designed to be adaptive in order to compensate for varying channels and environmental conditions such as temperature. Equalizer adaptation is critical as it relieves the burden of manually searching for optimal settings.

An example equalization architecture for high data rate application is as in Figure 1. It includes equalization on both transmitter and receiver sides. The ultimate goal with multiple equalizers is to ensure the link works within the BER target.

Transmitter (TX) FIR (Finite Impulse Response) applies equalization using a FIR filter (Figure 2(a)) [1], and it is able to compensate both pre-cursor and post-cursor ISI. Multiple-taps helps to better compensation different channels. The adaptation of TX FIR requires the support of back channel to determine the coefficients.

Receiver (RX) CTLE is implemented using analog circuit and is aiming to compensate both pre-cursor and post-cursor ISI. A tunable CTLE design can adjust high frequency gain and low frequency gain in order to achieve relative flat channel frequency response). The general CTLE design is shown in Figure 2(b).

4

RX FFE is equivalent to the TX FIR to remove pre-cursor and post-cursor ISI. The linear equalizer sum or subtract the incoming signal depending on the FFE coefficients. RX DFE uses an IIR (Infinite impulse response) structure for removing post-cursor ISI. It is a nonlinear filter that uses previously detected symbols to subtract ISI from the input. As Figure 2 (c) shows, the input to the feedback filter consists of the sequence of decisions from previously detected symbols, to remove ISI that was caused by those symbols.

Figure 1 Example link equalization architecture

(a) (b)

(c)

Figure 2 Equalizer structure (a) TX FIR (b) CTLE (c) DFE

5

The large number of adaptive parameters in equalizers poses better potential to recover the link performance, however, the high tuning dimensionality causes more difficulty to adapt and requires longer time to converge. The ability to intelligently control the equalizers is non-trivial. The adaptation algorithm highly relies on the equalization architecture and interface protocol requirements. A lot of equalizers rely on LMS (least-mean-squares) algorithm, which is slow in convergence. Also, it has some assumption on channel conditions for generating the adaptation sequences, therefore they may not be able to handle the real channel conditions efficiently. Moreover, some equalizer blocks have their own adaptation targets, and thus the relations among the adaptation parameters, especially when they are in different blocks are not revealed. The most critical is, the tuning time are bounded by the protocols, therefore the complex equalizers may not have enough time to fully use the adaptation capability before run out of time. If it fails, some links can be stuck in inferior performance with errors, or have to downgrade to a lower speed. One method to accelerate the convergence is to reduce the tuning dimensionality. In this work, we proposed a new method to reduce the tuning dimensions through Principal Component Analysis (PCA) and optimization in the reduced principal component space. The details are discussed in the following sections. 3 Proposed methodology: equalization optimization based on PCA analysis In this work, principal component analysis (PCA) [2] is considered as the method to reduce dimensionality. PCA is based on a linear transformation of the data to an orthonormal base that maximizes the variance of each dimension. PCA reveals correlation between variables in data and eliminates the redundancies while retaining most data’s relevant information. By considering only the most significant principal components (PC), the data can be presented in the reduced dimension space instead of the original space. The reduced dimension space is also called reduced latent space. There are two folded functions of doing PCA, first, it is able to do feature selection of finding the critical tuning parameters for the equalizers. Second, it generates tuning vectors in the reduced dimension. With the reduced dimensions, it will be easier for the optimization algorithm to search the solution space, not only searching runs faster, but also the results of the optimization could be also improved because of the reduced complexity of the design space. PCA dimensionality reduction procedure The theory of PCA is discussed in great detail by Jolliffe [2]. PCA represent the data in terms of principal components (PC) other than on its original axis. Each PC is a linear combination of original variables and PCs are orthogonal to each other. They are computed from the eigenvectors of the covariance matrix. The variance represented in the direction of each eigenvector is given by its corresponding eigenvalue. Dimensionality reduction is achieved by only selecting the significant principal components but removing the components associated to less variance. In this work, sampled data X is a m n× matrix. m is the sample size, n is the dimensions of adaptive equalizer parameters. Each row is a single set of equalizer settings,

6

1 11 1

1

nm n

m m mn

X x xX

X x x

×

= = ∈

Eq. (1)

• To eliminate the impact from wide ranges of the variables, the sampled data is pre-

processed by centering and scaling before PCA.

Mean: 1

1 mn

ii

x xm =

= ∈∑ Eq. (2)

Let Y is a version of X after centering and scaling, W is the scaling matrix

( ) m nY W X x ×= − ∈ Eq. (3)

• In the next step, covariance matrix of Y is computed

T11

n nS Y Ym

×= ∈−

Eq. (4)

• Then eigenvectors and eigenvalues for S are computed by singular value decomposition (SVD) [3]. Let 1, 2 n

na a a ∈， ， are the eigenvectors of S and

1, 2, , nλ λ λ

are the associated eigenvalues. The eigenvectors are the principal components. A is defined as the eigenvector matrix by arranging its column in order of decreasing eigenvalues.

1

1

( )0

m n

n nn

n

B YAA a aλ λ

×

×

= ∈

= ∈≥ ≥ ≥

Eq. (5)

• Dimensionality reduction is achieved by selecting only the first k dimensions of principal components according to the eigenvalues with the coverage of the variations:

1

1

( )k

jjn

jj

kλ

ψλ

=

=

=∑∑

Eq. (6)

• Let n k

kA ×∈ presents the first k columns of A . By reducing dimensionality, the data can be projected on to reduced principal component space which is called reduced latent space by the following transformation:

m k

k kB YA ×= ∈ Eq. (7)

7

Optimization based on PCA analysis In this work, we setup the optimization problem to find optimal equalizers which minimize the link BER. Link BER is the ultimate goal of equalizer optimization. Therefore, the optimization does not optimize any intermediate objectives. The optimization problem can be defined as follows, Find: * arg min ( )

yy f y= Eq. (8)

Subject to: L Uy y y≤ ≤ Assumes f is the objective function which represents the BER performance, and ny∈ is a vector of standardized equalizer adaptive parameters with upper and lower limit Uy and

Ly . Let data projection in principal component space by Tz A y= , then the optimization problem in Eq 8 is transferred in principal component space:

*Find: arg min ( )

Subject to: z

L U

z f Az

y Az y

=

≤ ≤ Eq. (9)

In order to have dimensionality reduction, partial eigenvector matrix n k

kA ×∈ is selected, and the data projecting in the reduced dimensionality is

Tk kz A y= Eq. (10)

The optimization problem is operated in the reduced principal component space, Eq.9 becomes:

*Find: arg min ( )

Subject to: k

k k kz

L U

z f A z

y Az y

=

≤ ≤ Eq. (11)

The constraint L Uy Az y≤ ≤ ensures that the solution is feasible in the original design space. The obtained solution *

kz maps back to the original space to find the corresponding solution in the original space.

In this work, as the adaptive tuning parameters are integers which can be controlled by the algorithm, genetic algorithm (GA) can be applied to solve the integer programming problem. Genetic algorithm [4] is based upon the process of natural selection and does not require gradient statistics. It starts with an initial population, selects parents from this population and applies crossover and mutation operators to generate the new off-springs. The off-springs replace the existing individuals in the population and the process repeats until termination criteria is reached.

8

The overall flow of the proposed methodology is summarized in Figure 3. Before GA starts operate, PCA is executed to reduce the dimensionality and transforms the data to reduced latent space. Optimization is performed in the reduced principal component space, by using PC as the tuning vectors, in order to find the optimal solution until it converged. The PCA based optimization is a general methodology which is not limited to GA solver, other solver can be applied in a similar flow.

Figure 3 Overview of PCA-based equalizer optimization with genetic algorithm (GA)

4 Application to high-speed Channels 8G PCIe Gen3 link PCA Analysis The method discussed in Section 3 was applied to a couple of channels. Figure 4 shows a channel composed of 8G PCIe Gen3 SerDes test board, pattern generators, ISI emulator, crosstalk emulator, and high speed cables. The 8G SerDes receiver has 10 dimensional tunable equalizer parameters in its adaptive CTLE and DFE. The definition of parameters are listed in Table 1.

9

In the experiment, we swept the ISI and crosstalk conditions in the channel. The channel insertion loss at 4GHz varied in 6-24dB and the crosstalk coupling varied from 0-100%. 25 samples of converged equalizer values were collected. The proposed PCA process was performed on the sampled data and principal components were generated. The coefficients for the first 6 PC are listed in Table 2. The columns represent specific principal components and the rows are the coefficients associated with each tunable parameter for a given principal component. It indicates in the first principal component (PC1), the

Figure 4 Test setup for 8G PCIe Gen3 link

Table 1 Adaptive tuning parameters for 8G PCIe Gen3 SerDes

x Description

1 4x − Rx CTLE control parameter 1-4

5 10x − Rx DFE control parameter 5-10

Table 2 The first 6 principal components and the corresponding coefficients

x PC1 PC2 PC3 PC4 PC5 PC6

1x -0.009 -0.4355 -0.1781 0.8236 0.2739 0.024

2x -0.3333 -0.3511 0.4291 -0.1282 0.1171 -0.0283

3x 0.2219 0.4308 -0.4738 0.1557 -0.1003 0.1433

4x 0 0 0 0 0 0

5x 0.3283 -0.4193 0.0765 -0.0634 -0.5087 0.6661

6x -0.4902 -0.0448 -0.1686 0.1401 -0.1927 0.1646

7x -0.2501 0.4936 0.2417 0.2594 0.0754 0.4978

8x 0.0711 0.2613 0.5907 0.4157 -0.4517 -0.2576

9x 0.4082 0.0884 0.3417 -0.0256 0.6161 0.3047

10x 0.5072 -0.0633 0.035 0.1288 -0.1267 -0.3166

10

parameters 10x , 9x , 6x explain most of the variation in the data. 4x converged to the same values and it does not have any variations. Therefore, its associated coefficients are all zero. The samples were projected in the PC space and the plot is shown in Figure 5. It was found out, the first principle component covers 39.12% variations and the first 4 PCs cover 90.94% of variations (Figure 6). So by using PCA, we’re able to reduce the 10 dimension of tuning parameters to 4 dimension of tuning vectors. The coefficients of the first 4 principal components can be used to generate the tuning vectors for link equalization optimization in the reduced dimensional space.

Figure 5 Data plot in PC1, PC2, PC3 space.

Figure 6 Percentage of variation coverage by the first 5 principal components and accumulated percentage for 8G PCIe Gen3 case

11

56G PAM4 link PCA analysis and optimization

Figure 7 show the test setup for 56G PAM4 link. It is composed of 56G PAM4 SerDes test board, ISI emulator, crosstalk emulator, and high speed cables. The test pattern was generated by SerDes transmitter. The receiver has adaptive CTLE, FFE and DFE which has 25 dimensional tunable equalizer parameters as listed in Table 3.

Figure 7 Test setup for 56G PAM4 link

Table 3 Adaptive tuning parameters for 56G PAM4 SerDes

x Description

1 6x − Rx CTLE control parameter 1-6

7 11x − Rx FFE control parameter 1-5

12 25x − Rx DFE control parameter 1-14

We swept the ISI and crosstalk conditions. The channel insertion loss at 14GHz varied in 6-21dB and the crosstalk coupling varied from 0-100%. 36 samples of converged receiver equalization values and the corresponding BER were collected. They were optimal solutions for the various channel conditions and were used for PCA analysis. The coefficients for the first 6 PC are listed in Table 4. The parameters which converged to the same values have coefficient all zero since they do not have any variations. As Figure 8 shows, the first principal component covers 58.13% variations, and the first 4 principal components cover 91.66% variations. Figure 9 and 10 show partial sample data that were projected in the PC space, in which the scatters of the points are related to the channel loss.

Table 4 The first 6 principal components and the corresponding coefficients

PC1 PC2 PC3 PC4 PC5 PC6

1x -0.2965 0.0556 -0.0322 0.1519 -0.2629 -0.0784

2x 0.213 -0.013 -0.4169 0.3793 -0.4381 -0.1588

3x 0 0 0 0 0 0

12

4x -0.2965 0.0556 -0.0322 0.1519 -0.2629 -0.0784

5x 0 0 0 0 0 0

6x 0 0 0 0 0 0

7x 0.2938 0.0821 0.06 -0.1871 0.0985 0.1912

8x -0.2912 -0.1014 0.0163 0.2177 0.0712 -0.1351

9x -0.2613 0.0175 -0.2692 0.2715 0.1975 0.2151

10x 0 0 0 0 0 0

11x -0.2138 0.1045 -0.3614 0.3356 0.5288 0.1606

12x 0 0 0 0 0 0

13x 0 0 0 0 0 0

14x 0.2388 0.1949 -0.3463 -0.147 0.1361 -0.1563

15x 0.299 -0.0184 -0.0743 0.0226 0.143 0.0716

16x 0.2763 -0.0214 -0.3181 0.0885 0.1723 0.0222

17x 0.2231 -0.3152 -0.1673 0.206 -0.1374 -0.1431

18x 0.2581 -0.2236 -0.0876 0.1141 -0.2106 -0.0228

19x 0.1165 0.47 -0.0671 0.0348 -0.2296 0.1604

20x 0.2167 -0.244 0.1709 0.2337 -0.0117 0.6598

21x 0.1022 0.4787 -0.0224 0.0206 -0.0791 0.0589

22x 0.1081 0.4675 0.0846 0.108 0.085 -0.1498

23x 0.2377 -0.1006 0.2745 0.2505 0.3704 -0.5423

24x 0.1241 0.1974 0.488 0.5703 -0.0313 0.0587

25x 0 0 0 0 0 0

Figure 8 Percentage of variation coverage by the first 6 principal components and accumulated percentage for 56G PAM4 case

13

Figure 9 Data plot in PC1, PC2, and PC3 3D space

(a) (b)

Figure 10 Data plot in 2D space (a) PC1 and PC2 (b) PC1 and PC3 Equalization optimization was performed based on the PCA results, in order to find the optimal raw BER performance. To simplify the problem, partial adaptive parameters were selected and their ranges are bounded as Table 5 shows. Other adaptive parameters were fixed at the baseline. The first 4 PCs were used to generate the tuning vectors for performing optimization in the reduced dimensional space. In this experiment, MATLAB global optimization toolbox [5] was used for the GA solver. The algorithm was executed with a population of 100 elements that were 100 samples in the reduced principal component space. The crossover fraction was 0.8 which specified 80% of the next generation was produced by crossover operation. Mutation was in default setting which added a random number from a Gaussian distribution to each parent vector.

14

Figure 11 show the best fitness function (here is BER) for each generation in general GA and PCA-based GA algorithms. The results show PCA based GA algorithm converged more quickly than the general GA algorithm with fewer generations. The obtained optimal solution in reduced principal component space was mapped back to the original space to get the corresponding equalizer settings in the original space. The best BER performance and the corresponding equalizer values are plotted in Table 5. Moreover, it was also observed the mean fitness function for each generation in PCA based GA algorithm is better than general GA.

Table 5 Selected adaptive parameters and their ranges

x Description Lx Ux

1x Rx CTLE control parameter 1 0 6

2x Rx CTLE control parameter 2 4 8

3x Rx CTLE control parameter 3 0 6

4x Rx FFE control parameter 1 -2 -6

5x Rx FFE control parameter 2 6 10

6x Rx FFE control parameter 3 0 4

7x Rx FFE control parameter 4 0 4

8x Rx DFE control parameter 1 10 12

9x Rx DFE control parameter 2 4 6

(a) (b) Figure 11 Best fitness function versus generation. (a) general GA (b) PCA based GA

Table 6 Converged equalizer values

Optimal x min BER GA-PCA 1 2 3 4 5 6 7 8 96, 2, 2, 4, 8, 1, 2, 10, 5x x x x x x x x x= = = = − = = = = = 3.06e-06

GA 1 2 3 4 5 6 7 8 96, 2, 2, 4, 8, 2, 0, 10, 5x x x x x x x x x= = = = − = = = = = 3.22e-06

15

5 Conclusion This paper proposed a new methodology to reduce the tuning dimensions for the adaptive equalization through principal component analysis. The method generates tuning vectors which can be used to accelerate the link optimization. The proposed PCA based method is able to do feature selection of finding the critical tuning parameters for the equalizers. The method effectively selects the parameters for dimension reduction by detecting the underlying data relations, without a need of knowing details in the equalizer design. The proposed method was applied to 8G PCIe Gen3 and 56G PAM4 link examples. The 8G PCIe Gen3 SerDes receiver equalizers contain 10 dimensional adaptive parameters. By applying the proposed method, 4 principal tuning vector are selected and they are able to cover 90.94% variations. The 56G PAM4 SerDes contains 25 dimensional adaptive parameters in its receiver equalizers. The proposed method selects the first 4 principle tuning vectors which are able to cover 91.65% variations. Thus, both cases demonstrated the proposed method can effectively reduce the adaptive tuning dimensions. We also demonstrated optimizing BER performance of 56G PAM4 link by combing PCA with genetic algorithm (GA). The results show the PCA based GA converged quicker in fewer generations than the general GA method. It is the first time to apply machine learning based analysis to reduce the tuning dimensions of adaptive equalizers, without a need of knowing internal details of the equalization design. By using machine learning-based analysis, it is a general solution for link optimization.

16

References

[1] Stephen H. Hall, Howard L. Heck, Advanced Signal Integrity for High-Speed Digital Designs, John Wiley & Sons, 2008

[2] Jolliffe, I. T., Principal Component Analysis, Springer-Verlag, New York: Springer, 2nd ed., 2002.

[3] Michael E. Wall1, Andreas Rechtsteiner, Luis M. Rocha, Singular value decomposition and principal component analysis, In A Practical Approach to Microarray Data Analysis, Kluwer: Norwell, MA, 2003. pp. 91-109.

[4] John McCall, Genetic algorithms for modelling and optimisation, Journal of Computational and Applied Mathematics, Volume 184, Issue 1, 1 December 2005, Pages 205-222

[5] MATLAB global optimization toolbox (https://www.mathworks.com/help/gads/index.html)