大规模多输入多输出（Massive Multiple-Input Multiple-Output, Massive MIMO）技术被认为是第五代无线通信网络中最具突破性技术之一。在大规模MIMO系统中，基站端部署了数百个天线元件的大规模天线阵，可以获得额外的空间自由度从而显著提高系统的吞吐量和能量效率。与传统的MIMO系统相比，大规模MIMO系统可以利用有限的时频资源，同时为海量的用户提供通信服务。然而，随着用户数量和天线规模的快速增长，大规模MIMO系统也面临着许多严峻的挑战。首先，随着信道矩阵维度的增加，传统的信号处理技术在保证系统性能的同时会产生系统难以承受的计算复杂度；同时，由于用户数量增加，用户间干扰加剧导致系统遭受严重的性能损失；其次，虽然毫米波通信为大规模MIMO系统提供了更加丰富的频带资源，但是毫米波频段信号较高的路径损耗也严重限制了信号的传输距离。针对大规模MIMO系统的这些挑战，大规模MIMO系统与毫米波通信、混合波束赋形、用户选择等关键技术相结合成为了有效解决方案并引起了极大关注。

本文专注于研究多用户毫米波大规模MIMO系统下行链路传输设计，目标在于通过优化混合波束赋形算法与用户选择算法实现大容量、低复杂度的毫米波大规模MIMO系统的数据传输。为实现这一目标，本文研究了两个优化方案：第一个是基于互信息准则的两阶段混合波束赋形器方案，第二个则是连续位置信息驱动的用户选择方案。本文的主要研究内容和工作成果如下：

本文研究了多用户大规模MIMO系统中基于互信息准则的混合波束赋形设计。在数字波束赋形阶段，通过块对角化 (Block Diagonal, BD)技术对数字波束赋形的预编码器和合并器进行设计，消除了多用户间干扰。接着，为获得等效基带信道容量的最大值，在模拟波束赋形阶段，采用分块矩阵的方式将高阶模拟波束赋形矩阵的求解问题分解为结构简单的低阶子矩阵进行求解。通过仿真分析，与现有的混合波束赋形方案相比，本文研究的两阶段混合波束赋形方案在传输速率和系统鲁棒性方面都具有显著优势。

在两阶段混合波束成形的基础上，本文提出了一种新颖的用户选择优化方案。利用位置信息对所有的候选用户进行带约束的空间密度聚类(DBCluC)，在聚类算法中考虑了障碍物约束，根据聚类结果筛除了信道能量最差的用户类，剩余的用户构成用户选择新的候选用户集。在多用户大规模MIMO系统中，用户的信道相关性和信道能量是制约系统性能的重要因素。为此，本文基于位置信息计算了可以等效表示信道间相关性的几何角，同时推演出表示信道能量的渐进SINR(Signal to Interference Plus Noise Ratio)作为用户选择的标准。本文提出的算法可以分为两步：第一步，以信道相关性为指标选择出干扰最小的半正交用户；第二步，为保证系统的增益利用推导获得的SINR为标准选择最优用户。仿真结果表明，与原有方案相比所提出的方案在保证系统性能的同时具有更低的计算复杂度，同时考虑了散射体约束，因此更接近现实场景。

Massive multi-input multi-output (Massive MIMO) technology is considered as one of the most breakthrough technologies in the fifth generation wireless communication networks. In Massive MIMO systems, a large-scale array with hundreds of antenna elements deployed at the base station can obtain additional spatial freedom to significantly improve the system's throughput and energy efficiency. Compared with traditional MIMO systems, Massive MIMO systems can utilize limited time-frequency resources and provide communication services for a large number of users. However, with the rapid growth of the number of users and antenna scale, Massive MIMO systems are also facing many serious challenges. Firstly, with the increase of channel matrix dimension, the traditional signal processing technologies  produce computational complexity that the system cannot withstand while maintaining system performance. At the same time, as the number of users increases, the interference between users intensifies, leading to serious performance loss of the system. Secondly, although millimeter-wave communication provides more abundant frequency band resources for Massive MIMO systems, the higher path loss of millimeter-wave band signals also severely limits the transmission distance of signals. To address these challenges of Massive MIMO, Massive MIMO systems combined with key technologies such as millimeter wave communication, hybrid beamforming, and user selection have become effective solutions and attracted great attention.

This paper focuses on the downlink transmission design of multi-user millimeter wave Massive MIMO system. The goal is to realize data transmission of millimeter wave Massive MIMO system with big system capacity and low complexity by optimizing hybrid beamforming algorithm and user selection algorithm. In order to achieve this goal, two optimization schemes are studied in this paper: the first is a two-stage hybrid beamforming scheme based on mutual information criteria, and the second is a user selection scheme driven by successive position information. The main research contents and achievements of this paper are as follows:

In this paper, a hybrid beamforming design based on mutual information criteria for multi-user Massive MIMO systems is studied. In the stage of digital beamforming, the block diagonalization BD technology is used to design the digital beamforming pre-encoder and merger, which eliminates the interference between multiple users. Then, in order to obtain the maximum equivalent baseband channel capacity, in the analog beamforming stage, the solution problem of high-order analog beamforming matrix is decomposed into low-order submatrix with simple structure by the way of block matrix. Through simulation analysis, compared with the existing hybrid beamforming schemes, the two-phase hybrid beamforming scheme studied in this paper has significant advantages in terms of transmission rate and system robustness.

Based on two-stage hybrid beamforming, a novel optimization user selection scheme is proposed in this paper. The density-based clustering with constrained (DBCluC) was carried out on all the candidate users using the location information. The obstacle constraint was considered in the clustering algorithm. According to the clustering results, the user class with the worst channel energy was screened out, and the remaining users formed the users to select a new candidate user set. In multi-user Massive MIMO systems, the channel dependence and channel energy of users are important factors that restrict system performance. In this paper, based on the location information, the geometric Angle which can represent the correlation between channels is calculated, and the progressive SINR which represents the channel energy is adopted as the criteria for user selection. The algorithm proposed in this paper can be divided into two steps. The first step is to select the semi-orthogonal user with the least interference based on the index of channel correlation. In the second step, to ensure the gain of the system, the derived SINR is used as the standard to select the optimal user. The simulation results show that compared with the original scheme, the proposed scheme has lower computational complexity while ensuring the system performance. Meanwhile, the scatterer constraint is taken into account, so it is closer to the real scene.