Abstract:

In federated learning, the need for a large amount of parameter exchange may lead to security threats from untrusted participating devices. In order to protect training data and model parameters, effective privacy protection measures must be taken. Given the imbalanced nature of heterogeneous data, this paper proposed an adaptive differential privacy method to protect the security of federated learning based on heterogeneous data. Firstly, different initial privacy budgets were set for different clients, and Gaussian noise was added to the gradient parameters of the local model; Secondly, during the training process, the privacy budget of each client was dynamically adjusted based on the loss function value of each iteration to accelerate convergence speed; Then, set a trusted central node to randomly exchange the parameters of each layer of local models from different clients, and then uploaded the confused local model parameters to the central server for aggregation; Finally, the central server aggregated the obfuscation parameters uploaded by trusted central nodes, added appropriate noise to the global model based on a pre-set global privacy budget threshold, and performed privacy correction to achieve server level privacy protection. The experimental results show that under the same heterogeneous data conditions, compared to ordinary differential privacy methods, the adaptive differential privacy method proposed in this paper has faster convergence speed and better model performance.

Key words: federated learning, heterogeneous data, differential privacy, Gaussian noise