3

I am a newbie in federated learning and just getting to know TensorFlow Federated TFF framework. I have some questions in my mind I would be really appreciated it if anybody can clarify them:

  1. Does Federated Averaging algorithm the only aggregation algorithm supported in TFF? and how it differs from Federated Stochastic Gradient Descent?
  2. Dose Federated Averaging require each client to be trained with the Neural Networks? or it is possible for local data to be trained with any machine learning algorithm?
  3. I have big data, and I am planning to partition my data into smaller datasets and simulated each part as one client? does this work in TFF? and does it consider horizontal or vertical federated learning?

Thanks in advance

Reem
  • 79
  • 6

1 Answers1

6

The Federated Learning for Image Classification tutorial would be a good start to learn TFF: https://www.tensorflow.org/federated/tutorials/federated_learning_for_image_classification The "Preparing the input data" section is related to your third question. About vertical vs. horizontal: I know there are many types of federated learning defined in recent publications. Personally I would call what you described as cross-silo federated learning, see Section 2.2 in this paper for more information: https://arxiv.org/abs/1912.04977

To answer your other questions:

See the above tutorial for how to create an iterative_process with federated averaging while setting SGD learning rate on both the server and client side. You can also implement customized federated learning algorithms: https://www.tensorflow.org/federated/tutorials/building_your_own_federated_learning_algorithm (this tutorial might also answer your second question about customized local training?)

Wennan Zhu
  • 136
  • 5
  • Thank you so much @mangodream for this detailed answer. I would ask further about the cross-silo. 1) My data is about one organization, the partitioning I want to do is to distribute different departments into clients. So each client is representing one department. I was thinking this is horizontal partitioning since all clients share the same features? what do you think? 2) About the FedAvg, I assume this is an aggregation function, used neural network, and calculate SGD in each client? is this right? Thanks again – Reem Oct 14 '21 at 22:10
  • 3
    Yeah, I agree it should be horizontal if all clients share the same features. About FedAvg: there are client side and server side computations, but you get right. See https://www.tensorflow.org/federated/api_docs/python/tff/learning/build_federated_averaging_process for more information. – Wennan Zhu Oct 15 '21 at 18:10
  • many thanks for your valuable comments. – Reem Oct 16 '21 at 20:28