ModularFed: Leveraging Modularity in Federated Learning Frameworks

Introduction

ModularFed is a research-focused federated learning [1] (FL) framework designed to provide a comprehensive set of extendable tools for integrating applications in the FL domain. This framework addresses the limitations of current approaches by offering a modular component support, which simplifies the integration process and enhances scalability and extendability for future projects. By incorporating a protocol-based architecture, ModularFed ensures that individual components can function independently, allowing for greater flexibility and reusability. This modularity not only facilitates the seamless integration of new technologies and methodologies but also reduces the time and effort required for implementation.

Furthermore, ModularFed's design supports a wide range of applications, making it adaptable to various FL scenarios and capable of addressing diverse research challenges. This adaptability is crucial for enabling continuous improvements in the federated learning environment. The framework’s ability to integrate seamlessly with different technologies and datasets enhances its utility, providing researchers and developers with a robust and versatile toolset to advance their FL projects. Overall, ModularFed offers an efficient solution for overcoming the limitations of existing FL frameworks, in terms of both adaptability and scalability.

Main Contributions

1. Modular Architecture: ModularFed introduces a protocol-based modular FL architecture, allowing for individualism and flexibility. The framework supports component independence, easing the integration of component-specific FL approaches and facilitating work extendability.
2. Adaptability: It enhances adaptability by incorporating an intuitive subscription-based architecture, enabling seamless third-party integrations.
3. Data Control Center: The framework integrates a modular data control center for client data simulations, capable of supporting external/context-specific datasets.
4. Extensive Experimentation: ModularFed’s capabilities are validated through extensive experiments in various contexts and configurations, comparing its performance to well-known frameworks such as FedML and TensorFlow Federated.

Framework Architecture

The framework acts as a bridge that enables federated learning for applications while supporting well-known technologies to complete its objectives. Three key components, as shown in Figure 1, distinguishes our architecture from the rest and make the bridge adaptable to both external layers: FL Abstraction Layer (FAL), FL Subscribers and Data Control Center.

1. Federated Abstract Layer (FAL): The framework uses a layered architecture called the Federated Abstract Layer (FAL) designed to address the modularity issues of existing FL frameworks. FAL ensures the flexibility and extendability of FL components.
2. FL Subscribers: A subscription design pattern enables the framework to extend functionalities with various utilities such as monitoring, logging and analysis. This pattern supports live broadcasts about the execution state of federated applications.
3. Data Control Center: Comprising Containers, Distributors and Providers, the Data Control Center standardizes data distribution strategies, supporting most Non-IID/IID scenarios. Moreover, this component eases the complexity of working with multiple sources.

Our architecture incorporates a range of middleware tools into the federated learning workflow using a subscription-based model. Built on the observable pattern, the framework supports real-time updates and a variety of functions, including logging, caching and metrics tracking, without modifying the core system. Figure 2 illustrates how the framework extends the workflow by integrating advanced subscribers, such as Logging and Caching Subscribers to improve execution tracking, and checkpoint management. Additionally, analysis tools monitor model evolution and client selection. This design facilitates seamless integration and collaboration, offering researchers significant flexibility in their methodologies.

Experimental Validation

The performance and flexibility of ModularFed were validated across various FL scenarios, including distribution issues, network consumption and client selection. Experiments showed that the framework effectively manages diverse data distributions and client behaviors, with minimal performance trade-offs compared to basic approaches. Table 2 illustrates the evolution of global model accuracy under different environmental parameters, such as epochs and distributions, which our framework can generate with minimal effort.

Figures 3 and 4 compare our approach with other federated learning frameworks. We compare against FedML [2] and TFF [3], both of which are relevant frameworks in the FL development environment. Unlike the existing approaches that often trade off features for lower performance, the figures demonstrate that ModularFed achieves superior performance and bootstrapping times comparable to native federated learning frameworks, while supporting a comprehensive set of features.

Conclusion

ModularFed provides a robust framework for federated learning, addressing the limitations of current FL approaches by offering modularity, flexibility, and extendability. The framework supports various FL-related issues and is designed to evolve with the integration of new technologies, ensuring its relevance in the rapidly advancing field of federated learning.

References

[1] McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. (2016). Communication-Efficient Learning of Deep Networks from Decentralized Data. ArXiv. /abs/1602.05629

[2] Chaoyang He, undefined., et al. "FedML: A Research Library and Benchmark for Federated Machine Learning," in CoRR, vol. abs/2007.13518, 2020.

[3] The TensorFlow Federated Authors, "TensorFlow Federated," Google, 2018. [Online]. URL: https://github.com/google-parf...

Additional Information

For more information on this research, please read the following paper: Mohamad Arafeh, Hadi Otrok, Hakima Ould-Slimane, Azzam Mourad, Chamseddine Talhi, Ernesto Damiani, ModularFed: Leveraging modularity in federated learning frameworks, Internet of Things, Volume 22, 2023.