July 2024

Conference Paper

Towards FAIR Workflows for Federated Experimental Sciences

By:
Saranathan, Gayathri; Foltin, Martin; Tripathy, Aalap; Justine, Annmary; Ziatdinov, Maxim; Ghosh, Ayana ; Roccapriore, Kevin M; Bhattacharya, Suparna; Faraboschi, Paolo
Page Number:
1436-1437
Book Title:
2024 IEEE Conference on Artificial Intelligence (CAI)
Publication Date:
July 2024
Publisher Location:
IEEE, New Jersey, United States of America
Conference Name:
2024 IEEE Conference on Artificial Intelligence (CAI)
Conference Location:
Singapore, Singapore
Conference Sponsor:
IEEE
View DOI Listing:
https://doi.org/10.1109/CAI59869.2024.00256

Abstract

A de-centralized, peer-to-peer AI metadata framework is demonstrated which can enable end-to-end metadata & lineage tracking for distributed Machine Learning pipelines spanning edge, High Performance Computing, and cloud environments. With a specific example of end-to-end microscopy algorithm and datasets, the proposed method shows how to enable reproducibility, audit trail, provenance of metadata artifacts. The emerging needs of automation in experimental sciences, ML-centric workflows, and FAIR metadata management across federated compute environments is addressed.


Related Researchers