Awesome seml

Awesome Software Engineering for Machine Learning ¶

机器学习软件工程是构建 ML 应用程序的技术和指南，这些应用程序不涉及核心 ML 问题——例如新算法的开发——而是围绕数据摄取、编码、测试、版本控制、部署、质量控制等周边活动，以及团队协作. 良好的软件工程实践可以使用机器学习组件增强生产级应用程序的开发、部署和维护.

⭐ 必读

科学出版物

基于这些文献，我们编制了一份关于对具有机器学习组件的应用程序采用软件工程实践的调查.

随意地 take and share the survey 并 read more!

Broad Overviews¶

这些资源涵盖了方方面面. - AI Engineering: 11 Foundational Practices ⭐ - Best Practices for Machine Learning Applications - Engineering Best Practices for Machine Learning ⭐ - Hidden Technical Debt in Machine Learning Systems ⭐ - Rules of Machine Learning: Best Practices for ML Engineering ⭐ - Software Engineering for Machine Learning: A Case Study ⭐

Data Management¶

如何管理您在机器学习中使用的数据集.

• A Survey on Data Collection for Machine Learning A Big Data - AI Integration Perspective_2019
• Automating Large-Scale Data Quality Verification
• Data management challenges in production machine learning
• Data Validation for Machine Learning
• How to organize data labelling for ML
• The curse of big data labeling and three ways to solve it
• The Data Linter: Lightweight, Automated Sanity Checking for ML Data Sets
• The ultimate guide to data labeling for ML

Model Training¶

如何组织模型训练实验.

• 10 Best Practices for Deep Learning
• Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement
• Fairness On The Ground: Applying Algorithmic FairnessApproaches To Production Systems🎓
• How do you manage your Machine Learning Experiments?
• Machine Learning Testing: Survey, Landscapes and Horizons
• Nitpicking Machine Learning Technical Debt
• On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach ⭐
• On human intellect and machine failures: Troubleshooting integrative machine learning systems
• Pitfalls and Best Practices in Algorithm Configuration
• Pitfalls of supervised feature selection
• Preparing and Architecting for Machine Learning
• Preliminary Systematic Literature Review of Machine Learning System Development Process
• Software development best practices in a deep learning environment
• Testing and Debugging in Machine Learning
• What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild

Deployment and Operation¶

如何在生产环境中部署和操作您的模型.

• Best Practices in Machine Learning Infrastructure
• Building Continuous Integration Services for Machine Learning
• Continuous Delivery for Machine Learning ⭐
• Continuous Training for Production ML in the TensorFlow Extended (TFX) Platform
• Fairness Indicators: Scalable Infrastructure for Fair ML Systems
• Machine Learning Logistics
• Machine learning: Moving from experiments to production
• ML Ops: Machine Learning as an engineered disciplined
• Model Governance Reducing the Anarchy of Production
• ModelOps: Cloud-based lifecycle management for reliable and trusted AI
• Operational Machine Learning
• Scaling Machine Learning as a Service🎓
• TFX: A tensorflow-based Production-Scale ML Platform
• The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• Versioning for end-to-end machine learning pipelines

Social Aspects¶

如何组织团队和项目以确保有效的协作和问责制.

• Data Scientists in Software Teams: State of the Art and Challenges
• Machine Learning Interviews
• Managing Machine Learning Projects
• Principled Machine Learning: Practices and Tools for Efficient Collaboration

Governance¶

• A Human-Centered Interpretability Framework Based on Weight of Evidence
• An Architectural Risk Analysis Of Machine Learning Systems
• Beyond Debiasing
• Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing
• Inherent trade-offs in the fair determination of risk scores
• Responsible AI practices ⭐
• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
• Understanding Software-2.0

Tooling¶

工具可以让您的生活更轻松.

我们仅共享开源工具或提供大量免费研究包的商业平台.

• Aim - Aim 是一个开源实验跟踪工具.
• Airflow - 以编程方式创作、安排和监控工作流程.
• Alibi Detect - 专注于异常值、对抗性和漂移检测的 Python 库.
• Archai - 神经结构搜索.
• Data Version Control (DVC) - DVC 是一种数据和 ML 实验管理工具.
• Facets Overview / Facets Dive - 强大的可视化功能有助于理解机器学习数据集.
• FairLearn - 评估和改进机器学习模型公平性的工具包.
• Git Large File System (LFS) - 用 Git 中的文本指针替换大文件，例如数据集.
• Great Expectations - 通过管道集成进行数据验证和测试.
• HParams - 一种用于机器学习项目配置管理的深思熟虑的方法.
• Kubeflow - 为想要构建和试验 ML 管道的数据科学家提供的平台.
• Label Studio - 具有标准化输出格式的多类型数据标记和注释工具.
• LiFT - Linkedin 公平工具包.
• MLFlow - 管理 ML 生命周期，包括实验、部署和中央模型注册表.
• Model Card Toolkit - 简化和自动化模型卡的生成； 用于模型文档.
• Neptune.ai - 实验跟踪工具为数据科学项目带来组织和协作.
• Neuraxle - 用于深度学习项目中的超参数调整和 AutoML 的类似 Sklearn 的框架.
• OpenML - 为机器学习构建开放、有组织的在线生态系统的包容性运动.
• PyTorch Lightning - 用于高性能 AI 研究的轻量级 PyTorch 包装器. 缩放模型，而不是样板.
• REVISE: REvealing VIsual biaSEs - 自动检测视觉数据集中的偏差.
• Robustness Metrics - 用于评估分类模型稳健性的轻量级模块.
• Seldon Core - 一个 MLOps 框架，用于在 Kubernetes 上打包、部署、监控和管理数以千计的生产机器学习模型.
• Spark Machine Learning - Spark 的 ML 库，包含常见的学习算法和实用程序.
• TensorBoard - TensorFlow 的可视化工具包.
• Tensorflow Extended (TFX) - 用于部署生产 ML 管道的端到端平台.
• Tensorflow Data Validation (TFDV) - 用于探索和验证机器学习数据的库. 类似于 Great Expectations，但用于 Tensorflow 数据.
• Weights & Biases - 实验跟踪、模型优化和数据集版本控制.

Contribute¶

欢迎投稿！ 阅读 contribution guidelines 第一