- Machine Learning With Go
- Daniel Whitenack
- 181字
- 2021-07-08 10:37:29
Pachyderm jargon
Think about versioning data in Pachyderm kind of like versioning code in Git. The primitives are similar:
- Repositories: These are versioned collections of data, similar to having versioned collections of code in Git repositories
- Commits: Data is versioned in Pachyderm by making commits of that data into data repositories
- Branches: These lightweight points to certain commits or sets of commits (for example, master points to the latest HEAD commit)
- Files: Data is versioned at the file level in Pachyderm, and Pachyderm automatically employs strategies, such as de-duplication, to keep your versioned data space efficient
Even though versioning data with Pachyderm feels similar to versioning code with Git, there are some major differences. For example, merging data doesn't exactly make sense. If there are merge conflicts on petabytes of data, no human could resolve these. Furthermore, the Git protocol would not be space efficient in general for large sets of data. Pachyderm uses its own internal logic to perform the versioning and work with versioned data, and the logic is both space efficient and processing efficient in terms of caching.
推薦閱讀
- Java Web開發學習手冊
- 案例式C語言程序設計
- Redis Applied Design Patterns
- MySQL數據庫應用與管理 第2版
- INSTANT Sencha Touch
- JS全書：JavaScript Web前端開發指南
- 51單片機C語言開發教程
- Learning Material Design
- OpenStack Networking Essentials
- Spring 5 Design Patterns
- Python從入門到精通（第3版）
- Arduino Wearable Projects
- PHP 8從入門到精通（視頻教學版）
- Yii2 By Example
- 3D Printing Designs：Design an SD Card Holder