Ch 4 - Continuous Delivery for Machine Learning Models

[eduardobonet]

Discussion over Chapter 4 - Continuous Delivery for Machine Learning Models

[RMHogervorst]

Shipping models into production is the number one objective, but doing so with very high confidence, in a resilient set of steps, is what you should strive for.

I think this chapter really nicely encapsulates the core values of mlops:

• automate the release process,
• automate checks,
• design your process to be easily debuggable,
• fail fast (early in the process, so get fast feedbackloops)

[RMHogervorst]

I did not really think about the difference between blue-green deployments and canary deployments, but in this chapter I read it as:

blue-green deployment:

• set up a new version of your code (green) parallel to the current production version (blue)
• switch over from blue to green (route all traffic that would have gone to blue to green)
• if things fail, go back to blue

canary deployments:

• set up a few instances (canary) of your newer code next to the current production version
• route a percentage of your traffic to those new instances
• check metrics to see if the performance is not worse
• if the performance is not worse or even improving increase the percentage of traffic that goes to newer versions
• keep checking metrics until 100% of traffic goes to new version

I guess you could make mix-forms too with the shadow deployment - release form:

• set up rules: when guardrail metrics are not out of bounds, and performance metric is equal or better, switch over
• set up a few instances of your newer code next to the current production version
• route traffic to old instances AND ALSO DUPLICATE TO newer instances
• collect and compare responses between shadow deployment and current production version
• look at metrics, and follow rules