06 - XP, Lean and DevOps: Technical Practices and Agile Culture

XP, Lean and DevOps: Technical Practices and Agile Culture

Beyond Scrum and Kanban, there are other fundamental methodologies and cultures for modern software development: Extreme Programming (XP) with its extreme technical practices, Lean focusing on waste elimination, and DevOps breaking down silos between development and operations.

Extreme Programming (XP)

History and Philosophy

Extreme Programming was born in the late 1990s, created by Kent Beck during the Chrysler C3 project. The name comes from taking good development practices to "extremes":

The 5 XP Values

1️⃣ COMMUNICATION
   └─ Constant, face-to-face, honest

2️⃣ SIMPLICITY
   └─ Do the simplest thing that works (YAGNI)

3️⃣ FEEDBACK
   └─ Fast and continuous (from tests, customer, team)

4️⃣ COURAGE
   └─ Courage to refactor, tell truth, change

5️⃣ RESPECT
   └─ Mutual respect among team members

The 12 XP Practices

1. Test-Driven Development (TDD)

TDD is XP's fundamental practice: write the test before production code.

🔴 RED: Write a failing test
   ├─ Write test for functionality that doesn't exist yet
   ├─ Test must fail (otherwise you're not testing anything!)
   └─ Focus on "what" not "how"

🟢 GREEN: Make the test pass
   ├─ Write minimal code to make test pass
   ├─ Doesn't matter if code is "ugly"
   └─ Goal: green bar as fast as possible

🔵 REFACTOR: Improve the code
   ├─ Improve design while keeping tests green
   ├─ Eliminate duplication
   ├─ Make code more readable
   └─ Tests protect you from regressions

♻️ REPEAT for each small functionality (2-10 min cycles)

// 🔴 RED: Write failing test
describe('isPalindrome', () => {{ '{' }}
  it('should return true for "racecar"', () => {{ '{' }}
    expect(isPalindrome('racecar')).toBe(true);
  {{ '}' }});
{{ '}' }});

// Test fails because isPalindrome doesn't exist!

// 🟢 GREEN: Implement minimal function
function isPalindrome(str: string): boolean {{ '{' }}
  return str === str.split('').reverse().join('');
{{ '}' }}

// Test passes! ✅

// 🔵 REFACTOR: Improve
function isPalindrome(str: string): boolean {{ '{' }}
  const normalized = str.toLowerCase().replace(/[^a-z0-9]/g, '');
  return normalized === normalized.split('').reverse().join('');
{{ '}' }}

// Add more tests (edge cases)
it('should return true for "A man, a plan, a canal: Panama"', () => {{ '{' }}
  expect(isPalindrome('A man, a plan, a canal: Panama')).toBe(true);
{{ '}' }});

it('should return false for "hello"', () => {{ '{' }}
  expect(isPalindrome('hello')).toBe(false);
{{ '}' }});

2. Pair Programming

Two developers work together at the same computer on the same code.

3. Continuous Integration (CI)

Integrate code into shared repository frequently (multiple times per day), with automated build and tests.

📝 DEVELOPER COMMIT (multiple times per day)
   ↓
🔔 CI SERVER TRIGGERED (webhook from Git)
   ↓
📦 BUILD
   ├─ Checkout latest code
   ├─ Install dependencies (npm install)
   ├─ Compile (tsc, webpack)
   └─ Package artifacts
   ↓
🧪 TEST
   ├─ Unit tests (Jest, Mocha)
   ├─ Integration tests
   ├─ Lint & code style (ESLint)
   └─ Security scan (npm audit)
   ↓
✅ SUCCESS: Notify team (Slack, email)
   └─ Artifact ready for deploy

❌ FAILURE: Notify team immediately
   └─ Fix becomes priority #1 (stop the line!)

4. Continuous Refactoring

Refactoring: changing internal code structure without modifying external behavior. Done continuously, not "when we have time".

5. Simple Design

The simplest design that passes all tests. 4 rules in priority order:

1️⃣ Passes all tests
   └─ Correct functionality is priority #1

2️⃣ Reveals intention
   └─ Code clearly expresses what it does

3️⃣ No duplication (DRY)
   └─ Eliminate duplicate code

4️⃣ Fewest elements
   └─ Minimum number of classes/methods needed

6. Collective Code Ownership

The entire team owns all code. Anyone can modify any part.

Other XP Practices

7️⃣  SMALL RELEASES
    └─ Frequent releases (weeks, not months)

8️⃣  40-HOUR WEEK
    └─ Sustainability, no overtime

9️⃣  ON-SITE CUSTOMER
    └─ Customer available daily

🔟 CODING STANDARDS
    └─ Team follows same conventions

1️⃣1️⃣ SYSTEM METAPHOR
    └─ Shared metaphor for architecture

1️⃣2️⃣ SUSTAINABLE PACE
    └─ Marathon, not sprint

Lean Software Development

Origins: Toyota Production System

Lean derives from the Toyota Production System (TPS) developed by Taiichi Ohno in the 1950s. Adapted to software by Mary and Tom Poppendieck (2003).

The 7 Lean Principles

1. Eliminate Waste

The 7 wastes (Muda) in software development:

1️⃣ PARTIALLY DONE WORK
   ├─ Ex: Unintegrated code, 90% complete feature
   └─ Solution: Rigorous Definition of Done, CI/CD

2️⃣ EXTRA FEATURES
   ├─ Ex: Gold plating, "nice to have" never used
   └─ Solution: YAGNI, MVP approach

3️⃣ RELEARNING
   ├─ Ex: Lost knowledge, absent documentation
   └─ Solution: Knowledge sharing, pair programming

4️⃣ HANDOFFS
   ├─ Ex: Dev → QA → Ops (silos)
   └─ Solution: Cross-functional teams, DevOps

5️⃣ DELAYS
   ├─ Ex: Waiting for approvals, dependencies
   └─ Solution: Remove impediments, team autonomy

6️⃣ TASK SWITCHING (Multitasking)
   ├─ Ex: Developer on 5 projects simultaneously
   └─ Solution: WIP limits, focus

7️⃣ DEFECTS
   ├─ Ex: Production bugs, rework
   └─ Solution: TDD, code review, automated testing

2. Build Quality In

Not "add quality after" (testing phase) but build quality in from the first moment.

3. Create Knowledge (Amplify Learning)

Software development is knowledge creation, not repetitive production.

4. Defer Commitment (Decide as Late as Possible)

Delay irreversible decisions until the last responsible moment. More information you have, better decisions you make.

5. Deliver Fast

Short development cycles → fast feedback → accelerated learning.

6. Respect People

People are the most important resource. Empowerment and trust.

7. Optimize the Whole

Optimize the value stream end-to-end, not local sub-optimizations.

VALUE STREAM: From customer request to delivered value

[Customer Request] → [Backlog] → [Dev] → [Test] → [Deploy] → [Customer Value]
     ⏱️ 1 day         ⏱️ 3 days  ⏱️ 5 days ⏱️ 2 days ⏱️ 1 day

📊 LEAD TIME: 12 days total
📊 VALUE-ADD TIME: 8 days (actual work)
📊 WASTE: 4 days (waits, handoffs)

🎯 OPTIMIZATION:
- Reduce backlog time (WIP limits)
- Automate testing (from 2 days to 2 hours)
- CI/CD for deploy (from 1 day to 10 min)

RESULT: Lead time 7 days → 2x faster!

DevOps: Development + Operations

What is DevOps?

DevOps is a culture and set of practices that breaks down silos between Development and Operations, enabling continuous delivery of value.

DEV: "Our goal is to release features quickly"
OPS: "Our goal is stability, no changes!"

RESULT: Conflict, deploy every 3 months, frequent outages

DevOps Team: "Shared goal: deliver value continuously AND safely"

RESULT: Multiple deploys per day, high reliability

CALMS Framework

The 5 pillars of DevOps:

🔵 C - CULTURE
   ├─ Dev-Ops collaboration
   ├─ Shared responsibility
   ├─ Blameless post-mortems
   └─ Continuous learning

🔵 A - AUTOMATION
   ├─ CI/CD pipelines
   ├─ Infrastructure as Code
   ├─ Automated testing
   └─ Self-service deployment

🔵 L - LEAN
   ├─ Eliminate waste
   ├─ Small batch sizes
   ├─ Value stream focus
   └─ Continuous improvement

🔵 M - MEASUREMENT
   ├─ Key metrics (DORA metrics)
   ├─ Monitoring & Observability
   ├─ Data-driven decisions
   └─ Feedback loops

🔵 S - SHARING
   ├─ Knowledge sharing
   ├─ Open communication
   ├─ Cross-team collaboration
   └─ Communities of practice

CI/CD in Detail

Continuous Integration (CI)

Already covered in XP. Developers integrate code frequently (daily), with automated build and tests.

Continuous Delivery (CD)

Every commit that passes CI is potentially deployable to production. Deploy requires manual approval.

Continuous Deployment

Every commit that passes CI is automatically deployed to production. No manual intervention.

🔵 COMMIT & PUSH
   ↓
📦 CI: BUILD & TEST (2-5 min)
   ├─ Compile
   ├─ Unit tests
   ├─ Lint
   └─ Security scan
   ↓
🟢 DEPLOY TO STAGING (auto) (1 min)
   ├─ Blue/Green deployment
   ├─ Smoke tests
   └─ Integration tests (5 min)
   ↓
🟡 MANUAL APPROVAL (Continuous Delivery)
   ├─ Product Owner reviews staging
   ├─ Click "Deploy to Prod" button
   └─ OR: Automatic (Continuous Deployment)
   ↓
🔴 DEPLOY TO PRODUCTION (1 min)
   ├─ Canary release (5% traffic)
   ├─ Monitor metrics
   ├─ Gradual rollout 100%
   └─ Success! 🎉 (or auto-rollback if errors)

Infrastructure as Code (IaC)

Manage infrastructure with versioned code, instead of manual configuration.

// Define infrastructure as code
resource "aws_instance" "web_server" {{ '{' }}
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"

  tags = {{ '{' }}
    Name = "ProductionWebServer"
  {{ '}' }}
{{ '}' }}

resource "aws_s3_bucket" "assets" {{ '{' }}
  bucket = "my-app-assets"
  acl    = "public-read"
{{ '}' }}

// terraform apply → Infrastructure created automatically!
// terraform destroy → Infrastructure deleted
// Version control → Git for infra changes

DORA Metrics

DevOps Research and Assessment identified 4 key metrics to measure DevOps performance:

1️⃣ DEPLOYMENT FREQUENCY
   └─ How often do we deploy to prod?
      Elite: Multiple per day
      High: Weekly to monthly
      Low: Monthly to every 6 months

2️⃣ LEAD TIME FOR CHANGES
   └─ Time from commit to production?
      Elite: < 1 hour
      High: 1 day to 1 week
      Low: 1 month to 6 months

3️⃣ TIME TO RESTORE SERVICE
   └─ How long to restore after incident?
      Elite: < 1 hour
      High: < 1 day
      Low: 1 week to 1 month

4️⃣ CHANGE FAILURE RATE
   └─ % deploys that cause failure?
      Elite: 0-15%
      High: 16-30%
      Low: 31-45%

Integrating XP, Lean and DevOps with Scrum

📅 SPRINT PLANNING (Scrum)
   └─ Team selects User Stories (Lean: pull system)

💻 DEVELOPMENT (XP Practices)
   ├─ Pair Programming
   ├─ TDD (Red-Green-Refactor)
   ├─ Continuous Refactoring
   └─ Simple Design

🔄 EVERY COMMIT (DevOps)
   ├─ CI pipeline runs
   ├─ Automated tests
   ├─ Deploy to staging
   └─ Ready for production

📊 DAILY STANDUP (Scrum)
   └─ Walk the board, identify impediments

🎬 SPRINT REVIEW (Scrum)
   └─ Demo features deployed to staging

🔁 RETROSPECTIVE (Scrum + Lean)
   ├─ Inspect & Adapt
   ├─ Identify waste
   └─ Kaizen experiments

🚀 DEPLOY TO PRODUCTION (DevOps)
   └─ Continuous Delivery: when PO approves

Conclusions

XP, Lean and DevOps are complementary and integrate perfectly with Agile frameworks like Scrum. Together they form a complete approach for modern software development: fast, high-quality, continuous delivery.