Ceph Storage Fundamentals - Level 1

Executive Summary

Ceph is a free and open-source software-defined storage platform that provides object storage, block storage, and file storage built on a common distributed cluster foundation. Ceph provides distributed operation without a single point of failure and scalability to the exabyte level.

This comprehensive 3-day training program introduces participants to Ceph storage fundamentals, covering the latest Squid release (19.2.x) features and best practices. The Squid release features key upgrades across various components including performance enhancements, BlueStore optimizations for snapshot-heavy workloads, and improved RADOS scrub scheduling behavior.

Training Objectives

Upon completion of this training, participants will be able to:

1. Understand Ceph architecture and core components (RADOS, RBD, RGW, CephFS)
2. Deploy a basic Ceph cluster using cephadm orchestrator
3. Configure storage pools with replication and erasure coding
4. Manage Ceph block devices (RBD) for virtualization environments
5. Implement object storage using RADOS Gateway (RGW)
6. Monitor cluster health and perform basic troubleshooting
7. Apply best practices for performance optimization and data protection

Detailed Training Program

Day 1: Ceph Fundamentals and Architecture

Module 1: Introduction to Software-Defined Storage (9:00 AM - 10:30 AM)

1.1 Evolution of Storage Technologies

• Traditional storage limitations
• Software-defined storage benefits
• Ceph's approach to scalability, flexibility, and fault tolerance on commodity infrastructure

1.2 Ceph Overview

• Ceph's history from Sage Weil's doctoral dissertation to a fully functional open source storage platform
• Current ecosystem and adoption
• Active releases: Squid (19.2.2) and Reef (18.2.7)

1.3 Use Cases and Deployment Scenarios

• Private cloud infrastructure (hyper-converged and disaggregated)
• Big data analytics and data lakes
• Kubernetes persistent storage
• Backup and archive solutions

☕ Break: 10:30 AM - 10:45 AM

Module 2: Ceph Architecture Deep Dive (10:45 AM - 12:00 PM)

2.1 RADOS - The Foundation

• RADOS (Reliable Autonomic Distributed Object Store) as the foundation of Ceph
• Object storage concepts
• CRUSH algorithm fundamentals

2.2 Core Components

• Cluster monitors (ceph-mon) that maintain the map of the cluster state
• Managers (ceph-mgr) that maintain cluster runtime metrics and dashboarding
• Object storage devices (ceph-osd) that store data and handle replication
• Metadata servers (ceph-mds) for CephFS

2.3 Storage Interfaces

• Block Devices: RBD service provides resizable, thin-provisioned block devices
• Object Storage: RGW service provides RESTful APIs compatible with Amazon S3
• Filesystem: CephFS service provides a POSIX compliant filesystem

🍽️ Lunch Break: 12:00 PM - 2:00 PM

Module 3: Deployment Prerequisites (2:00 PM - 3:30 PM)

3.1 Hardware Requirements

• Ceph runs on commodity hardware, making it a cost-effective solution
• Recommended specifications for OSD nodes
• Network requirements and considerations
• SSD/NVMe for performance tiers

3.2 Operating System Preparation

• Supported Linux distributions
• Kernel requirements
• System configuration (time sync, hostnames, SSH)
• Firewall rules and port requirements

3.3 Planning Your Cluster

• Sizing considerations
• CRUSH failure domains and replication strategies
• Network topology planning
• Performance expectations

☕ Break: 3:30 PM - 3:45 PM

Module 4: Hands-on Lab - Environment Setup (3:45 PM - 6:00 PM)

Lab 1: Preparing the Infrastructure

• Setting up virtual machines or physical servers
• Configuring networking and hostname resolution
• Installing required packages
• Setting up SSH key authentication

Lab 2: Pre-deployment Checks

• Verifying system requirements
• Testing network connectivity
• Time synchronization verification
• Creating deployment user

Day 2: Deployment and Core Services

Module 5: Ceph Deployment with Cephadm (9:00 AM - 10:30 AM)

5.1 Introduction to Cephadm

• Orchestrator architecture
• Container-based deployment benefits
• Cephadm vs legacy deployment tools

5.2 Bootstrap Process

• Installing cephadm
• Bootstrap command and options
• Initial monitor deployment
• Dashboard access configuration

5.3 Expanding the Cluster

• Adding OSD hosts
• Adding monitors for high availability
• Deploying manager daemons
• Service placement specifications

☕ Break: 10:30 AM - 10:45 AM

Module 6: Storage Configuration (10:45 AM - 12:00 PM)

6.1 OSD Management

• BlueStore optimizations for better performance in snapshot-intensive workloads
• Adding OSDs to the cluster
• Device classes (HDD, SSD, NVMe)
• OSD removal and replacement

6.2 Pool Creation and Management

• Replicated pools configuration
• Erasure coding profiles and overhead calculations
• Pool quotas and limits
• Enabling EC overwrites for RBD and CephFS

6.3 CRUSH Map Customization

• Understanding CRUSH rules
• New CRUSH rule types for flexible EC configurations
• Failure domain configuration
• Custom placement rules

🍽️ Lunch Break: 12:00 PM - 2:00 PM

Module 7: Ceph Block Storage (RBD) (2:00 PM - 3:30 PM)

7.1 RBD Fundamentals

• RBD architecture built on RADOS foundation with librados interface
• Image creation and management
• Snapshots and clones capabilities inherited from librados
• Thin provisioning benefits

7.2 RBD Features in Squid

• Local diff-iterate execution for dramatic performance improvement in QEMU backups
• Multiplexing of image mappings for reduced TCP session overhead
• Persistent write-back cache
• Image migration and mirroring

7.3 Integration with Virtualization

• Integration with Kernel-based Virtual Machines (KVMs)
• QEMU/KVM configuration
• OpenStack Cinder integration
• Performance optimization tips

☕ Break: 3:30 PM - 3:45 PM

Module 8: Hands-on Lab - RBD Operations (3:45 PM - 6:00 PM)

Lab 3: Working with RBD

• Creating and configuring RBD pools
• Creating, resizing, and deleting RBD images
• Taking and managing snapshots
• Cloning images for rapid provisioning
• Mapping RBD devices to clients
• Performance testing with fio

Lab 4: RBD Integration

• Configuring RBD for KVM/QEMU
• Creating VM disks on Ceph
• Live migration scenarios
• Monitoring RBD performance

Day 3: Object Storage, Monitoring, and Best Practices

Module 9: Ceph Object Storage (RGW) (9:00 AM - 10:30 AM)

9.1 RADOS Gateway Architecture

• RGW as a FastCGI service providing RESTful HTTP API
• Multi-tenancy support
• Zone and zone group concepts

9.2 S3 and Swift APIs

• New AWS-compatible IAM APIs for self-service management
• API compatibility overview
• Authentication methods
• Bucket policies and ACLs

9.3 Advanced RGW Features

• Multi-site replication fixes for encrypted S3 objects
• Bucket lifecycle policies
• Object versioning
• Server-side encryption

☕ Break: 10:30 AM - 10:45 AM

Module 10: Monitoring and Maintenance (10:45 AM - 12:00 PM)

10.1 Ceph Dashboard

• UI/UX overhaul making navigation more intuitive for system administrators
• Dashboard features and capabilities
• Performance metrics visualization
• Alert configuration

10.2 Command-line Monitoring

• Health checks and status commands
• Performance counters
• Log analysis
• Troubleshooting tools

10.3 Maintenance Operations

• Improved scrub scheduling behavior in RADOS
• Rolling upgrades procedure
• Backup strategies
• Disaster recovery planning

🍽️ Lunch Break: 12:00 PM - 2:00 PM

Module 11: Performance Optimization (2:00 PM - 3:30 PM)

11.1 Performance Tuning Basics

• BlueStore RocksDB LZ4 compression enabled by default
• Network optimization
• OSD tuning parameters
• Client-side caching

11.2 Workload-Specific Optimization

• Optimizations for snapshot-heavy workloads
• Database workload tuning
• Large object optimization
• Small file performance

11.3 Troubleshooting Performance Issues

• Identifying bottlenecks
• Using performance counters
• Analyzing slow requests
• Capacity planning

☕ Break: 3:30 PM - 3:45 PM

Module 12: Best Practices and Q&A (3:45 PM - 6:00 PM)

12.1 Production Best Practices

• Hardware selection guidelines
• Erasure coding vs replication trade-offs
• Security considerations
• Monitoring and alerting setup

12.2 Common Pitfalls and Solutions

• Known issues like iSCSI upgrade bug in 19.2.0
• Recovery scenarios
• Data migration strategies
• Version compatibility

Lab 5: Capstone Project

• Design a Ceph cluster for a given scenario
• Implement basic monitoring
• Configure alerts
• Document the deployment

12.3 Course Wrap-up

• Q&A session
• Additional resources
• Certification path discussion
• Feedback collection

Required Materials

Hardware Requirements (per participant)

• 4 VMs or physical servers (minimum)
  • 16GB RAM per node (minimum)
  • 2 CPU cores per node
  • 1 OS disk (20GB) + 2 data disks (50GB each) per OSD node
  • 1Gbps network (10Gbps recommended)

Software Requirements

• Rocky Linux 8.x or Ubuntu 22.04 LTS
• Docker/Podman container runtime
• Python 3.6+
• SSH client
• Web browser for dashboard access

Pre-course Preparation

1. Review basic Linux commands
2. Understand storage concepts (RAID, LUN, filesystem)
3. Basic networking knowledge (TCP/IP, VLANs)
4. Familiarity with virtualization concepts

Certification and Next Steps

Assessment Method

• Practical lab exercises (60%)
• Quiz on theoretical concepts (20%)
• Capstone project (20%)

Certificate of Completion

Participants who successfully complete all modules and assessments will receive a "Ceph Storage Fundamentals - Level 1" certificate from ECINTELLIGENCE.

Recommended Next Steps

1. Ceph Level 2 - Advanced Administration
   • CephFS deep dive
   • Advanced troubleshooting
   • Performance optimization
   • Multi-site deployments
2. Red Hat Certified Specialist in Ceph Storage
   • Official certification exam (EX260)
   • Advanced configuration skills
   • Production deployment scenarios
3. Hands-on Practice
   • Deploy Ceph in test environment
   • Join Ceph community forums
   • Contribute to documentation

Additional Resources

Official Documentation

• Ceph Documentation
• Ceph Community Blog
• Ceph GitHub Repository

Community Resources

• Ceph Users Mailing List
• Ceph IRC Channel (#ceph on OFTC)
• Cephalocon Conference Materials

Recommended Reading

• "Learning Ceph" - Second Edition
• "Mastering Ceph" - Infrastructure Design
• Ceph Architecture Papers

About the Instructor

Bekroundjo Akoley Aristide is a seasoned storage infrastructure expert with extensive experience in designing, deploying, and managing large-scale Ceph clusters. With a deep understanding of distributed storage systems and cloud infrastructure, Aristide brings practical insights and real-world scenarios to the training program.