Architecting Big Data Applications: Batch Mode Application Engineering

Introduction

• Architecting big data applications

1. Introduction to Big Data Applications

• Characteristics of batch processing
• Challenges building batch applications
• Technologies for batch big data engineering
• Use cases for batch big data
• Architecture process for data engineering

2. Big Data Architecture Principles

• Making the choice: Real-time vs. batch
• Horizontal scaling
• Distributed processing
• Technology selection
• Technology integrations

3. Batch Application Architecture Principles

• Schedule selection
• Minimizing data volumes
• Uniform load distribution
• Using caches
• Reprocessing

4. Use Case 1: Audit Trail Data Archive

• Audit trail: Define the problem
• Audit trail: Study requirements
• Audit trail: Create a workflow
• Audit trail: Scale the workflow
• Audit trail: Select technologies
• Audit trail: Review final architecture

5. Use Case 2: Advertising Analytics

• Advertising analytics: Define the problem
• Advertising analytics: Study requirements
• Advertising analytics: Create a workflow
• Advertising analytics: Scale the workflow
• Advertising analytics: Select technologies
• Advertising analytics: Review final architecture

6. Use Case 3: Product Recommendations

• Product recommendations: Define the problem
• Product recommendations: Study requirements
• Product recommendations: Create a workflow
• Product recommendations: Scale the workflow
• Product recommendations: Select technologies
• Product recommendations: Review the final architecture

Conclusion

• Continuing to architect big data applications