omeka2dsp Documentation

Modified

August 29, 2025

Complete documentation for the Omeka to DaSCH Service Platform (DSP) data migration tool.

📚 Core Documentation

  • 🏗️ System Architecture – Overview of system components, design patterns, and data flow architecture with class diagrams illustrating core components and their interactions.
  • 🔄 Data Migration Workflows – Complete workflow documentation with Mermaid diagrams covering the main migration workflow (end-to-end process), data extraction workflow (Omeka API interaction), data transformation workflow (format conversion), synchronization workflow (incremental updates), file upload workflow (media processing) and error handling workflow (recovery and retry)
  • 🔧 API Reference – Comprehensive documentation of all 5 Python modules with 50+ functions
  • 🧩 Data Model – Data model documentation

📒 Guides

🚀 Quick Start

  1. Setup Environment: Configure your environment variables (see Configuration Guide)
  2. Install Dependencies: Install required Python packages
  3. Run Migration: Execute the data transfer script
# Set up environment variables
cp example.env .env
# Edit .env with your configuration

# Run the migration
uv run python scripts/data_2_dasch.py

System Overview

The omeka2dsp system transfers research data from Stadt.Geschichte.Basel (SGB)’s Omeka instance to the DaSCH Service Platform (DSP) for long-term preservation.

Features

  • ✅ Full Data Migration – Transfer metadata and media files
  • ✅ Incremental Sync – Update existing resources with changes
  • ✅ Multiple Modes – Process all data, samples, or test data
  • ✅ Error Handling – Comprehensive logging and error recovery
  • ✅ Configuration – Flexible environment-based configuration

Key Components

graph LR
    A[Omeka API] --> B[data_2_dasch.py]
    B --> C[DSP API]
    B --> D[File Storage]
    
    E[process_data_from_omeka.py] --> B
    F[Configuration] --> B
    
    style A fill:#e1f5fe
    style C fill:#e8f5e8
    style B fill:#fff3e0

Main Scripts

  • data_2_dasch.py – Main migration script with sync capabilities
  • process_data_from_omeka.py – Omeka API data extraction utilities
  • api_get_project.py – Fetch DSP project information
  • api_get_lists.py – Retrieve DSP list configurations
  • api_get_lists_detailed.py – Get detailed list metadata

For detailed API documentation, see API Reference.

Architecture

The system follows a modular architecture with clear separation of concerns:

  • Data Extraction Layer – Interfaces with Omeka API
  • Transformation Layer – Converts data formats between systems
  • Upload Layer – Manages file transfers and API interactions
  • Synchronization Layer – Handles incremental updates and conflict resolution

For detailed architecture documentation, see System Architecture.

Back to top