Files
kjvstudy.org/kjvstudy_org/data/README.md
T
kennethreitz 222fcf2717 Add comprehensive README for data directory
Created detailed documentation for all 106 JSON data files in the data directory,
including:

## Documentation Coverage
- Complete file listing with sizes and descriptions
- Data structure examples for each JSON file
- Usage information (which modules load each file)
- Editing guidelines and best practices
- Validation and testing procedures
- Performance optimization details
- Troubleshooting guide
- Development workflow

## File Categories Documented
- Bible metadata (67 files): books, chapters, abbreviations
- Interlinear data (12M compressed): Greek/Hebrew with Strong's numbers
- Study resources (25+ files): study guides, commentary, word studies
- Reference materials (10+ files): topics, reading plans, cross-references
- Stories (24 files): biblical narratives by category

## Benefits
- Non-developers can now understand and edit JSON data
- Clear examples of data structure for each file type
- Testing and validation procedures documented
- Performance characteristics explained
- Future enhancement roadmap included

Total documentation: 717 lines covering all data files and best practices.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-27 18:22:38 -05:00

17 KiB

KJV Study Data Directory

This directory contains all biblical reference data for the KJV Study application in structured JSON format. All data is externalized from code for easier maintenance, editing, and version control.

Overview

  • 106 JSON files containing biblical reference data
  • Total size: ~2.1 MB (uncompressed, excluding interlinear data)
  • Coverage: Complete Bible reference materials, study resources, and theological content
  • Format: UTF-8 encoded JSON with consistent structure

Quick Stats

Category Files Size Description
Bible Text & Metadata 67 ~85K Book metadata, chapters, verses, abbreviations
Interlinear Data 1 12M (compressed) Greek/Hebrew words with Strong's numbers
Study Resources 25+ ~1.8M Study guides, commentary, word studies, resources
Reference Materials 10+ ~250K Topics, reading plans, cross-references, timelines

Core Bible Data

bible_metadata.json (5.7K)

Contains: Old Testament books (39), New Testament books (27), book abbreviations (~150)

Structure:

{
  "old_testament_books": ["Genesis", "Exodus", ...],
  "new_testament_books": ["Matthew", "Mark", ...],
  "book_abbreviations": {
    "Gen": "Genesis",
    "Matt": "Matthew",
    ...
  }
}

Used by: utils/books.py, book normalization, sitemap generation


books/ Directory (66 files)

Contains: Individual JSON file for each Bible book with introduction, summary, outline, and key themes

Files: genesis.json, matthew.json, revelation.json, etc.

Structure:

{
  "name": "Genesis",
  "testament": "Old Testament",
  "genre": "Law/Pentateuch",
  "author": "Moses",
  "date_written": "~1440-1400 BC",
  "summary": "...",
  "key_themes": [...],
  "outline": [...]
}

Used by: books.py, book detail pages


Interlinear & Language Data

interlinear.json.gz (12M compressed)

Contains: Complete interlinear Bible data for 31,031 verses with original Greek/Hebrew words, transliterations, Strong's numbers, parsing, and definitions

Structure:

{
  "John:3:16": [
    {
      "position": 1,
      "original": "Οὕτως",
      "transliteration": "houtōs",
      "strongs": "G3779",
      "english": "For so",
      "parsing": "adv",
      "definition": "in this manner, thus, so"
    },
    ...
  ]
}

Used by: interlinear_loader.py, interlinear verse views, API /api/interlinear endpoint

Note: Compressed with gzip for efficient storage and lazy-loaded on first access


word_studies.json (35K)

Contains: 53 biblical terms with Hebrew/Greek definitions, transliterations, and theological notes

Structure:

{
  "love": {
    "ot_term": "אַהֲבָה",
    "ot_transliteration": "ahavah",
    "ot_meaning": "love, affection",
    "ot_note": "...",
    "nt_term": "ἀγάπη",
    "nt_transliteration": "agape",
    "nt_meaning": "divine, self-sacrificial love",
    "nt_note": "..."
  }
}

Coverage: Divine Names (5), Salvation & Redemption (9), Spiritual Nature (9), Worship & Practice (7), Religious Roles (4), Spiritual States (7), Law & Judgment (3), Other Key Terms (9)

Used by: routes/commentary.py, word study sidenotes in verse commentary


Study Resources

study_guides.json (265K)

Contains: 25 complete study guides with 183 sections and 732 verse references

Structure:

{
  "catalog": {
    "Foundational Studies": [
      {
        "title": "New Believer's Guide",
        "description": "...",
        "slug": "new-believer",
        "verses": ["John 3:16", "Romans 8:28", ...]
      }
    ]
  },
  "content": {
    "new-believer": {
      "title": "New Believer's Guide",
      "sections": [
        {
          "title": "God's Infinite Love for You",
          "verses": ["John 3:16", ...],
          "content": "..."
        }
      ]
    }
  }
}

Categories: Foundational Studies, Character & Living, Biblical Themes, Doctrinal Studies, Thematic Studies, Family & Relationships

Used by: routes/study_guides.py, /study-guides pages


verse_commentary.json (56K)

Contains: 22 verses with detailed theological analysis, historical context, applications, and reflection questions (66 total questions)

Structure:

{
  "John 3:16": {
    "analysis": "...",
    "historical_context": "...",
    "application": "...",
    "questions": [
      "How does the phrase 'God so loved the world' challenge...",
      ...
    ]
  }
}

Coverage: 14 books (9 NT, 5 OT) including Genesis, John, Romans, Ephesians, Revelation

Used by: routes/commentary.py, enhanced verse commentary pages


resources.json (1.3M)

Contains: All biblical reference resources including angels, prophets, parables, covenants, apostles, women, festivals, fruits of the Spirit, miracles, prayers, beatitudes, commandments, armor of God, I AM statements, and systematic theology topics

Structure:

{
  "angels": {
    "Major Angels": {
      "Michael": {
        "title": "Michael the Archangel",
        "description": "...",
        "key_verses": [...],
        "content": "..."
      }
    }
  },
  "prophets": {...},
  "parables": {...},
  ...
}

Resource Types:

  • Biblical Figures: Angels (4), Prophets (8), Apostles (12), Women (12)
  • Biblical Content: Parables (8), Miracles, Prayers, Beatitudes (9), Commandments (10)
  • Theological Topics: Trinity, Christology, Soteriology, Pneumatology, Eschatology, Ecclesiology, etc.
  • Special Topics: Armor of God (6), I AM Statements (7), Types & Shadows, Messianic Prophecies, Blood in Scripture, Kingdom of God, Names of Christ, Spirits & Demons, Personifications

Used by: data/__init__.py, routes/resources.py, all resource detail pages


Reference Materials

topics.json (160K)

Contains: Topical verse index organized by major theological and practical topics with subtopics and verse references

Structure:

{
  "Salvation": {
    "description": "God's plan to redeem humanity through Jesus Christ",
    "subtopics": {
      "By Grace": {
        "verses": ["Ephesians 2:8-9", "Romans 3:24", ...]
      },
      "Through Faith": {
        "verses": ["Romans 5:1", "Acts 16:31", ...]
      }
    }
  }
}

Used by: topics.py, /topics pages, topical search


reading_plans.json (122K)

Contains: 8 comprehensive Bible reading plans with daily readings and themes

Plans:

  • Chronological (365 days) - Read in historical order
  • One Year (365 days) - Systematic OT/NT/Psalms/Proverbs
  • New Testament (90 days)
  • Gospels & Acts (30 days)
  • Psalms & Proverbs (60 days)
  • Pentateuch (50 days)
  • Prophets (90 days)
  • Paul's Epistles (30 days)

Structure:

{
  "chronological_plan": {
    "1": ["Genesis 1-3"],
    "2": ["Genesis 4-7"],
    ...
  }
}

Used by: reading_plans.py, /reading-plans pages


cross_references.json (50K)

Contains: Cross-reference system linking related Bible verses by theme and narrative

Structure:

{
  "John:3:16": [
    {
      "ref": "Romans 5:8",
      "note": "God's love demonstrated through Christ"
    },
    {
      "ref": "1 John 4:9",
      "note": "God's love revealed in sending His Son"
    }
  ]
}

Used by: cross_references.py, verse detail pages with cross-reference links


biblical_timeline.json (65K)

Contains: Major biblical events with dates, descriptions, and verse references

Structure:

{
  "events": [
    {
      "date": "~4004 BC",
      "title": "Creation",
      "description": "...",
      "verses": ["Genesis 1:1", ...],
      "category": "Beginning"
    }
  ]
}

Used by: routes/biblical_timeline.py, /biblical-timeline page


biographies.json (78K)

Contains: Biographical information for major biblical figures

Structure:

{
  "Abraham": {
    "name": "Abraham",
    "title": "Father of Faith",
    "born": "~2166 BC",
    "summary": "...",
    "key_verses": [...],
    "life_events": [...]
  }
}

Used by: Biography pages, family tree integration


Metadata & Configuration

chapter_explanations.json (1.9K)

Contains: Explanatory text for popular/significant Bible chapters

Structure:

{
  "John": {
    "3": "Contains John 3:16 - 'For God so loved the world'...",
    "1": "The Word became flesh - Jesus as the eternal Logos..."
  }
}

Used by: utils/helpers.py, chapter popularity explanations


Contains: Popularity scores (1-10) for chapters and high readership books

Structure:

{
  "popular_chapters": {
    "John": {"3": 10, "1": 9},
    "Psalms": {"23": 10, "91": 9}
  },
  "high_readership_books": ["Matthew", "Mark", "Luke", "John", ...]
}

Used by: utils/helpers.py, chapter popularity scoring


Contains: 31 featured verses for verse-of-the-day rotation

Structure:

{
  "verses": [
    {"book": "John", "chapter": 3, "verse": 16},
    {"book": "Psalms", "chapter": 23, "verse": 1},
    ...
  ]
}

Used by: utils/helpers.py, verse-of-the-day feature (rotates by day of year)


resource_slugs.json (1.8K)

Contains: URL slugs for all resource types used in sitemap generation

Structure:

{
  "study_guides": ["sermon-on-the-mount", "lords-prayer", ...],
  "angels": ["michael", "gabriel", "lucifer", "abaddon"],
  "prophets": ["moses", "elijah", "isaiah", ...],
  ...
}

Resource Types: study_guides (16), angels (4), prophets (8), names_of_god (10), parables (8), covenants (5), apostles (12), women (12), festivals (7), fruits_of_spirit (9)

Used by: routes/utility.py, sitemap.xml generation


verse_collections.json (11K)

Contains: Curated collections of verses grouped by theme

Structure:

{
  "comfort": {
    "title": "Verses of Comfort",
    "description": "...",
    "verses": [
      {"ref": "Psalms 23:4", "text": "..."},
      ...
    ]
  }
}

Used by: Verse collection pages, themed verse displays


Story Resources

stories/ Directory (24 files)

Contains: Biblical stories organized by category for narrative study

Files: 01_creation.json, 02_patriarchs.json, 03_job_suffering.json, etc.

Structure:

{
  "category": "Creation and Early Earth",
  "slug": "creation",
  "description": "...",
  "stories": [
    {
      "title": "The Seven Days of Creation",
      "slug": "seven-days-creation",
      "summary": "...",
      "key_verses": [...],
      "content": "...",
      "age_appropriate": ["kids", "teens", "adults"]
    }
  ]
}

Categories: Creation, Patriarchs, Job, Exodus, Conquest, Judges, Kings, Prophets, Exile, Jesus' Birth, Jesus' Ministry, Miracles, Parables, Crucifixion/Resurrection, Early Church, Paul's Journeys

Used by: stories.py, routes/stories.py, /stories pages


Editing Guidelines

JSON Format

  • Encoding: UTF-8
  • Indentation: 2 spaces (no tabs)
  • Line Endings: LF (Unix-style)
  • Trailing Commas: Not allowed in JSON
  • Quotes: Use double quotes for strings

Validation

After editing any JSON file, validate with:

# Test JSON syntax
python -m json.tool kjvstudy_org/data/your_file.json > /dev/null

# Run application tests
uv run pytest tests/ -v

Best Practices

  1. Verse References: Use consistent format: "Book Chapter:Verse" (e.g., "John 3:16")
  2. Book Names: Use full canonical names matching bible_metadata.json
  3. HTML in Content: Use <strong>, <em>, <br> for formatting (will be rendered in templates)
  4. Slugs: Use lowercase with hyphens (e.g., "sermon-on-the-mount")
  5. Hebrew/Greek: Ensure UTF-8 encoding preserved for special characters
  6. Descriptions: Keep concise, clear, and theologically accurate

Adding New Content

New Study Guide:

  1. Add to study_guides.json under appropriate category in catalog
  2. Add full content with sections to content object
  3. Add slug to resource_slugs.json under study_guides
  4. Test with: curl http://localhost:8000/study-guides/your-slug

New Resource:

  1. Add to appropriate section in resources.json
  2. Add slug to resource_slugs.json
  3. Update sitemap in routes/utility.py if needed
  4. Test resource page rendering

New Topic:

  1. Add to topics.json with description and subtopics
  2. Include verse references in standard format
  3. Test with: /topics/Your-Topic

Data Sources & Credits

Original Sources

  • Bible Text: King James Version (public domain)
  • Interlinear Data: tahmmee/interlinear_bibledata repository
  • Strong's Numbers: Strong's Exhaustive Concordance (public domain)
  • Cross References: Treasury of Scripture Knowledge (public domain)
  • Book Introductions: Compiled from public domain Bible study resources

Content Creation

  • Study Guides: Original content created for KJV Study
  • Commentary: Original theological analysis
  • Word Studies: Compiled from multiple lexical sources
  • Stories: Retold from biblical narrative text

Theological Review

All theological content has been reviewed for:

  • Biblical accuracy
  • Doctrinal soundness
  • Historical context
  • Practical application

File Sizes & Performance

Optimization Strategies

  1. Compression: interlinear.json.gz compressed with gzip (141 MB → 12 MB, 91% reduction)
  2. Lazy Loading: Interlinear data loaded on first access via interlinear_loader.py
  3. Caching: All JSON loaders use @lru_cache for single-load-per-process
  4. Separation: Large resources split into individual files (books, stories)

Loading Performance

File Size Load Time* Cache
bible_metadata.json 5.7K <1ms Yes
study_guides.json 265K ~10ms Yes
resources.json 1.3M ~50ms Yes
interlinear.json.gz 12M ~500ms Yes (lazy)

*Approximate times on modern hardware

Memory Usage

  • Total Data: ~2.1 MB uncompressed (excluding interlinear)
  • Peak Memory: ~25 MB with all data cached
  • Interlinear: ~140 MB when decompressed and cached

Version Control

Git LFS

Not currently using Git LFS, but interlinear.json.gz (12M) could benefit if repository size becomes an issue.

.gitignore

The following files are ignored:

  • __pycache__/
  • *.pyc
  • interlinear_data.py (deprecated, replaced by interlinear.json.gz)

Backup Strategy

All JSON files are version controlled in Git. For production deployments:

  1. Verify JSON integrity before commit
  2. Test application after data changes
  3. Use feature branches for major content updates
  4. Tag releases for stable data versions

Troubleshooting

Common Issues

JSON Syntax Error

# Validate JSON syntax
python -m json.tool data/your_file.json

Missing Data

  • Check file exists: ls -lh kjvstudy_org/data/
  • Check file permissions: chmod 644 kjvstudy_org/data/*.json
  • Check encoding: file kjvstudy_org/data/your_file.json (should show UTF-8)

Interlinear Data Not Loading

# Check compressed file exists
ls -lh kjvstudy_org/data/interlinear.json.gz

# Test decompression
gunzip -c kjvstudy_org/data/interlinear.json.gz | head -100

Cache Issues

  • Restart application to clear @lru_cache
  • Check for stale imports in Python
  • Verify no hardcoded data remains in .py files

Development Workflow

Typical Data Update Flow

  1. Edit JSON file in your editor with proper syntax highlighting
  2. Validate syntax: python -m json.tool data/file.json > /dev/null
  3. Test locally: uv run uvicorn kjvstudy_org.server:app --reload
  4. Run tests: uv run pytest tests/ -v
  5. Commit changes: git add data/file.json && git commit -m "Update: ..."
  6. Deploy: Changes automatically deployed via CI/CD

Testing Changes

# Test specific functionality
uv run pytest tests/test_advanced_routes.py::TestStudyGuidesRoutes -v

# Test all routes
uv run pytest tests/test_web_routes.py -v

# Test API endpoints
uv run pytest tests/test_api.py -v

Future Enhancements

Potential Additions

  • Concordance Data: Complete Strong's concordance in JSON
  • Lexicon Data: Full Hebrew/Greek lexicons
  • Maps Data: GeoJSON for biblical locations
  • Chronology Data: Detailed timeline with BCE/CE dates
  • Manuscript Data: Textual variants and manuscript information
  • Archaeological Data: Historical/archaeological context

Format Improvements

  • JSON Schema: Add JSON schema validation files
  • TypeScript Types: Generate TypeScript interfaces from JSON
  • API Versioning: Version data files for backward compatibility
  • Multilingual: Support for multiple Bible translations

Questions & Support

For questions about data structure or content:


Last Updated: 2025-01-27 Total Files: 106 JSON files (15 root-level, 66 books, 24 stories, 1 compressed) Total Size: ~2.1 MB (uncompressed) + 12 MB (interlinear compressed)