odata-referentiel-metiers

📖 Context

This repository contains the generation pipeline for the OPT-NC job reference framework. The project transforms structured business data into DuckDB and SQLite databases, then generates AsciiDoc, PDF, EPUB, CSV exports, schema documentation, and a Hugo website.

🎯 What the Project Does

• Loads business data from src/duck.sql and src/sqlite.sql into a DuckDB and a SQLite database.
• Automatically builds a directory of professions, job families, and skills.
• Writes an AsciiDoc file (data/output/docs/referentiel_metiers.adoc).
• Produces PDF and EPUB documentation and CSV files.
• Generates a Hugo website in site/public/.

💡 Why This Project Exists

Automated generation helps maintain a single source of truth, ensures consistency across job descriptions, and reduces the risk of manual errors. By reusing a database and a Python script to produce the documentation, the reference framework can be updated simply by modifying the source data.

📂 Repository Structure

• src/duck.sql: SQL script to build the DuckDB database
• src/sqlite.sql: SQL script to build the SQLite file
• src/generate-adoc.py: Python script that reads DuckDB and generates the AsciiDoc file
• src/generate-md.py: Python script that reads DuckDB and generates the Hugo Markdown site content
• data/input/: CSV files and input data
• data/output/csv/: Generated CSV exports
• data/output/docs/: Generated documentation (AsciiDoc and PDF)
• etc/themes/pdf-theme.yml: PDF formatting theme for Asciidoctor
• Taskfile.yml: Execute automated tasks with only one command line

⚙️ Prerequisites

• Python 3.14.5+
• duckdb (1.5.3+) to load the main database
• uv (0.11.15+) to execute tasks (pip install uv)
• sqlite3 command-line interface
• schemacrawler (17.11.1+) to generate database schema documentation
• asciidoctor-pdf (2.3.15+) to convert AsciiDoc into PDF
• pandoc (3.9.0.2+) to convert AsciiDoc to PDF
• hugo (0.163.0) to generate the website
• task (3.50.0+) (not strictly necessary, but useful to avoid typing long commands)

🚀 How to Generate the Documentation files

The easiest and recommended way to install dependencies and generate all the files is to use Task. You can do everything with a single command:

task

OR, if you prefer to run the steps individually :

1. Build the DuckDB and SQLite databases:

task duckdb

2. Generate the complete documentation (using Taskfile is recommended):

task docs

3. Generate the Hugo website:

task site

This command creates the Hugo site in site/, generates the Markdown pages with src/generate-md.py, installs the Relearn theme, then builds the static website in site/public/.

📦 What the Generation Produces

• dist/ref-metiers-opt-nc.duckdb : Built DuckDB database
• dist/ref-metiers-opt-nc.sqlite : Built SQLite database
• data/output/docs/referentiel_metiers.adoc : AsciiDoc source of the reference framework
• dist/ref-metiers-opt-nc-schema.html : HTML documentation of the schema
• dist/ref-metiers-opt-nc-schema.png : PNG image of the schema
• dist/ref-metiers-opt-nc-schema.pdf : PDF format of the schema
• data/output/docs/referentiel_metiers.pdf : Final PDF document (via Asciidoctor)
• site/public/ : Generated Hugo website
• And the exported CSVs from DuckDB located in data/output/csv/

🛠️ And you can also modify the Reference Framework !

• Update the CSV sources in data/input/ or the SQL in src/duck.sql
• Run task duckdb to rebuild the database
• Run task docs to regenerate the documents
• Run task site to regenerate the Hugo website

Notes

• The src/generate-adoc.py script reads the DuckDB database and builds a structured document featuring job families, active professions, and skills classified by group.
• The src/generate-md.py script uses the same DuckDB source to generate the Hugo pages, so the website stays aligned with the PDF and EPUB documentation.
• This makes the generation reproducible and maintains a clear history of data sources and the final output.