Skip to content

Log Analysis Parameter Reference

All parameters below are extracted from source where possible. Validate against --help and OpenAPI (/docs) before production use.

CLI parameters

Name Type Required Default Choices Description
--config Path optional None — YAML config path
--input Path optional None — Log file or directory (repeatable)
--output Path optional None — Output directory
--preset str optional None — Parser preset name (e.g. generic, springboot, logback, json)
--line-regex str optional None — Custom line regex (named groups: timestamp, level, message, …)
--auto-detect flag optional false — Auto-detect parser preset from log sample
--preset-dir str optional None — Directory with user preset YAML files (repeatable; also MD_LOG_PRESET_DIRS, ~/.mdengine/log/presets)
--async flag optional false — Enqueue background job (prints job_id; SQLite job store)
--export-jsonl flag optional false — Export embedding-ready JSONL chunks
--export-parquet flag optional false — Export embedding-ready Parquet chunks
--resume flag optional false — Enable incremental checkpoint resume
--frontmatter flag optional false — Emit YAML frontmatter on artifacts
--source str optional 'tail' ['tail', 'stdin', 'kafka', 'redis', 'websocket'] —
--config Path optional None — —
--input Path optional None — File path for tail source
--output Path optional None — —

API routes (HTTP parameters)

Method Path Type Required Default Description
GET /health HTTP — — FastAPI route
POST /log-to-md/run HTTP — — FastAPI route
POST /log-to-md/job HTTP — — FastAPI route
POST /log-to-md/run/upload HTTP — — FastAPI route
POST /log-to-md/job/upload HTTP — — FastAPI route
GET /log-to-md/job/{job_id} HTTP — — FastAPI route
GET /log-to-md/job/{job_id}/download HTTP — — FastAPI route
GET /log-to-md/job/{job_id}/events HTTP — — FastAPI route
GET /log-to-md/job/{job_id}/stream HTTP — — FastAPI route

Environment variables

Name Type Required Default Choices Description
LOG_TO_MD_PORT string/int optional varies — HTTP API listen port

Config files

File Type Required Default Choices Description
src\md_generator\log\config\default.yaml YAML optional — — Packaged or preset config
src\md_generator\log\config\presets\generic.yaml YAML optional — — Packaged or preset config
src\md_generator\log\config\presets\json.yaml YAML optional — — Packaged or preset config
src\md_generator\log\config\presets\logback.yaml YAML optional — — Packaged or preset config
src\md_generator\log\config\presets\springboot.yaml YAML optional — — Packaged or preset config

YAML config keys (from packaged defaults)

File Key Default / sample Required Choices Description
src\md_generator\log\config\default.yaml input.paths [] optional — From packaged YAML
src\md_generator\log\config\default.yaml input.otel_path null optional — From packaged YAML
src\md_generator\log\config\default.yaml parser.preset 'generic' optional — From packaged YAML
src\md_generator\log\config\default.yaml parser.fuzzy_timestamp False optional — From packaged YAML
src\md_generator\log\config\default.yaml parser.auto_detect False optional — From packaged YAML
src\md_generator\log\config\default.yaml parser.preset_dirs [] optional — From packaged YAML
src\md_generator\log\config\default.yaml normalization.redact_pii False optional — From packaged YAML
src\md_generator\log\config\default.yaml normalization.normalize_numbers False optional — From packaged YAML
src\md_generator\log\config\default.yaml normalization.normalize_uuid False optional — From packaged YAML
src\md_generator\log\config\default.yaml normalization.normalize_paths False optional — From packaged YAML
src\md_generator\log\config\default.yaml aggregation.timeline 'hourly' optional — From packaged YAML
src\md_generator\log\config\default.yaml clustering.enabled False optional — From packaged YAML
src\md_generator\log\config\default.yaml clustering.algorithm 'kmeans' optional — From packaged YAML
src\md_generator\log\config\default.yaml clustering.n_clusters 8 optional — From packaged YAML
src\md_generator\log\config\default.yaml clustering.random_state 42 optional — From packaged YAML
src\md_generator\log\config\default.yaml clustering.max_features 4096 optional — From packaged YAML
src\md_generator\log\config\default.yaml output.path './log-docs' optional — From packaged YAML
src\md_generator\log\config\default.yaml output.split_by_level True optional — From packaged YAML
src\md_generator\log\config\default.yaml output.generate_incidents True optional — From packaged YAML
src\md_generator\log\config\default.yaml output.generate_clusters False optional — From packaged YAML
src\md_generator\log\config\default.yaml output.generate_chunks False optional — From packaged YAML
src\md_generator\log\config\default.yaml output.frontmatter False optional — From packaged YAML
src\md_generator\log\config\default.yaml chunk.enabled False optional — From packaged YAML
src\md_generator\log\config\default.yaml chunk.lines_per_chunk 100000 optional — From packaged YAML
src\md_generator\log\config\default.yaml chunk.records_per_md_chunk 500 optional — From packaged YAML
src\md_generator\log\config\default.yaml execution.workers 4 optional — From packaged YAML
src\md_generator\log\config\default.yaml execution.max_lines_per_file null optional — From packaged YAML
src\md_generator\log\config\default.yaml execution.encoding_fallbacks 'utf-8', 'utf-8-sig', 'latin-1', 'cp1252' optional — From packaged YAML
src\md_generator\log\config\default.yaml execution.batch_records 10000 optional — From packaged YAML
src\md_generator\log\config\default.yaml execution.use_runtime False optional — From packaged YAML
src\md_generator\log\config\default.yaml execution.distributed False optional — From packaged YAML
src\md_generator\log\config\default.yaml plugins.enrichers [] optional — From packaged YAML
src\md_generator\log\config\default.yaml incidents.min_occurrences 2 optional — From packaged YAML
src\md_generator\log\config\default.yaml incidents.levels 'ERROR', 'FATAL', 'WARN' optional — From packaged YAML
src\md_generator\log\config\default.yaml incidents.stacktrace_aware True optional — From packaged YAML
src\md_generator\log\config\default.yaml chunking.enabled False optional — From packaged YAML
src\md_generator\log\config\default.yaml chunking.strategies 'incident', 'timeline', 'stacktrace', 'cluster', ... optional — From packaged YAML
src\md_generator\log\config\default.yaml chunking.chunk_id_namespace 'chunk' optional — From packaged YAML
src\md_generator\log\config\default.yaml chunking.max_chunk_bytes 256000 optional — From packaged YAML
src\md_generator\log\config\default.yaml embeddings.enabled False optional — From packaged YAML

Run config dataclass fields

No entries detected.

Options / dataclass fields (sample)

Class Field Type Required Default Description
InputSectionModel paths list[str] varies — Field on options/config class
InputSectionModel otel_path str | None varies — Field on options/config class
ParserSectionModel preset str varies — Field on options/config class
ParserSectionModel line_regex str | None varies — Field on options/config class
ParserSectionModel fuzzy_timestamp bool varies — Field on options/config class
ParserSectionModel auto_detect bool varies — Field on options/config class
ParserSectionModel preset_dirs list[str] varies — Field on options/config class
NormalizationSectionModel redact_pii bool varies — Field on options/config class
NormalizationSectionModel normalize_numbers bool varies — Field on options/config class
NormalizationSectionModel normalize_uuid bool varies — Field on options/config class
NormalizationSectionModel normalize_paths bool varies — Field on options/config class
AggregationSectionModel timeline Literal['none', 'hourly', 'daily'] varies — Field on options/config class
ClusteringSectionModel enabled bool varies — Field on options/config class
ClusteringSectionModel algorithm str varies — Field on options/config class
ClusteringSectionModel n_clusters int varies — Field on options/config class
ClusteringSectionModel random_state int varies — Field on options/config class
ClusteringSectionModel max_features int varies — Field on options/config class
OutputSectionModel path str varies — Field on options/config class
OutputSectionModel split_by_level bool varies — Field on options/config class
OutputSectionModel generate_incidents bool varies — Field on options/config class
OutputSectionModel generate_clusters bool varies — Field on options/config class
OutputSectionModel generate_chunks bool varies — Field on options/config class
OutputSectionModel frontmatter bool varies — Field on options/config class
ChunkSectionModel enabled bool varies — Field on options/config class
ChunkSectionModel lines_per_chunk int varies — Field on options/config class
ChunkSectionModel records_per_md_chunk int varies — Field on options/config class
ExecutionSectionModel workers int varies — Field on options/config class
ExecutionSectionModel max_lines_per_file int | None varies — Field on options/config class
ExecutionSectionModel encoding_fallbacks list[str] varies — Field on options/config class
ExecutionSectionModel batch_records int varies — Field on options/config class
ExecutionSectionModel use_runtime bool varies — Field on options/config class
ExecutionSectionModel distributed bool varies — Field on options/config class
IncrementalSectionModel enabled bool varies — Field on options/config class
IncrementalSectionModel checkpoint_path str | None varies — Field on options/config class
StreamingSectionModel enabled bool varies — Field on options/config class
StreamingSectionModel source str varies — Field on options/config class
StreamingSectionModel batch_size int varies — Field on options/config class
PluginsSectionModel enrichers list[str] varies — Field on options/config class
InputSection paths list[str] varies — Field on options/config class
InputSection otel_path str | None varies — Field on options/config class

Validation and edge cases

  • Positional CLI arguments are required unless documented as optional flags.
  • Upload APIs enforce size limits via environment variables (413 on exceed).
  • Missing optional dependencies raise install-time or runtime errors referencing the PyPI extra log.