vctp2/README.md

Overview

vCTP is a vSphere Chargeback Tracking Platform, designed for a specific customer, so some decisions may not be applicable for your use case.

Snapshots and Reports

• Hourly snapshots capture inventory per vCenter (concurrency via hourly_snapshot_concurrency).
• Daily summaries aggregate the hourly snapshots for the day; monthly summaries aggregate daily summaries for the month.
• Snapshots are registered in snapshot_registry so regeneration via /api/snapshots/aggregate can locate the correct tables (fallback scanning is also supported).
• Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory.
• Prometheus metrics are exposed at /metrics:
  • Snapshots/aggregations: vctp_hourly_snapshots_total, vctp_hourly_snapshots_failed_total, vctp_hourly_snapshot_last_unix, vctp_hourly_snapshot_last_rows, vctp_daily_aggregations_total, vctp_daily_aggregations_failed_total, vctp_daily_aggregation_duration_seconds, vctp_monthly_aggregations_total, vctp_monthly_aggregations_failed_total, vctp_monthly_aggregation_duration_seconds, vctp_reports_available
  • vCenter health/perf: vctp_vcenter_connect_failures_total{vcenter}, vctp_vcenter_snapshot_duration_seconds{vcenter}, vctp_vcenter_inventory_size{vcenter}

RPM Layout (summary)

The RPM installs the service and defaults under /usr/bin, config under /etc/dtms, and data under /var/lib/vctp:

• Binary: /usr/bin/vctp-linux-amd64
• Systemd unit: /etc/systemd/system/vctp.service
• Defaults/env: /etc/dtms/vctp.yml (override with -settings), /etc/default/vctp (environment)
• TLS cert/key: /etc/dtms/vctp.crt and /etc/dtms/vctp.key (generated if absent)
• Data: SQLite DB and reports default to /var/lib/vctp (reports under /var/lib/vctp/reports)
• Scripts: preinstall/postinstall handle directory creation and permissions.

Settings File

Configuration now lives in the YAML settings file. By default the service reads /etc/dtms/vctp.yml, or you can override it with the -settings flag.

vctp -settings /path/to/vctp.yml

Database Configuration

By default the app uses SQLite and creates/opens db.sqlite3. You can opt into PostgreSQL by updating the settings file:

• settings.database_driver: sqlite (default) or postgres
• settings.database_url: SQLite file path/DSN or PostgreSQL DSN

Examples:

settings:
  database_driver: sqlite
  database_url: ./db.sqlite3

settings:
  database_driver: postgres
  database_url: postgres://user:pass@localhost:5432/vctp?sslmode=disable

PostgreSQL migrations live in db/migrations_postgres, while SQLite migrations remain in db/migrations.

Snapshot Retention

Hourly and daily snapshot table retention can be configured in the settings file:

• settings.hourly_snapshot_max_age_days (default: 60)
• settings.daily_snapshot_max_age_months (default: 12)

Settings Reference

All configuration lives under the top-level settings: key in vctp.yml.

General:

• settings.log_level: logging verbosity (e.g., debug, info, warn, error)
• settings.log_output: log format, text or json

Database:

• settings.database_driver: sqlite or postgres
• settings.database_url: SQLite file path/DSN or PostgreSQL DSN

HTTP/TLS:

• settings.bind_ip: IP address to bind the HTTP server
• settings.bind_port: TCP port to bind the HTTP server
• settings.bind_disable_tls: true to serve plain HTTP (no TLS)
• settings.tls_cert_filename: PEM certificate path (TLS mode)
• settings.tls_key_filename: PEM private key path (TLS mode)

vCenter:

• settings.vcenter_username: vCenter username
• settings.vcenter_password: vCenter password (encrypted at startup)
• settings.vcenter_insecure: true to skip TLS verification
• settings.vcenter_event_polling_seconds: event polling interval (0 disables)
• settings.vcenter_inventory_polling_seconds: inventory polling interval (0 disables)
• settings.vcenter_inventory_snapshot_seconds: hourly snapshot cadence (seconds)
• settings.vcenter_inventory_aggregate_seconds: daily aggregation cadence (seconds)
• settings.vcenter_addresses: list of vCenter SDK URLs to monitor

Snapshots:

• settings.hourly_snapshot_concurrency: max concurrent vCenter snapshots (0 = unlimited)
• settings.hourly_snapshot_max_age_days: retention for hourly tables
• settings.daily_snapshot_max_age_months: retention for daily tables
• settings.snapshot_cleanup_cron: cron expression for cleanup job
• settings.reports_dir: directory to store generated XLSX reports (default: /var/lib/vctp/reports)

Filters/chargeback:

• settings.tenants_to_filter: list of tenant name patterns to exclude
• settings.node_charge_clusters: list of cluster name patterns for node chargeback
• settings.srm_activeactive_vms: list of SRM Active/Active VM name patterns

Developer setup

Pre-requisite tools

go install github.com/a-h/templ/cmd/templ@latest
go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest
go install github.com/swaggo/swag/cmd/swag@latest

Database

This project now uses goose for DB migrations.

Install via brew install goose on a mac, or install via golang with command go install github.com/pressly/goose/v3/cmd/goose@latest

Create a new up/down migration file with this command

goose -dir db/migrations sqlite3 ./db.sqlite3 create init sql

sqlc generate

HTML templates

Run templ generate -path ./components to generate code based on template files

Documentation

Run swag init --exclude "pkg.mod,pkg.build,pkg.tools" -o server/router/docs

CI/CD (Drone)

• .drone.yml defines a Docker pipeline:
  • Restore/build caches for Go modules/tools.
  • Build step installs generators (templ, sqlc, swag), regenerates code/docs, runs project scripts, and produces the vctp-linux-amd64 binary.
  • RPM step packages via nfpm using vctp.yml, emits RPMs into ./build/.
  • Optional SFTP deploy step uploads build artifacts (e.g., vctp*) to a remote host.
  • Cache rebuild step preserves Go caches across runs.