Nathan Coad 32d4a352dc
All checks were successful
continuous-integration/drone/push Build is passing
reduced the places where we probe hourly tables
2026-01-21 11:44:13 +11:00
2026-01-16 12:36:53 +11:00
2026-01-14 09:28:30 +11:00
2026-01-14 09:28:30 +11:00
2026-01-15 15:53:39 +11:00
2026-01-17 12:51:11 +11:00
2024-09-12 08:57:44 +10:00
2024-09-12 08:57:44 +10:00
2026-01-15 09:57:05 +11:00
2024-09-12 08:57:44 +10:00
2026-01-15 11:43:29 +11:00
2026-01-15 11:43:29 +11:00
2024-09-12 12:47:10 +10:00
2026-01-20 17:18:43 +11:00
2026-01-13 20:09:19 +11:00
2024-09-12 08:57:44 +10:00
2024-09-12 08:57:44 +10:00
2024-09-12 08:57:44 +10:00
2024-09-12 08:57:44 +10:00
2024-09-12 08:57:44 +10:00
2026-01-16 17:37:55 +11:00

Overview

vCTP is a vSphere Chargeback Tracking Platform, designed for a specific customer, so some decisions may not be applicable for your use case.

Snapshots and Reports

  • Hourly snapshots capture inventory per vCenter (concurrency via hourly_snapshot_concurrency).
  • Daily summaries aggregate the hourly snapshots for the day; monthly summaries aggregate daily summaries for the month.
  • Snapshots are registered in snapshot_registry so regeneration via /api/snapshots/aggregate can locate the correct tables (fallback scanning is also supported).
  • Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory.
  • Prometheus metrics are exposed at /metrics:
    • Snapshots/aggregations: vctp_hourly_snapshots_total, vctp_hourly_snapshots_failed_total, vctp_hourly_snapshot_last_unix, vctp_hourly_snapshot_last_rows, vctp_daily_aggregations_total, vctp_daily_aggregations_failed_total, vctp_daily_aggregation_duration_seconds, vctp_monthly_aggregations_total, vctp_monthly_aggregations_failed_total, vctp_monthly_aggregation_duration_seconds, vctp_reports_available
    • vCenter health/perf: vctp_vcenter_connect_failures_total{vcenter}, vctp_vcenter_snapshot_duration_seconds{vcenter}, vctp_vcenter_inventory_size{vcenter}

RPM Layout (summary)

The RPM installs the service and defaults under /usr/bin, config under /etc/dtms, and data under /var/lib/vctp:

  • Binary: /usr/bin/vctp-linux-amd64
  • Systemd unit: /etc/systemd/system/vctp.service
  • Defaults/env: /etc/dtms/vctp.yml (override with -settings), /etc/default/vctp (environment)
  • TLS cert/key: /etc/dtms/vctp.crt and /etc/dtms/vctp.key (generated if absent)
  • Data: SQLite DB and reports default to /var/lib/vctp (reports under /var/lib/vctp/reports)
  • Scripts: preinstall/postinstall handle directory creation and permissions.

Settings File

Configuration now lives in the YAML settings file. By default the service reads /etc/dtms/vctp.yml, or you can override it with the -settings flag.

vctp -settings /path/to/vctp.yml

If you just want to run a single inventory snapshot across all configured vCenters and exit (no scheduler/server), use:

vctp -settings /path/to/vctp.yml -run-inventory

Database Configuration

By default the app uses SQLite and creates/opens db.sqlite3. You can opt into PostgreSQL by updating the settings file:

  • settings.database_driver: sqlite (default) or postgres
  • settings.database_url: SQLite file path/DSN or PostgreSQL DSN

Examples:

settings:
  database_driver: sqlite
  database_url: ./db.sqlite3

settings:
  database_driver: postgres
  database_url: postgres://user:pass@localhost:5432/vctp?sslmode=disable

PostgreSQL migrations live in db/migrations_postgres, while SQLite migrations remain in db/migrations.

Snapshot Retention

Hourly and daily snapshot table retention can be configured in the settings file:

  • settings.hourly_snapshot_max_age_days (default: 60)
  • settings.daily_snapshot_max_age_months (default: 12)

Settings Reference

All configuration lives under the top-level settings: key in vctp.yml.

General:

  • settings.log_level: logging verbosity (e.g., debug, info, warn, error)
  • settings.log_output: log format, text or json

Database:

  • settings.database_driver: sqlite or postgres
  • settings.database_url: SQLite file path/DSN or PostgreSQL DSN

HTTP/TLS:

  • settings.bind_ip: IP address to bind the HTTP server
  • settings.bind_port: TCP port to bind the HTTP server
  • settings.bind_disable_tls: true to serve plain HTTP (no TLS)
  • settings.tls_cert_filename: PEM certificate path (TLS mode)
  • settings.tls_key_filename: PEM private key path (TLS mode)

vCenter:

  • settings.vcenter_username: vCenter username
  • settings.vcenter_password: vCenter password (encrypted at startup)
  • settings.vcenter_insecure: true to skip TLS verification
  • settings.vcenter_event_polling_seconds: event polling interval (0 disables)
  • settings.vcenter_inventory_polling_seconds: inventory polling interval (0 disables)
  • settings.vcenter_inventory_snapshot_seconds: hourly snapshot cadence (seconds)
  • settings.vcenter_inventory_aggregate_seconds: daily aggregation cadence (seconds)
  • settings.vcenter_addresses: list of vCenter SDK URLs to monitor

Snapshots:

  • settings.hourly_snapshot_concurrency: max concurrent vCenter snapshots (0 = unlimited)
  • settings.hourly_snapshot_max_age_days: retention for hourly tables
  • settings.daily_snapshot_max_age_months: retention for daily tables
  • settings.snapshot_cleanup_cron: cron expression for cleanup job
  • settings.reports_dir: directory to store generated XLSX reports (default: /var/lib/vctp/reports)
  • settings.hourly_snapshot_retry_seconds: interval for retrying failed hourly snapshots (default: 300 seconds)
  • settings.hourly_snapshot_max_retries: maximum retry attempts per vCenter snapshot (default: 3)

Filters/chargeback:

  • settings.tenants_to_filter: list of tenant name patterns to exclude
  • settings.node_charge_clusters: list of cluster name patterns for node chargeback
  • settings.srm_activeactive_vms: list of SRM Active/Active VM name patterns

Developer setup

Pre-requisite tools

go install github.com/a-h/templ/cmd/templ@latest
go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest
go install github.com/swaggo/swag/cmd/swag@latest

Database

This project now uses goose for DB migrations.

Install via brew install goose on a mac, or install via golang with command go install github.com/pressly/goose/v3/cmd/goose@latest

Create a new up/down migration file with this command

goose -dir db/migrations sqlite3 ./db.sqlite3 create init sql
sqlc generate

HTML templates

Run templ generate -path ./components to generate code based on template files

Documentation

Run swag init --exclude "pkg.mod,pkg.build,pkg.tools" -o server/router/docs

CI/CD (Drone)

  • .drone.yml defines a Docker pipeline:
    • Restore/build caches for Go modules/tools.
    • Build step installs generators (templ, sqlc, swag), regenerates code/docs, runs project scripts, and produces the vctp-linux-amd64 binary.
    • RPM step packages via nfpm using vctp.yml, emits RPMs into ./build/.
    • Optional SFTP deploy step uploads build artifacts (e.g., vctp*) to a remote host.
    • Cache rebuild step preserves Go caches across runs.
Description
No description provided
Readme 15 MiB
Languages
Go 92.1%
templ 5.4%
Shell 2%
Dockerfile 0.2%
Makefile 0.2%