nathan/vctp2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update monthly aggregation and docs
All checks were successful
continuous-integration/drone/push Build is passing
Details

Browse Source
This commit is contained in:
Nathan Coad
2026-01-23 15:35:10 +11:00
parent 0d509179aa
commit 73ec80bb6f
5 changed files with 41 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
README.md
View File

@@ -7,6 +7,7 @@ vCTP is a vSphere Chargeback Tracking Platform, designed for a specific customer
- Snapshots are registered in `snapshot_registry` so regeneration via `/api/snapshots/aggregate` can locate the correct tables (fallback scanning is also supported). - Snapshots are registered in `snapshot_registry` so regeneration via `/api/snapshots/aggregate` can locate the correct tables (fallback scanning is also supported).
- Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory. - Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory.
- Hourly totals in reports are interval-based: each row represents `[HH:00, HH+1:00)` and uses the first snapshot at or after the hour end (including cross-day snapshots) to prorate VM presence by creation/deletion overlap. - Hourly totals in reports are interval-based: each row represents `[HH:00, HH+1:00)` and uses the first snapshot at or after the hour end (including cross-day snapshots) to prorate VM presence by creation/deletion overlap.
- Monthly aggregation reports include a Daily Totals sheet with full-day interval labels (`YYYY-MM-DD to YYYY-MM-DD`) and prorated totals derived from daily summaries.
- Prometheus metrics are exposed at `/metrics`: - Prometheus metrics are exposed at `/metrics`:
- Snapshots/aggregations: `vctp_hourly_snapshots_total`, `vctp_hourly_snapshots_failed_total`, `vctp_hourly_snapshot_last_unix`, `vctp_hourly_snapshot_last_rows`, `vctp_daily_aggregations_total`, `vctp_daily_aggregations_failed_total`, `vctp_daily_aggregation_duration_seconds`, `vctp_monthly_aggregations_total`, `vctp_monthly_aggregations_failed_total`, `vctp_monthly_aggregation_duration_seconds`, `vctp_reports_available` - Snapshots/aggregations: `vctp_hourly_snapshots_total`, `vctp_hourly_snapshots_failed_total`, `vctp_hourly_snapshot_last_unix`, `vctp_hourly_snapshot_last_rows`, `vctp_daily_aggregations_total`, `vctp_daily_aggregations_failed_total`, `vctp_daily_aggregation_duration_seconds`, `vctp_monthly_aggregations_total`, `vctp_monthly_aggregations_failed_total`, `vctp_monthly_aggregation_duration_seconds`, `vctp_reports_available`
- vCenter health/perf: `vctp_vcenter_connect_failures_total{vcenter}`, `vctp_vcenter_snapshot_duration_seconds{vcenter}`, `vctp_vcenter_inventory_size{vcenter}` - vCenter health/perf: `vctp_vcenter_connect_failures_total{vcenter}`, `vctp_vcenter_snapshot_duration_seconds{vcenter}`, `vctp_vcenter_inventory_size{vcenter}`

37
components/views/index.templ
View File

@@ -51,31 +51,32 @@ templ Index(info BuildInfo) {
<section class="grid gap-6 lg:grid-cols-3"> <section class="grid gap-6 lg:grid-cols-3">
<div class="web2-card"> <div class="web2-card">
<h2 class="text-lg font-semibold">Overview</h2> <h2 class="text-lg font-semibold mb-2">Overview</h2>
<p class="mt-2 text-sm text-slate-600"> <p class="mt-2 text-sm text-slate-600">
vCTP is a vSphere Chargeback Tracking Platform. vCTP is a vSphere Chargeback Tracking Platform.
</p> </p>
</div> </div>
<div class="web2-card"> <div class="web2-card">
<h2 class="text-lg font-semibold">Snapshots and Reports</h2> <h2 class="text-lg font-semibold mb-2">Snapshots and Reports</h2>
<ul class="mt-3 space-y-2 text-sm text-slate-600 list-disc pl-5"> <div class="mt-3 text-sm text-slate-600 web2-paragraphs">
<li>Hourly snapshots capture inventory per vCenter (concurrency via `hourly_snapshot_concurrency`).</li> <p>Hourly snapshots capture inventory per vCenter (concurrency via <code class="web2-code">hourly_snapshot_concurrency</code>).</p>
<li>Daily summaries aggregate the hourly snapshots for the day; monthly summaries aggregate daily summaries for the month (or hourly snapshots if configured).</li> <p>Daily summaries aggregate the hourly snapshots for the day; monthly summaries aggregate daily summaries for the month (or hourly snapshots if configured).</p>
<li>Snapshots are registered in `snapshot_registry` so regeneration via `/api/snapshots/aggregate` can locate the correct tables (fallback scanning is also supported).</li> <p>Snapshots are registered in <code class="web2-code">snapshot_registry</code> so regeneration via <code class="web2-code">/api/snapshots/aggregate</code> can locate the correct tables (fallback scanning is also supported).</p>
<li>Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory.</li> <p>Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory.</p>
<li>Hourly totals are interval-based: each row represents [HH:00, HH+1:00) and uses the first snapshot at or after the hour end (including cross-day snapshots) to prorate VM presence.</li> <p>Hourly totals are interval-based: each row represents <code class="web2-code">[HH:00, HH+1:00)</code> and uses the first snapshot at or after the hour end (including cross-day snapshots) to prorate VM presence.</p>
</ul> <p>Monthly aggregation reports include a Daily Totals sheet with full-day interval labels (YYYY-MM-DD to YYYY-MM-DD) and prorated totals.</p>
</div>
</div> </div>
<div class="web2-card"> <div class="web2-card">
<h2 class="text-lg font-semibold">Prorating and Aggregation</h2> <h2 class="text-lg font-semibold mb-2">Prorating and Aggregation</h2>
<ul class="mt-3 space-y-2 text-sm text-slate-600 list-disc pl-5"> <div class="mt-3 space-y-2 text-sm text-slate-600 web2-paragraphs">
<li>SamplesPresent is the count of snapshots in which the VM appears; TotalSamples is the count of unique snapshot times for the vCenter.</li> <p>SamplesPresent is the count of snapshots in which the VM appears; TotalSamples is the count of unique snapshot times for the vCenter.</p>
<li>AvgIsPresent = SamplesPresent / TotalSamples (0 when TotalSamples is 0).</li> <p>AvgIsPresent = SamplesPresent / TotalSamples (0 when TotalSamples is 0).</p>
<li>Daily AvgVcpuCount/AvgRamGB/AvgProvisionedDisk = sum of per-sample values divided by TotalSamples (time-weighted).</li> <p>Daily AvgVcpuCount/AvgRamGB/AvgProvisionedDisk = sum of per-sample values divided by TotalSamples (time-weighted).</p>
<li>Daily pool percentages use pool hits divided by SamplesPresent, so they reflect only the time the VM existed.</li> <p>Daily pool percentages use pool hits divided by SamplesPresent, so they reflect only the time the VM existed.</p>
<li>Monthly aggregation weights daily averages by daily total samples, then divides by monthly total samples.</li> <p>Monthly aggregation weights daily averages by daily total samples, then divides by monthly total samples.</p>
<li>CreationTime is only set when vCenter provides it; otherwise it remains 0.</li> <p>CreationTime is only set when vCenter provides it; otherwise it remains 0.</p>
</ul> </div>
</div> </div>
</section> </section>
</main> </main>

2
components/views/index_templ.go
View File

@@ -86,7 +86,7 @@ func Index(info BuildInfo) templ.Component {
if templ_7745c5c3_Err != nil { if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err return templ_7745c5c3_Err
} }
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 5, "</p></div></section><section class=\"grid gap-6 lg:grid-cols-3\"><div class=\"web2-card\"><h2 class=\"text-lg font-semibold\">Overview</h2><p class=\"mt-2 text-sm text-slate-600\">vCTP is a vSphere Chargeback Tracking Platform.</p></div><div class=\"web2-card\"><h2 class=\"text-lg font-semibold\">Snapshots and Reports</h2><ul class=\"mt-3 space-y-2 text-sm text-slate-600 list-disc pl-5\"><li>Hourly snapshots capture inventory per vCenter (concurrency via `hourly_snapshot_concurrency`).</li><li>Daily summaries aggregate the hourly snapshots for the day; monthly summaries aggregate daily summaries for the month (or hourly snapshots if configured).</li><li>Snapshots are registered in `snapshot_registry` so regeneration via `/api/snapshots/aggregate` can locate the correct tables (fallback scanning is also supported).</li><li>Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory.</li></ul></div><div class=\"web2-card\"><h2 class=\"text-lg font-semibold\">Prorating and Aggregation</h2><ul class=\"mt-3 space-y-2 text-sm text-slate-600 list-disc pl-5\"><li>SamplesPresent is the count of snapshots in which the VM appears; TotalSamples is the count of unique snapshot times for the vCenter.</li><li>AvgIsPresent = SamplesPresent / TotalSamples (0 when TotalSamples is 0).</li><li>Daily AvgVcpuCount/AvgRamGB/AvgProvisionedDisk = sum of per-sample values divided by TotalSamples (time-weighted).</li><li>Daily pool percentages use pool hits divided by SamplesPresent, so they reflect only the time the VM existed.</li><li>Monthly aggregation weights daily averages by daily total samples, then divides by monthly total samples.</li><li>CreationTime is only set when vCenter provides it; otherwise it remains 0.</li></ul></div></section></main></body>") templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 5, "</p></div></section><section class=\"grid gap-6 lg:grid-cols-3\"><div class=\"web2-card\"><h2 class=\"text-lg font-semibold mb-2\">Overview</h2><p class=\"mt-2 text-sm text-slate-600\">vCTP is a vSphere Chargeback Tracking Platform.</p></div><div class=\"web2-card\"><h2 class=\"text-lg font-semibold mb-2\">Snapshots and Reports</h2><div class=\"mt-3 text-sm text-slate-600 web2-paragraphs\"><p>Hourly snapshots capture inventory per vCenter (concurrency via <code class=\"web2-code\">hourly_snapshot_concurrency</code>).</p><p>Daily summaries aggregate the hourly snapshots for the day; monthly summaries aggregate daily summaries for the month (or hourly snapshots if configured).</p><p>Snapshots are registered in <code class=\"web2-code\">snapshot_registry</code> so regeneration via <code class=\"web2-code\">/api/snapshots/aggregate</code> can locate the correct tables (fallback scanning is also supported).</p><p>Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory.</p><p>Hourly totals are interval-based: each row represents <code class=\"web2-code\">[HH:00, HH+1:00)</code> and uses the first snapshot at or after the hour end (including cross-day snapshots) to prorate VM presence.</p><p>Monthly aggregation reports include a Daily Totals sheet with full-day interval labels (YYYY-MM-DD to YYYY-MM-DD) and prorated totals.</p></div></div><div class=\"web2-card\"><h2 class=\"text-lg font-semibold mb-2\">Prorating and Aggregation</h2><div class=\"mt-3 space-y-2 text-sm text-slate-600 web2-paragraphs\"><p>SamplesPresent is the count of snapshots in which the VM appears; TotalSamples is the count of unique snapshot times for the vCenter.</p><p>AvgIsPresent = SamplesPresent / TotalSamples (0 when TotalSamples is 0).</p><p>Daily AvgVcpuCount/AvgRamGB/AvgProvisionedDisk = sum of per-sample values divided by TotalSamples (time-weighted).</p><p>Daily pool percentages use pool hits divided by SamplesPresent, so they reflect only the time the VM existed.</p><p>Monthly aggregation weights daily averages by daily total samples, then divides by monthly total samples.</p><p>CreationTime is only set when vCenter provides it; otherwise it remains 0.</p></div></div></section></main></body>")
if templ_7745c5c3_Err != nil { if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err return templ_7745c5c3_Err
} }

12
dist/assets/css/web3.css vendored
View File

@@ -61,6 +61,18 @@ body {
font-size: 0.85rem; font-size: 0.85rem;
letter-spacing: 0.02em; letter-spacing: 0.02em;
} }
.web2-code {
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace;
background: #f1f5f9;
border: 1px solid var(--web2-border);
border-radius: 3px;
padding: 0.1rem 0.35rem;
font-size: 0.85em;
color: #0f172a;
}
.web2-paragraphs p + p {
margin-top: 0.85rem;
}
.web2-link { .web2-link {
color: var(--web2-blue); color: var(--web2-blue);
text-decoration: none; text-decoration: none;

9
internal/report/snapshots.go
View File

@@ -1319,6 +1319,10 @@ func formatHourIntervalLabel(start, end time.Time) string {
return fmt.Sprintf("%s to %s", startLabel, end.Format("2006-01-02 15:04")) return fmt.Sprintf("%s to %s", startLabel, end.Format("2006-01-02 15:04"))
} }
func formatDayIntervalLabel(start, end time.Time) string {
return fmt.Sprintf("%s to %s", start.Format("2006-01-02"), end.Format("2006-01-02"))
}
func buildDailyTotals(ctx context.Context, dbConn *sqlx.DB, records []SnapshotRecord, prorateByAvg bool) ([]totalsPoint, error) { func buildDailyTotals(ctx context.Context, dbConn *sqlx.DB, records []SnapshotRecord, prorateByAvg bool) ([]totalsPoint, error) {
points := make([]totalsPoint, 0, len(records)) points := make([]totalsPoint, 0, len(records))
tinExpr := `COALESCE(SUM(CASE WHEN "Tin" IS NOT NULL THEN "Tin" ELSE 0 END) / 100.0, 0)` tinExpr := `COALESCE(SUM(CASE WHEN "Tin" IS NOT NULL THEN "Tin" ELSE 0 END) / 100.0, 0)`
@@ -1374,8 +1378,11 @@ WHERE %s
if err := dbConn.GetContext(ctx, &row, query); err != nil { if err := dbConn.GetContext(ctx, &row, query); err != nil {
return nil, err return nil, err
} }
dayTime := record.SnapshotTime.Local()
dayStart := time.Date(dayTime.Year(), dayTime.Month(), dayTime.Day(), 0, 0, 0, 0, dayTime.Location())
dayEnd := dayStart.AddDate(0, 0, 1)
points = append(points, totalsPoint{ points = append(points, totalsPoint{
Label: record.SnapshotTime.Local().Format("2006-01-02"), Label: formatDayIntervalLabel(dayStart, dayEnd),
VmCount: float64(row.VmCount), VmCount: float64(row.VmCount),
VcpuTotal: row.VcpuTotal, VcpuTotal: row.VcpuTotal,
RamTotal: row.RamTotal, RamTotal: row.RamTotal,