diff --git a/README.md b/README.md index 1970db7..8146763 100644 --- a/README.md +++ b/README.md @@ -5,6 +5,15 @@ vCTP is a vSphere Chargeback Tracking Platform, designed for a specific customer - Hourly snapshots capture inventory per vCenter (concurrency via `hourly_snapshot_concurrency`). - Daily summaries aggregate the hourly snapshots for the day; monthly summaries aggregate daily summaries for the month (or hourly snapshots if configured). - Snapshots are registered in `snapshot_registry` so regeneration via `/api/snapshots/aggregate` can locate the correct tables (fallback scanning is also supported). +- vCenter totals pages now provide two views: + - Daily Aggregated (`/vcenters/totals/daily`) for fast long-range trends. + - Hourly Detail 45d (`/vcenters/totals/hourly`) for recent granular change tracking. +- vCenter totals performance is accelerated with compact cache tables: + - `vcenter_latest_totals` (one latest row per vCenter) + - `vcenter_aggregate_totals` (hourly/daily/monthly per-vCenter totals by snapshot time) +- VM Trace now supports two modes on `/vm/trace`: + - `view=hourly` (default) for full snapshot detail + - `view=daily` for daily aggregated trend lines (using `vm_daily_rollup` when available) - Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory. - Hourly totals in reports are interval-based: each row represents `[HH:00, HH+1:00)` and uses the first snapshot at or after the hour end (including cross-day snapshots) to prorate VM presence by creation/deletion overlap. - Monthly aggregation reports include a Daily Totals sheet with full-day interval labels (`YYYY-MM-DD to YYYY-MM-DD`) and prorated totals derived from daily summaries. @@ -87,6 +96,11 @@ If you want a one-time cache backfill for the vCenter totals cache tables vctp -settings /path/to/vctp.yml -backfill-vcenter-cache ``` +The backfill command: +- Ensures/migrates `snapshot_registry` when needed. +- Rebuilds hourly/latest vCenter totals caches. +- Recomputes daily/monthly rows for `vcenter_aggregate_totals` from registered summary snapshots. + ## Database Configuration By default the app uses SQLite and creates/opens `db.sqlite3`. diff --git a/components/views/index.templ b/components/views/index.templ index a876c66..2b0a911 100644 --- a/components/views/index.templ +++ b/components/views/index.templ @@ -55,6 +55,9 @@ templ Index(info BuildInfo) {

vCTP is a vSphere Chargeback Tracking Platform.

+

+ Use fast vCenter totals views (Daily Aggregated and Hourly Detail 45d) and VM Trace views (Hourly Detail and Daily Aggregated) to move between long-range trends and granular timelines. +

Snapshots and Reports

@@ -64,6 +67,8 @@ templ Index(info BuildInfo) {

Daily tracks: SamplesPresent, TotalSamples, AvgIsPresent, AvgVcpuCount, AvgRamGB, AvgProvisionedDisk, PoolTinPct, PoolBronzePct, PoolSilverPct, PoolGoldPct, plus chargeback totals columns Tin, Bronze, Silver, Gold.

Monthly tracks: the same daily aggregate fields, with monthly values weighted by per-day sample volume so partial-day VMs and config changes stay proportional.

Snapshots are registered in snapshot_registry so regeneration via /api/snapshots/aggregate can locate the correct tables (fallback scanning is also supported).

+

vCenter totals pages are accelerated by compact cache tables: vcenter_latest_totals and vcenter_aggregate_totals.

+

VM Trace daily mode uses the vm_daily_rollup cache when available, and falls back to daily summary tables if needed.

Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory.

Hourly totals are interval-based: each row represents [HH:00, HH+1:00) and uses the first snapshot at or after the hour end (including cross-day snapshots) to prorate VM presence.

Monthly aggregation reports include a Daily Totals sheet with full-day interval labels (YYYY-MM-DD to YYYY-MM-DD) and prorated totals.

diff --git a/components/views/index_templ.go b/components/views/index_templ.go index dfc690c..b2d212b 100644 --- a/components/views/index_templ.go +++ b/components/views/index_templ.go @@ -86,7 +86,7 @@ func Index(info BuildInfo) templ.Component { if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 5, "

Overview

vCTP is a vSphere Chargeback Tracking Platform.

Snapshots and Reports

Hourly snapshots capture inventory per vCenter (concurrency via hourly_snapshot_concurrency), then daily and monthly summaries are derived from those snapshots.

Hourly tracks: VM identity (InventoryId, Name, VmId, VmUuid, Vcenter, EventKey, CloudId), lifecycle (CreationTime, DeletionTime, SnapshotTime), placement (Datacenter, Cluster, Folder, ResourcePool), and sizing/state (VcpuCount, RamGB, ProvisionedDisk, PoweredOn, IsTemplate, SrmPlaceholder).

Daily tracks: SamplesPresent, TotalSamples, AvgIsPresent, AvgVcpuCount, AvgRamGB, AvgProvisionedDisk, PoolTinPct, PoolBronzePct, PoolSilverPct, PoolGoldPct, plus chargeback totals columns Tin, Bronze, Silver, Gold.

Monthly tracks: the same daily aggregate fields, with monthly values weighted by per-day sample volume so partial-day VMs and config changes stay proportional.

Snapshots are registered in snapshot_registry so regeneration via /api/snapshots/aggregate can locate the correct tables (fallback scanning is also supported).

Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory.

Hourly totals are interval-based: each row represents [HH:00, HH+1:00) and uses the first snapshot at or after the hour end (including cross-day snapshots) to prorate VM presence.

Monthly aggregation reports include a Daily Totals sheet with full-day interval labels (YYYY-MM-DD to YYYY-MM-DD) and prorated totals.

Prorating and Aggregation

SamplesPresent is the count of snapshots in which the VM appears; TotalSamples is the count of unique snapshot times for that vCenter/day.

AvgIsPresent = SamplesPresent / TotalSamples (0 when TotalSamples is 0).

Daily AvgVcpuCount, AvgRamGB, and AvgProvisionedDisk are per-sample sums divided by TotalSamples (time-weighted).

Daily pool percentages (PoolTinPct/PoolBronzePct/PoolSilverPct/PoolGoldPct) use pool-hit counts divided by SamplesPresent.

Monthly aggregation converts each day into weighted sums using sample volume, then recomputes monthly averages and pool percentages from those weighted totals.

CreationTime is only set when vCenter provides it; otherwise it remains 0.

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 5, "

Overview

vCTP is a vSphere Chargeback Tracking Platform.

Use fast vCenter totals views (Daily Aggregated and Hourly Detail 45d) and VM Trace views (Hourly Detail and Daily Aggregated) to move between long-range trends and granular timelines.

Snapshots and Reports

Hourly snapshots capture inventory per vCenter (concurrency via hourly_snapshot_concurrency), then daily and monthly summaries are derived from those snapshots.

Hourly tracks: VM identity (InventoryId, Name, VmId, VmUuid, Vcenter, EventKey, CloudId), lifecycle (CreationTime, DeletionTime, SnapshotTime), placement (Datacenter, Cluster, Folder, ResourcePool), and sizing/state (VcpuCount, RamGB, ProvisionedDisk, PoweredOn, IsTemplate, SrmPlaceholder).

Daily tracks: SamplesPresent, TotalSamples, AvgIsPresent, AvgVcpuCount, AvgRamGB, AvgProvisionedDisk, PoolTinPct, PoolBronzePct, PoolSilverPct, PoolGoldPct, plus chargeback totals columns Tin, Bronze, Silver, Gold.

Monthly tracks: the same daily aggregate fields, with monthly values weighted by per-day sample volume so partial-day VMs and config changes stay proportional.

Snapshots are registered in snapshot_registry so regeneration via /api/snapshots/aggregate can locate the correct tables (fallback scanning is also supported).

vCenter totals pages are accelerated by compact cache tables: vcenter_latest_totals and vcenter_aggregate_totals.

VM Trace daily mode uses the vm_daily_rollup cache when available, and falls back to daily summary tables if needed.

Reports (XLSX with totals/charts) are generated automatically after hourly, daily, and monthly jobs and written to a reports directory.

Hourly totals are interval-based: each row represents [HH:00, HH+1:00) and uses the first snapshot at or after the hour end (including cross-day snapshots) to prorate VM presence.

Monthly aggregation reports include a Daily Totals sheet with full-day interval labels (YYYY-MM-DD to YYYY-MM-DD) and prorated totals.

Prorating and Aggregation

SamplesPresent is the count of snapshots in which the VM appears; TotalSamples is the count of unique snapshot times for that vCenter/day.

AvgIsPresent = SamplesPresent / TotalSamples (0 when TotalSamples is 0).

Daily AvgVcpuCount, AvgRamGB, and AvgProvisionedDisk are per-sample sums divided by TotalSamples (time-weighted).

Daily pool percentages (PoolTinPct/PoolBronzePct/PoolSilverPct/PoolGoldPct) use pool-hit counts divided by SamplesPresent.

Monthly aggregation converts each day into weighted sums using sample volume, then recomputes monthly averages and pool percentages from those weighted totals.

CreationTime is only set when vCenter provides it; otherwise it remains 0.

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } diff --git a/components/views/vm_trace.templ b/components/views/vm_trace.templ index 35781a3..1bb84ec 100644 --- a/components/views/vm_trace.templ +++ b/components/views/vm_trace.templ @@ -24,7 +24,16 @@ type VmTraceChart struct { ConfigJSON string } -templ VmTracePage(query string, display_query string, vm_id string, vm_uuid string, vm_name string, creationLabel string, deletionLabel string, creationApprox bool, entries []VmTraceEntry, chart VmTraceChart) { +type VmTraceMeta struct { + ViewType string + TypeLabel string + HourlyLink string + DailyLink string + HourlyClass string + DailyClass string +} + +templ VmTracePage(query string, display_query string, vm_id string, vm_uuid string, vm_name string, creationLabel string, deletionLabel string, creationApprox bool, entries []VmTraceEntry, chart VmTraceChart, meta VmTraceMeta) { @core.Header() @@ -35,13 +44,14 @@ templ VmTracePage(query string, display_query string, vm_id string, vm_uuid stri
VM Trace

Snapshot history{display_query}

-

Timeline of vCPU, RAM, and resource pool changes across snapshots.

+

Timeline of vCPU, RAM, and resource pool changes across { meta.TypeLabel } snapshots.

Dashboard
+
@@ -59,11 +69,15 @@ templ VmTracePage(query string, display_query string, vm_id string, vm_uuid stri Clear
+
+ Hourly Detail + Daily Aggregated +
-

Snapshot Timeline

+

{ meta.TypeLabel } Timeline

{len(entries)} samples
if chart.ConfigJSON != "" { diff --git a/components/views/vm_trace_templ.go b/components/views/vm_trace_templ.go index 76da532..bb36e88 100644 --- a/components/views/vm_trace_templ.go +++ b/components/views/vm_trace_templ.go @@ -32,7 +32,16 @@ type VmTraceChart struct { ConfigJSON string } -func VmTracePage(query string, display_query string, vm_id string, vm_uuid string, vm_name string, creationLabel string, deletionLabel string, creationApprox bool, entries []VmTraceEntry, chart VmTraceChart) templ.Component { +type VmTraceMeta struct { + ViewType string + TypeLabel string + HourlyLink string + DailyLink string + HourlyClass string + DailyClass string +} + +func VmTracePage(query string, display_query string, vm_id string, vm_uuid string, vm_name string, creationLabel string, deletionLabel string, creationApprox bool, entries []VmTraceEntry, chart VmTraceChart, meta VmTraceMeta) templ.Component { return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil { @@ -68,251 +77,360 @@ func VmTracePage(query string, display_query string, vm_id string, vm_uuid strin var templ_7745c5c3_Var2 string templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(display_query) if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 37, Col: 74} + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 46, Col: 74} } _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "

Timeline of vCPU, RAM, and resource pool changes across snapshots.

Dashboard

Timeline of vCPU, RAM, and resource pool changes across ") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } var templ_7745c5c3_Var3 string - templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(vm_id) + templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(meta.TypeLabel) if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 47, Col: 123} + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 47, Col: 119} } _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var3)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 4, "\" placeholder=\"vm-12345\">

Dashboard
Clear

Snapshot Timeline

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 6, "\" placeholder=\"vm-12345\">
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 7, "\" placeholder=\"uuid...\">
") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } + var templ_7745c5c3_Var7 string + templ_7745c5c3_Var7, templ_7745c5c3_Err = templ.JoinStringErrs(vm_name) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 65, Col: 123} } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 10, "

Creation time

") + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var7)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - var templ_7745c5c3_Var8 string - templ_7745c5c3_Var8, templ_7745c5c3_Err = templ.JoinStringErrs(creationLabel) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 86, Col: 76} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var8)) + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 8, "\" placeholder=\"VM name\">

Clear
") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 11, "

") + var templ_7745c5c3_Var8 = []any{meta.HourlyClass} + templ_7745c5c3_Err = templ.RenderCSSItems(ctx, templ_7745c5c3_Buffer, templ_7745c5c3_Var8...) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - if creationApprox { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 12, "

Approximate (earliest snapshot)

") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 13, "

Deletion time

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 9, "

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 10, "\" href=\"") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - for _, e := range entries { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 15, "") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 18, "\">
") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 25, "
SnapshotVM NameVmIdVmUuidVcenterResource PoolvCPUsRAM (GB)Disk
") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var10 string - templ_7745c5c3_Var10, templ_7745c5c3_Err = templ.JoinStringErrs(e.Snapshot) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 114, Col: 25} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var10)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 16, "") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var11 string - templ_7745c5c3_Var11, templ_7745c5c3_Err = templ.JoinStringErrs(e.Name) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 115, Col: 21} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var11)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 17, "") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var12 string - templ_7745c5c3_Var12, templ_7745c5c3_Err = templ.JoinStringErrs(e.VmId) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 116, Col: 21} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var12)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 18, "") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var13 string - templ_7745c5c3_Var13, templ_7745c5c3_Err = templ.JoinStringErrs(e.VmUuid) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 117, Col: 23} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var13)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 19, "") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var14 string - templ_7745c5c3_Var14, templ_7745c5c3_Err = templ.JoinStringErrs(e.Vcenter) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 118, Col: 24} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var14)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 20, "") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var15 string - templ_7745c5c3_Var15, templ_7745c5c3_Err = templ.JoinStringErrs(e.ResourcePool) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 119, Col: 29} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var15)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 21, "") + var templ_7745c5c3_Var10 templ.SafeURL + templ_7745c5c3_Var10, templ_7745c5c3_Err = templ.JoinURLErrs(meta.HourlyLink) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 73, Col: 59} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var10)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 11, "\">Hourly Detail ") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var11 = []any{meta.DailyClass} + templ_7745c5c3_Err = templ.RenderCSSItems(ctx, templ_7745c5c3_Buffer, templ_7745c5c3_Var11...) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 12, "Daily Aggregated

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var14 string + templ_7745c5c3_Var14, templ_7745c5c3_Err = templ.JoinStringErrs(meta.TypeLabel) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 80, Col: 56} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var14)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 15, " Timeline

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var15 string + templ_7745c5c3_Var15, templ_7745c5c3_Err = templ.JoinStringErrs(len(entries)) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 81, Col: 44} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var15)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 16, " samples
") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + if chart.ConfigJSON != "" { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 17, "
") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var17 string - templ_7745c5c3_Var17, templ_7745c5c3_Err = templ.JoinStringErrs(e.RamGB) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 121, Col: 41} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var17)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 23, "") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var18 string - templ_7745c5c3_Var18, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%.1f", e.ProvisionedDisk)) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 122, Col: 72} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var18)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 24, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 19, "

Creation time

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var17 string + templ_7745c5c3_Var17, templ_7745c5c3_Err = templ.JoinStringErrs(creationLabel) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 100, Col: 76} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var17)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 20, "

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + if creationApprox { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 21, "

Approximate (earliest snapshot)

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 22, "

Deletion time

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var18 string + templ_7745c5c3_Var18, templ_7745c5c3_Err = templ.JoinStringErrs(deletionLabel) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 107, Col: 76} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var18)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 23, "

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + for _, e := range entries { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 24, "") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 34, "
SnapshotVM NameVmIdVmUuidVcenterResource PoolvCPUsRAM (GB)Disk
") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var19 string + templ_7745c5c3_Var19, templ_7745c5c3_Err = templ.JoinStringErrs(e.Snapshot) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 128, Col: 25} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var19)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 25, "") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var20 string + templ_7745c5c3_Var20, templ_7745c5c3_Err = templ.JoinStringErrs(e.Name) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 129, Col: 21} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var20)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 26, "") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var21 string + templ_7745c5c3_Var21, templ_7745c5c3_Err = templ.JoinStringErrs(e.VmId) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 130, Col: 21} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var21)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 27, "") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var22 string + templ_7745c5c3_Var22, templ_7745c5c3_Err = templ.JoinStringErrs(e.VmUuid) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 131, Col: 23} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var22)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 28, "") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var23 string + templ_7745c5c3_Var23, templ_7745c5c3_Err = templ.JoinStringErrs(e.Vcenter) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 132, Col: 24} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var23)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 29, "") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var24 string + templ_7745c5c3_Var24, templ_7745c5c3_Err = templ.JoinStringErrs(e.ResourcePool) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 133, Col: 29} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var24)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 30, "") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var25 string + templ_7745c5c3_Var25, templ_7745c5c3_Err = templ.JoinStringErrs(e.VcpuCount) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 134, Col: 45} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var25)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 31, "") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var26 string + templ_7745c5c3_Var26, templ_7745c5c3_Err = templ.JoinStringErrs(e.RamGB) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 135, Col: 41} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var26)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 32, "") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var27 string + templ_7745c5c3_Var27, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%.1f", e.ProvisionedDisk)) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `views/vm_trace.templ`, Line: 136, Col: 72} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var27)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 33, "
") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -320,7 +438,7 @@ func VmTracePage(query string, display_query string, vm_id string, vm_uuid strin if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 26, "") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 35, "") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } diff --git a/db/helpers.go b/db/helpers.go index 429bb93..94cc715 100644 --- a/db/helpers.go +++ b/db/helpers.go @@ -605,6 +605,7 @@ CREATE TABLE IF NOT EXISTS vm_hourly_stats ( } _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_hourly_stats_vmuuid_time_idx ON vm_hourly_stats ("VmUuid","SnapshotTime")`) _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_hourly_stats_vmid_time_idx ON vm_hourly_stats ("VmId","SnapshotTime")`) + _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_hourly_stats_name_time_idx ON vm_hourly_stats (lower("Name"),"SnapshotTime")`) _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_hourly_stats_snapshottime_idx ON vm_hourly_stats ("SnapshotTime")`) return nil } @@ -627,6 +628,8 @@ CREATE TABLE IF NOT EXISTS vm_lifecycle_cache ( return err } _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_lifecycle_cache_vmuuid_idx ON vm_lifecycle_cache ("VmUuid")`) + _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_lifecycle_cache_vmid_idx ON vm_lifecycle_cache ("VmId")`) + _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_lifecycle_cache_name_idx ON vm_lifecycle_cache (lower("Name"))`) return nil } @@ -947,6 +950,9 @@ CREATE TABLE IF NOT EXISTS vm_daily_rollup ( } _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_daily_rollup_date_idx ON vm_daily_rollup ("Date")`) _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_daily_rollup_vcenter_date_idx ON vm_daily_rollup ("Vcenter","Date")`) + _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_daily_rollup_vmid_date_idx ON vm_daily_rollup ("VmId","Date")`) + _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_daily_rollup_vmuuid_date_idx ON vm_daily_rollup ("VmUuid","Date")`) + _, _ = execLog(ctx, dbConn, `CREATE INDEX IF NOT EXISTS vm_daily_rollup_name_date_idx ON vm_daily_rollup (lower("Name"),"Date")`) return nil } @@ -1666,6 +1672,22 @@ type VmLifecycle struct { DeletionTime int64 } +func vmLookupPredicate(vmID, vmUUID, name string) (string, []interface{}, bool) { + vmID = strings.TrimSpace(vmID) + vmUUID = strings.TrimSpace(vmUUID) + name = strings.TrimSpace(name) + switch { + case vmID != "": + return `"VmId" = ?`, []interface{}{vmID}, true + case vmUUID != "": + return `"VmUuid" = ?`, []interface{}{vmUUID}, true + case name != "": + return `lower("Name") = ?`, []interface{}{strings.ToLower(name)}, true + default: + return "", nil, false + } +} + // FetchVmTrace returns combined hourly snapshot records for a VM (by id/uuid/name) ordered by snapshot time. // It prefers the shared vm_hourly_stats history table and falls back to per-snapshot tables. func FetchVmTrace(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name string) ([]VmTraceRow, error) { @@ -1681,24 +1703,142 @@ func FetchVmTrace(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name strin return fetchVmTraceFromSnapshotTables(ctx, dbConn, vmID, vmUUID, name) } +// FetchVmTraceDaily returns one row per day for a VM, preferring vm_daily_rollup cache. +func FetchVmTraceDaily(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name string) ([]VmTraceRow, error) { + if TableExists(ctx, dbConn, "vm_daily_rollup") { + if err := EnsureVmDailyRollup(ctx, dbConn); err != nil { + slog.Warn("failed to ensure vm_daily_rollup indexes", "error", err) + } + rows, err := fetchVmTraceDailyFromRollup(ctx, dbConn, vmID, vmUUID, name) + if err != nil { + slog.Warn("vm daily trace cache query failed; falling back to daily summary tables", "error", err) + } else if len(rows) > 0 { + slog.Debug("vm daily trace loaded from daily rollup cache", "row_count", len(rows)) + return rows, nil + } + } + return fetchVmTraceDailyFromSummaryTables(ctx, dbConn, vmID, vmUUID, name) +} + +func fetchVmTraceDailyFromRollup(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name string) ([]VmTraceRow, error) { + matchWhere, args, ok := vmLookupPredicate(vmID, vmUUID, name) + if !ok { + return nil, nil + } + query := fmt.Sprintf(` +SELECT "Date" AS "SnapshotTime", + COALESCE("Name",'') AS "Name", + COALESCE("Vcenter",'') AS "Vcenter", + COALESCE("VmId",'') AS "VmId", + COALESCE("VmUuid",'') AS "VmUuid", + COALESCE("LastResourcePool",'') AS "ResourcePool", + CAST(CASE + WHEN COALESCE("SamplesPresent",0) > 0 THEN ROUND(1.0 * COALESCE("SumVcpu",0) / "SamplesPresent") + ELSE COALESCE("LastVcpuCount",0) + END AS BIGINT) AS "VcpuCount", + CAST(CASE + WHEN COALESCE("SamplesPresent",0) > 0 THEN ROUND(1.0 * COALESCE("SumRam",0) / "SamplesPresent") + ELSE COALESCE("LastRamGB",0) + END AS BIGINT) AS "RamGB", + COALESCE("LastProvisionedDisk",0) AS "ProvisionedDisk", + COALESCE("CreationTime",0) AS "CreationTime", + COALESCE("DeletionTime",0) AS "DeletionTime" +FROM vm_daily_rollup +WHERE %s +ORDER BY "Date" +`, matchWhere) + query = dbConn.Rebind(query) + rows := make([]VmTraceRow, 0, 128) + if err := selectLog(ctx, dbConn, &rows, query, args...); err != nil { + return nil, err + } + return rows, nil +} + +func fetchVmTraceDailyFromSummaryTables(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name string) ([]VmTraceRow, error) { + matchWhere, args, ok := vmLookupPredicate(vmID, vmUUID, name) + if !ok { + return nil, nil + } + if !TableExists(ctx, dbConn, "snapshot_registry") { + return nil, nil + } + var tables []struct { + TableName string `db:"table_name"` + SnapshotTime int64 `db:"snapshot_time"` + } + if err := selectLog(ctx, dbConn, &tables, ` +SELECT table_name, snapshot_time +FROM snapshot_registry +WHERE snapshot_type = 'daily' +ORDER BY snapshot_time +`); err != nil { + return nil, err + } + if len(tables) == 0 { + return nil, nil + } + + rows := make([]VmTraceRow, 0, len(tables)) + for _, t := range tables { + if err := ValidateTableName(t.TableName); err != nil { + continue + } + query := fmt.Sprintf(` +SELECT %d AS "SnapshotTime", + COALESCE("Name",'') AS "Name", + COALESCE("Vcenter",'') AS "Vcenter", + COALESCE("VmId",'') AS "VmId", + COALESCE("VmUuid",'') AS "VmUuid", + COALESCE("ResourcePool",'') AS "ResourcePool", + CAST(COALESCE("AvgVcpuCount","VcpuCount",0) AS BIGINT) AS "VcpuCount", + CAST(COALESCE("AvgRamGB","RamGB",0) AS BIGINT) AS "RamGB", + COALESCE("AvgProvisionedDisk","ProvisionedDisk",0) AS "ProvisionedDisk", + COALESCE("CreationTime",0) AS "CreationTime", + COALESCE("DeletionTime",0) AS "DeletionTime" +FROM %s +WHERE %s +`, t.SnapshotTime, t.TableName, matchWhere) + query = dbConn.Rebind(query) + var tmp []VmTraceRow + if err := selectLog(ctx, dbConn, &tmp, query, args...); err != nil { + continue + } + rows = append(rows, tmp...) + } + + sort.Slice(rows, func(i, j int) bool { + return rows[i].SnapshotTime < rows[j].SnapshotTime + }) + return rows, nil +} + func fetchVmTraceFromHourlyCache(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name string) ([]VmTraceRow, error) { - query := ` + matchWhere, args, ok := vmLookupPredicate(vmID, vmUUID, name) + if !ok { + return nil, nil + } + query := fmt.Sprintf(` SELECT "SnapshotTime","Name","Vcenter","VmId","VmUuid","ResourcePool","VcpuCount","RamGB","ProvisionedDisk", COALESCE("CreationTime",0) AS "CreationTime", COALESCE("DeletionTime",0) AS "DeletionTime" FROM vm_hourly_stats -WHERE ("VmId" = ? OR "VmUuid" = ? OR lower("Name") = lower(?)) +WHERE %s ORDER BY "SnapshotTime" -` +`, matchWhere) query = dbConn.Rebind(query) var rows []VmTraceRow - if err := selectLog(ctx, dbConn, &rows, query, vmID, vmUUID, name); err != nil { + if err := selectLog(ctx, dbConn, &rows, query, args...); err != nil { return nil, err } return rows, nil } func fetchVmTraceFromSnapshotTables(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name string) ([]VmTraceRow, error) { + matchWhere, args, ok := vmLookupPredicate(vmID, vmUUID, name) + if !ok { + return nil, nil + } var tables []struct { TableName string `db:"table_name"` SnapshotTime int64 `db:"snapshot_time"` @@ -1716,7 +1856,6 @@ ORDER BY snapshot_time } rows := make([]VmTraceRow, 0, len(tables)) - driver := strings.ToLower(dbConn.DriverName()) slog.Debug("vm trace scanning tables", "table_count", len(tables), "vm_id", vmID, "vm_uuid", vmUUID, "name", name) @@ -1731,14 +1870,9 @@ SELECT %d AS "SnapshotTime", COALESCE("CreationTime",0) AS "CreationTime", COALESCE("DeletionTime",0) AS "DeletionTime" FROM %s -WHERE ("VmId" = ? OR "VmUuid" = ? OR lower("Name") = lower(?)) -`, t.SnapshotTime, t.TableName) - args := []interface{}{vmID, vmUUID, name} - if driver != "sqlite" { - query = strings.Replace(query, "?", "$1", 1) - query = strings.Replace(query, "?", "$2", 1) - query = strings.Replace(query, "?", "$3", 1) - } +WHERE %s +`, t.SnapshotTime, t.TableName, matchWhere) + query = dbConn.Rebind(query) var tmp []VmTraceRow if err := selectLog(ctx, dbConn, &tmp, query, args...); err != nil { slog.Warn("vm trace query failed for table", "table", t.TableName, "error", err) @@ -1778,6 +1912,10 @@ func FetchVmLifecycle(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name s } func fetchVmLifecycleFromHourlyCache(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name string) (VmLifecycle, bool, error) { + matchWhere, args, ok := vmLookupPredicate(vmID, vmUUID, name) + if !ok { + return VmLifecycle{}, false, nil + } var row struct { Rows int64 `db:"rows"` Creation sql.NullInt64 `db:"creation_time"` @@ -1793,10 +1931,10 @@ SELECT MAX("SnapshotTime") AS last_seen, MIN(NULLIF("DeletionTime",0)) AS deletion_time FROM vm_hourly_stats -WHERE ("VmId" = ? OR "VmUuid" = ? OR lower("Name") = lower(?)) +WHERE ` + matchWhere + ` ` query = dbConn.Rebind(query) - if err := getLog(ctx, dbConn, &row, query, vmID, vmUUID, name); err != nil { + if err := getLog(ctx, dbConn, &row, query, args...); err != nil { return VmLifecycle{}, false, err } if row.Rows == 0 { @@ -1820,6 +1958,10 @@ WHERE ("VmId" = ? OR "VmUuid" = ? OR lower("Name") = lower(?)) } func fetchVmLifecycleFromLifecycleCache(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name string) (VmLifecycle, bool, error) { + matchWhere, args, ok := vmLookupPredicate(vmID, vmUUID, name) + if !ok { + return VmLifecycle{}, false, nil + } var row struct { Rows int64 `db:"rows"` FirstSeen sql.NullInt64 `db:"first_seen"` @@ -1833,10 +1975,10 @@ SELECT MAX(NULLIF("LastSeen",0)) AS last_seen, MIN(NULLIF("DeletedAt",0)) AS deletion_time FROM vm_lifecycle_cache -WHERE ("VmId" = ? OR "VmUuid" = ? OR lower("Name") = lower(?)) +WHERE ` + matchWhere + ` ` query = dbConn.Rebind(query) - if err := getLog(ctx, dbConn, &row, query, vmID, vmUUID, name); err != nil { + if err := getLog(ctx, dbConn, &row, query, args...); err != nil { return VmLifecycle{}, false, err } if row.Rows == 0 { @@ -1884,6 +2026,10 @@ func mergeVmLifecycle(base, overlay VmLifecycle) VmLifecycle { func fetchVmLifecycleFromSnapshotTables(ctx context.Context, dbConn *sqlx.DB, vmID, vmUUID, name string) (VmLifecycle, error) { var lifecycle VmLifecycle + matchWhere, args, ok := vmLookupPredicate(vmID, vmUUID, name) + if !ok { + return lifecycle, nil + } var tables []struct { TableName string `db:"table_name"` SnapshotTime int64 `db:"snapshot_time"` @@ -1896,7 +2042,6 @@ ORDER BY snapshot_time `); err != nil { return lifecycle, err } - driver := strings.ToLower(dbConn.DriverName()) minCreation := int64(0) consecutiveMissing := 0 @@ -1907,14 +2052,9 @@ ORDER BY snapshot_time query := fmt.Sprintf(` SELECT MIN(NULLIF("CreationTime",0)) AS min_creation, COUNT(1) AS cnt FROM %s -WHERE ("VmId" = ? OR "VmUuid" = ? OR lower("Name") = lower(?)) -`, t.TableName) - args := []interface{}{vmID, vmUUID, name} - if driver != "sqlite" { - query = strings.Replace(query, "?", "$1", 1) - query = strings.Replace(query, "?", "$2", 1) - query = strings.Replace(query, "?", "$3", 1) - } +WHERE %s +`, t.TableName, matchWhere) + query = dbConn.Rebind(query) var probe struct { MinCreation sql.NullInt64 `db:"min_creation"` Cnt int64 `db:"cnt"` diff --git a/db/helpers_cache_and_index_test.go b/db/helpers_cache_and_index_test.go index f8b408e..d910c3d 100644 --- a/db/helpers_cache_and_index_test.go +++ b/db/helpers_cache_and_index_test.go @@ -126,6 +126,20 @@ INSERT INTO vm_hourly_stats ( if traceRows[0].SnapshotTime != 1000 || traceRows[1].SnapshotTime != 2000 { t.Fatalf("trace rows are not sorted by snapshot time: %#v", traceRows) } + traceRowsByName, err := FetchVmTrace(ctx, dbConn, "", "", "DEMO-VM") + if err != nil { + t.Fatalf("FetchVmTrace by name failed: %v", err) + } + if len(traceRowsByName) != 2 { + t.Fatalf("expected 2 trace rows by name, got %d", len(traceRowsByName)) + } + emptyTraceRows, err := FetchVmTrace(ctx, dbConn, "", "", "") + if err != nil { + t.Fatalf("FetchVmTrace with empty identifier failed: %v", err) + } + if len(emptyTraceRows) != 0 { + t.Fatalf("expected 0 trace rows for empty identifier, got %d", len(emptyTraceRows)) + } lifecycle, err := FetchVmLifecycle(ctx, dbConn, "vm-1", "", "") if err != nil { @@ -143,6 +157,125 @@ INSERT INTO vm_hourly_stats ( if lifecycle.DeletionTime != 2500 { t.Fatalf("expected DeletionTime=2500 from lifecycle cache, got %d", lifecycle.DeletionTime) } + lifecycleByName, err := FetchVmLifecycle(ctx, dbConn, "", "", "DEMO-VM") + if err != nil { + t.Fatalf("FetchVmLifecycle by name failed: %v", err) + } + if lifecycleByName.FirstSeen != 900 || lifecycleByName.LastSeen != 2000 { + t.Fatalf("unexpected lifecycle for name lookup: %#v", lifecycleByName) + } + emptyLifecycle, err := FetchVmLifecycle(ctx, dbConn, "", "", "") + if err != nil { + t.Fatalf("FetchVmLifecycle with empty identifier failed: %v", err) + } + if emptyLifecycle.FirstSeen != 0 || emptyLifecycle.LastSeen != 0 || emptyLifecycle.CreationTime != 0 || emptyLifecycle.DeletionTime != 0 { + t.Fatalf("expected empty lifecycle for empty identifier, got %#v", emptyLifecycle) + } +} + +func TestFetchVmTraceDailyFromRollup(t *testing.T) { + ctx := context.Background() + dbConn := newTestSQLiteDB(t) + + if err := EnsureVmDailyRollup(ctx, dbConn); err != nil { + t.Fatalf("failed to ensure vm_daily_rollup: %v", err) + } + if err := UpsertVmDailyRollup(ctx, dbConn, 1700000000, VmDailyRollupRow{ + Vcenter: "vc-a", + VmId: "vm-1", + VmUuid: "uuid-1", + Name: "demo-vm", + CreationTime: 1699999000, + SamplesPresent: 8, + SumVcpu: 32, + SumRam: 64, + LastVcpuCount: 4, + LastRamGB: 8, + LastResourcePool: "Tin", + }); err != nil { + t.Fatalf("failed to insert daily rollup row 1: %v", err) + } + if err := UpsertVmDailyRollup(ctx, dbConn, 1700086400, VmDailyRollupRow{ + Vcenter: "vc-a", + VmId: "vm-1", + VmUuid: "uuid-1", + Name: "demo-vm", + CreationTime: 1699999000, + SamplesPresent: 4, + SumVcpu: 20, + SumRam: 36, + LastVcpuCount: 5, + LastRamGB: 9, + LastResourcePool: "Gold", + LastProvisionedDisk: 150.5, + }); err != nil { + t.Fatalf("failed to insert daily rollup row 2: %v", err) + } + + rows, err := FetchVmTraceDaily(ctx, dbConn, "vm-1", "", "") + if err != nil { + t.Fatalf("FetchVmTraceDaily failed: %v", err) + } + if len(rows) != 2 { + t.Fatalf("expected 2 daily trace rows, got %d", len(rows)) + } + if rows[0].SnapshotTime != 1700000000 || rows[0].VcpuCount != 4 || rows[0].RamGB != 8 { + t.Fatalf("unexpected first daily row: %#v", rows[0]) + } + if rows[1].SnapshotTime != 1700086400 || rows[1].VcpuCount != 5 || rows[1].RamGB != 9 || rows[1].ProvisionedDisk != 150.5 { + t.Fatalf("unexpected second daily row: %#v", rows[1]) + } +} + +func TestFetchVmTraceDailyFallbackToSummaryTables(t *testing.T) { + ctx := context.Background() + dbConn := newTestSQLiteDB(t) + + if _, err := dbConn.ExecContext(ctx, ` +CREATE TABLE snapshot_registry ( + snapshot_type TEXT, + table_name TEXT, + snapshot_time BIGINT +)`); err != nil { + t.Fatalf("failed to create snapshot_registry: %v", err) + } + + summaryTable := "inventory_daily_summary_20260106" + if _, err := dbConn.ExecContext(ctx, fmt.Sprintf(` +CREATE TABLE %s ( + "Name" TEXT, + "Vcenter" TEXT, + "VmId" TEXT, + "VmUuid" TEXT, + "ResourcePool" TEXT, + "AvgVcpuCount" REAL, + "AvgRamGB" REAL, + "AvgProvisionedDisk" REAL, + "CreationTime" BIGINT, + "DeletionTime" BIGINT +)`, summaryTable)); err != nil { + t.Fatalf("failed to create summary table: %v", err) + } + if _, err := dbConn.ExecContext(ctx, fmt.Sprintf(` +INSERT INTO %s ("Name","Vcenter","VmId","VmUuid","ResourcePool","AvgVcpuCount","AvgRamGB","AvgProvisionedDisk","CreationTime","DeletionTime") +VALUES (?,?,?,?,?,?,?,?,?,?) +`, summaryTable), "demo-vm", "vc-a", "vm-1", "uuid-1", "Silver", 3.2, 6.7, 123.4, int64(1699999000), int64(0)); err != nil { + t.Fatalf("failed to insert summary row: %v", err) + } + if _, err := dbConn.ExecContext(ctx, `INSERT INTO snapshot_registry (snapshot_type, table_name, snapshot_time) VALUES (?,?,?)`, "daily", summaryTable, int64(1700500000)); err != nil { + t.Fatalf("failed to insert snapshot_registry row: %v", err) + } + + rows, err := FetchVmTraceDaily(ctx, dbConn, "", "uuid-1", "") + if err != nil { + t.Fatalf("FetchVmTraceDaily fallback failed: %v", err) + } + if len(rows) != 1 { + t.Fatalf("expected 1 fallback daily row, got %d", len(rows)) + } + if rows[0].SnapshotTime != 1700500000 || rows[0].VcpuCount != 3 || rows[0].RamGB != 6 { + t.Fatalf("unexpected fallback daily row: %#v", rows[0]) + } } func TestParseHourlySnapshotUnix(t *testing.T) { diff --git a/server/handler/vcenters.go b/server/handler/vcenters.go index 85bbe17..8686db8 100644 --- a/server/handler/vcenters.go +++ b/server/handler/vcenters.go @@ -20,7 +20,7 @@ const ( // VcenterList renders a list of vCenters being monitored. // @Summary List vCenters -// @Description Lists all vCenters with recorded snapshot totals. +// @Description Lists all vCenters with recorded snapshot totals, linking to the fast daily aggregated totals page. // @Tags vcenters // @Produce text/html // @Success 200 {string} string "HTML page" @@ -72,7 +72,7 @@ func (h *Handler) VcenterTotals(w http.ResponseWriter, r *http.Request) { // VcenterTotalsDaily renders the daily-aggregation totals page for one vCenter. // @Summary vCenter daily totals -// @Description Shows daily aggregated VM count/vCPU/RAM totals for a vCenter. +// @Description Shows daily aggregated VM count/vCPU/RAM totals for a vCenter (cache-backed for fast loading). // @Tags vcenters // @Produce text/html // @Param vcenter query string true "vCenter URL" diff --git a/server/handler/vmTrace.go b/server/handler/vmTrace.go index eb5d11c..c3bfccf 100644 --- a/server/handler/vmTrace.go +++ b/server/handler/vmTrace.go @@ -3,20 +3,22 @@ package handler import ( "fmt" "net/http" + "net/url" "strings" "time" "vctp/components/views" "vctp/db" ) -// VmTrace shows per-snapshot details for a VM across all snapshots. +// VmTrace shows VM history in either hourly-detail or daily-aggregated mode. // @Summary Trace VM history -// @Description Shows VM resource history across snapshots, with chart and table. +// @Description Shows VM resource history with an hourly detail view and a daily aggregated view, with chart and table output. // @Tags vm // @Produce text/html // @Param vm_id query string false "VM ID" // @Param vm_uuid query string false "VM UUID" // @Param name query string false "VM name" +// @Param view query string false "hourly|daily (default: hourly)" // @Success 200 {string} string "HTML page" // @Failure 400 {string} string "Missing identifier" // @Router /vm/trace [get] @@ -25,6 +27,11 @@ func (h *Handler) VmTrace(w http.ResponseWriter, r *http.Request) { vmID := r.URL.Query().Get("vm_id") vmUUID := r.URL.Query().Get("vm_uuid") name := r.URL.Query().Get("name") + viewType := strings.ToLower(strings.TrimSpace(r.URL.Query().Get("view"))) + if viewType != "daily" { + viewType = "hourly" + } + meta := buildVmTraceMeta(viewType, vmID, vmUUID, name) var entries []views.VmTraceEntry chart := views.VmTraceChart{} @@ -39,12 +46,20 @@ func (h *Handler) VmTrace(w http.ResponseWriter, r *http.Request) { // Only fetch data when a query is provided; otherwise render empty page with form. if vmID != "" || vmUUID != "" || name != "" { - h.Logger.Info("vm trace request", "vm_id", vmID, "vm_uuid", vmUUID, "name", name) + h.Logger.Info("vm trace request", "view", viewType, "vm_id", vmID, "vm_uuid", vmUUID, "name", name) lifecycle, lifeErr := db.FetchVmLifecycle(ctx, h.Database.DB(), vmID, vmUUID, name) if lifeErr != nil { h.Logger.Warn("failed to fetch VM lifecycle", "error", lifeErr) } - rows, err := db.FetchVmTrace(ctx, h.Database.DB(), vmID, vmUUID, name) + var ( + rows []db.VmTraceRow + err error + ) + if viewType == "daily" { + rows, err = db.FetchVmTraceDaily(ctx, h.Database.DB(), vmID, vmUUID, name) + } else { + rows, err = db.FetchVmTrace(ctx, h.Database.DB(), vmID, vmUUID, name) + } if err != nil { h.Logger.Error("failed to fetch VM trace", "error", err) http.Error(w, fmt.Sprintf("failed to fetch VM trace: %v", err), http.StatusInternalServerError) @@ -99,11 +114,47 @@ func (h *Handler) VmTrace(w http.ResponseWriter, r *http.Request) { } w.Header().Set("Content-Type", "text/html; charset=utf-8") - if err := views.VmTracePage(queryLabel, displayQuery, vmID, vmUUID, name, creationLabel, deletionLabel, creationApprox, entries, chart).Render(ctx, w); err != nil { + if err := views.VmTracePage(queryLabel, displayQuery, vmID, vmUUID, name, creationLabel, deletionLabel, creationApprox, entries, chart, meta).Render(ctx, w); err != nil { http.Error(w, "Failed to render template", http.StatusInternalServerError) } } +func buildVmTraceMeta(viewType, vmID, vmUUID, name string) views.VmTraceMeta { + if viewType != "daily" { + viewType = "hourly" + } + addQuery := func(targetView string) string { + q := url.Values{} + q.Set("view", targetView) + if strings.TrimSpace(vmID) != "" { + q.Set("vm_id", vmID) + } + if strings.TrimSpace(vmUUID) != "" { + q.Set("vm_uuid", vmUUID) + } + if strings.TrimSpace(name) != "" { + q.Set("name", name) + } + return "/vm/trace?" + q.Encode() + } + + meta := views.VmTraceMeta{ + ViewType: viewType, + TypeLabel: "Hourly", + HourlyLink: addQuery("hourly"), + DailyLink: addQuery("daily"), + HourlyClass: "web3-button", + DailyClass: "web3-button", + } + if viewType == "daily" { + meta.TypeLabel = "Daily" + meta.DailyClass = "web3-button active" + } else { + meta.HourlyClass = "web3-button active" + } + return meta +} + func buildVmTraceChart(entries []views.VmTraceEntry) views.VmTraceChart { if len(entries) == 0 { return views.VmTraceChart{} diff --git a/server/router/docs/docs.go b/server/router/docs/docs.go index 76d4e5f..9a15f87 100644 --- a/server/router/docs/docs.go +++ b/server/router/docs/docs.go @@ -876,7 +876,7 @@ const docTemplate = `{ }, "/vcenters": { "get": { - "description": "Lists all vCenters with recorded snapshot totals.", + "description": "Lists all vCenters with recorded snapshot totals, linking to the fast daily aggregated totals page.", "produces": [ "text/html" ], @@ -943,7 +943,7 @@ const docTemplate = `{ }, "/vcenters/totals/daily": { "get": { - "description": "Shows daily aggregated VM count/vCPU/RAM totals for a vCenter.", + "description": "Shows daily aggregated VM count/vCPU/RAM totals for a vCenter (cache-backed for fast loading).", "produces": [ "text/html" ], @@ -1019,7 +1019,7 @@ const docTemplate = `{ }, "/vm/trace": { "get": { - "description": "Shows VM resource history across snapshots, with chart and table.", + "description": "Shows VM resource history with an hourly detail view and a daily aggregated view, with chart and table output.", "produces": [ "text/html" ], @@ -1045,6 +1045,12 @@ const docTemplate = `{ "description": "VM name", "name": "name", "in": "query" + }, + { + "type": "string", + "description": "hourly|daily (default: hourly)", + "name": "view", + "in": "query" } ], "responses": { diff --git a/server/router/docs/swagger.json b/server/router/docs/swagger.json index 62ccec3..62a0eb3 100644 --- a/server/router/docs/swagger.json +++ b/server/router/docs/swagger.json @@ -865,7 +865,7 @@ }, "/vcenters": { "get": { - "description": "Lists all vCenters with recorded snapshot totals.", + "description": "Lists all vCenters with recorded snapshot totals, linking to the fast daily aggregated totals page.", "produces": [ "text/html" ], @@ -932,7 +932,7 @@ }, "/vcenters/totals/daily": { "get": { - "description": "Shows daily aggregated VM count/vCPU/RAM totals for a vCenter.", + "description": "Shows daily aggregated VM count/vCPU/RAM totals for a vCenter (cache-backed for fast loading).", "produces": [ "text/html" ], @@ -1008,7 +1008,7 @@ }, "/vm/trace": { "get": { - "description": "Shows VM resource history across snapshots, with chart and table.", + "description": "Shows VM resource history with an hourly detail view and a daily aggregated view, with chart and table output.", "produces": [ "text/html" ], @@ -1034,6 +1034,12 @@ "description": "VM name", "name": "name", "in": "query" + }, + { + "type": "string", + "description": "hourly|daily (default: hourly)", + "name": "view", + "in": "query" } ], "responses": { diff --git a/server/router/docs/swagger.yaml b/server/router/docs/swagger.yaml index 6763ef8..be95c1b 100644 --- a/server/router/docs/swagger.yaml +++ b/server/router/docs/swagger.yaml @@ -871,7 +871,8 @@ paths: - snapshots /vcenters: get: - description: Lists all vCenters with recorded snapshot totals. + description: Lists all vCenters with recorded snapshot totals, linking to the + fast daily aggregated totals page. produces: - text/html responses: @@ -916,7 +917,8 @@ paths: - vcenters /vcenters/totals/daily: get: - description: Shows daily aggregated VM count/vCPU/RAM totals for a vCenter. + description: Shows daily aggregated VM count/vCPU/RAM totals for a vCenter (cache-backed + for fast loading). parameters: - description: vCenter URL in: query @@ -967,7 +969,8 @@ paths: - vcenters /vm/trace: get: - description: Shows VM resource history across snapshots, with chart and table. + description: Shows VM resource history with an hourly detail view and a daily + aggregated view, with chart and table output. parameters: - description: VM ID in: query @@ -981,6 +984,10 @@ paths: in: query name: name type: string + - description: 'hourly|daily (default: hourly)' + in: query + name: view + type: string produces: - text/html responses: