6 Commits

Author SHA1 Message Date
Maxim
a2525b53f0 Classify and enrich guest services 2026-08-06 23:24:43 +03:00
Maxim
331f0ee39a Add detailed guest agent diagnostics 2026-08-06 21:12:42 +03:00
Maxim
12aba383f1 Add secure automatic agent updates 2026-08-06 21:01:54 +03:00
Maxim
c618eb018c Expand guest agent system monitoring 2026-08-06 20:48:27 +03:00
Maxim
f1ff15e344 Add read-only guest monitoring agents 2026-08-06 20:24:13 +03:00
Maxim
5401b5b0f2 Add maintenance windows for planned work 2026-08-06 19:35:33 +03:00
13 changed files with 1666 additions and 64 deletions

View File

@@ -322,6 +322,56 @@ apcaccess status
- `GET /api/events` — поток метрик SSE с интервалом две секунды.
- `GET/POST/PUT/DELETE /api/services` — управление проверками сервисов.
- `GET /api/services/history` — семидневная история доступности и отклика.
- `GET/PUT/DELETE /api/settings/maintenance` — состояние, запуск и досрочное завершение плановых работ.
- `GET/DELETE /api/agents` — подключённые гостевые агенты.
- `POST /api/agents/enrollment-token` — одноразовый токен регистрации на 15 минут.
## Maintenance-режим
Во вкладке `Настройки → Maintenance` можно на заданное время отключить все
алерты либо выбрать конкретные сервисы и VM/LXC. Подавленные события не
отправляются по почте и не отображаются как активные, но сохраняются в истории
с причиной плановых работ. После окончания окна обычный мониторинг включается
автоматически.
## Агенты VM и LXC
Read-only агент показывает состояние Linux внутри VM/LXC: CPU, load average,
память, корневой раздел, uptime, ОС, ядро, IP-адреса, systemd-службы, доступные
APT-обновления и необходимость перезагрузки. Для Docker отображаются контейнеры,
health-check, число перезапусков, CPU и память. Агент сам подключается к Dashboard
каждые 10 секунд; открывать входящий порт внутри гостевой системы не нужно.
В подробностях агента systemd-службы разделяются на системные, установленные
дополнительно и локальные unit-файлы. Для каждой службы показываются пакет и путь
unit-файла, время запуска, число перезапусков, последний exit code, память и
накопленное CPU-время, когда соответствующий accounting доступен в systemd.
Откройте `Настройки → Агенты`, укажите адрес Dashboard, доступный из VM/LXC, и
VMID. Кнопка создаст готовую команду установки. Одноразовый токен действует 15
минут и после регистрации заменяется индивидуальным секретом агента.
После однократной установки агент обновляется автоматически вслед за Dashboard.
Центральный сервер отдаёт агенту собственный бинарник и SHA-256; агент проверяет
контрольную сумму, атомарно заменяет файл и перезапускается через systemd. Если
агент был удалён из Dashboard, повторная команда с новым токеном автоматически
выполнит новую регистрацию.
В первой версии агент не выполняет команды и не перезапускает службы или
контейнеры. Для чтения Docker ему требуется доступ к `/var/run/docker.sock`,
поэтому служба устанавливается от root. Сам Dashboard пока не имеет
авторизации: используйте локальную сеть или HTTPS через доверенный reverse
proxy и не публикуйте агентские API напрямую в интернет.
Полностью удалить агент из VM/LXC:
```bash
curl -fsSL https://git.myown.center/maxim/ProxmoxDash/raw/branch/main/scripts/uninstall-agent.sh | sh
```
Оформление, пороги, почта и обновления Dashboard находятся в обычных настройках.
Рабочие инструменты — сервисы, агенты, плановые работы и UPS-сценарий — вынесены
в отдельный раздел `Управление` в шапке Dashboard.
## Сборка

917
agent.go Normal file
View File

@@ -0,0 +1,917 @@
package main
import (
"bytes"
"context"
"crypto/hmac"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/http"
"os"
"os/exec"
"path/filepath"
"runtime"
"strconv"
"strings"
"sync"
"syscall"
"time"
)
const agentOfflineAfter = 45 * time.Second
type agentOptions struct {
Server, EnrollmentToken, Name, ConfigPath string
VMID int
}
type agentConfig struct {
Server string `json:"server"`
ID string `json:"id"`
Secret string `json:"secret"`
Name string `json:"name"`
VMID int `json:"vmid"`
}
type AgentContainer struct {
ID string `json:"id"`
Name string `json:"name"`
Image string `json:"image"`
State string `json:"state"`
Status string `json:"status"`
Health string `json:"health,omitempty"`
Restarts int64 `json:"restarts"`
CPUPercent float64 `json:"cpuPercent"`
MemoryUsage uint64 `json:"memoryUsage"`
Ports []string `json:"ports,omitempty"`
}
type AgentInterface struct {
Name string `json:"name"`
Addresses []string `json:"addresses"`
}
type AgentService struct {
Name string `json:"name"`
State string `json:"state"`
SubState string `json:"subState"`
Description string `json:"description"`
Origin string `json:"origin"`
UnitPath string `json:"unitPath,omitempty"`
Package string `json:"package,omitempty"`
Restarts uint64 `json:"restarts"`
ExitCode int `json:"exitCode"`
StartedAt int64 `json:"startedAt,omitempty"`
MemoryBytes uint64 `json:"memoryBytes,omitempty"`
CPUSeconds float64 `json:"cpuSeconds,omitempty"`
}
type AgentIssue struct {
Time int64 `json:"time"`
Unit string `json:"unit"`
Message string `json:"message"`
}
type AgentReport struct {
Hostname string `json:"hostname"`
MachineID string `json:"machineId"`
Version string `json:"version"`
OS string `json:"os"`
Arch string `json:"arch"`
OSName string `json:"osName"`
Kernel string `json:"kernel"`
UptimeSeconds int64 `json:"uptimeSeconds"`
Load1 float64 `json:"load1"`
CPUPercent float64 `json:"cpuPercent"`
MemoryTotal uint64 `json:"memoryTotal"`
MemoryUsed uint64 `json:"memoryUsed"`
RootTotal uint64 `json:"rootTotal"`
RootUsed uint64 `json:"rootUsed"`
DockerAvailable bool `json:"dockerAvailable"`
DockerError string `json:"dockerError,omitempty"`
Containers []AgentContainer `json:"containers"`
Interfaces []AgentInterface `json:"interfaces"`
Services []AgentService `json:"services"`
Issues []AgentIssue `json:"issues"`
UpdatesAvailable int `json:"updatesAvailable"`
RebootRequired bool `json:"rebootRequired"`
InventoryAt int64 `json:"inventoryAt"`
CollectedAt int64 `json:"collectedAt"`
}
type agentInventory struct {
OSName string
Kernel string
Services []AgentService
UpdatesAvailable int
RebootRequired bool
CollectedAt time.Time
}
type AgentUpdateManifest struct {
Version string `json:"version"`
SHA256 string `json:"sha256"`
URL string `json:"url"`
}
type agentReportResponse struct {
Accepted bool `json:"accepted"`
Update *AgentUpdateManifest `json:"update,omitempty"`
}
var guestInventory struct {
sync.Mutex
value agentInventory
}
var guestIssues struct {
sync.Mutex
value []AgentIssue
collectedAt time.Time
}
var agentBinaryInfo struct {
sync.Once
manifest AgentUpdateManifest
path string
err error
}
func currentAgentUpdate() (AgentUpdateManifest, string, error) {
agentBinaryInfo.Do(func() {
agentBinaryInfo.path, agentBinaryInfo.err = os.Executable()
if agentBinaryInfo.err != nil {
return
}
file, err := os.Open(agentBinaryInfo.path)
if err != nil {
agentBinaryInfo.err = err
return
}
defer file.Close()
hash := sha256.New()
if _, err = io.Copy(hash, file); err != nil {
agentBinaryInfo.err = err
return
}
agentBinaryInfo.manifest = AgentUpdateManifest{Version: version, SHA256: hex.EncodeToString(hash.Sum(nil)), URL: "/api/agent/binary"}
})
return agentBinaryInfo.manifest, agentBinaryInfo.path, agentBinaryInfo.err
}
type ManagedAgent struct {
ID string `json:"id"`
Name string `json:"name"`
VMID int `json:"vmid"`
Hostname string `json:"hostname"`
Version string `json:"version"`
LastSeen int64 `json:"lastSeen"`
EnrolledAt int64 `json:"enrolledAt"`
Online bool `json:"online"`
RemoteAddress string `json:"remoteAddress"`
Report AgentReport `json:"report"`
}
type agentEnrollmentRequest struct {
Token, Name, Hostname, MachineID string
VMID int
}
func randomHex(bytesCount int) (string, error) {
b := make([]byte, bytesCount)
if _, err := rand.Read(b); err != nil {
return "", err
}
return hex.EncodeToString(b), nil
}
func tokenHash(value string) string {
sum := sha256.Sum256([]byte(value))
return hex.EncodeToString(sum[:])
}
func (s *Store) CreateAgentEnrollmentToken() (string, error) {
token, err := randomHex(24)
if err != nil {
return "", err
}
_, err = s.db.Exec(`INSERT INTO agent_enrollment_tokens(token_hash,created_at,expires_at) VALUES(?,?,?)`, tokenHash(token), time.Now().Unix(), time.Now().Add(15*time.Minute).Unix())
return token, err
}
func (s *Store) EnrollAgent(req agentEnrollmentRequest) (agentConfig, error) {
tx, err := s.db.Begin()
if err != nil {
return agentConfig{}, err
}
defer tx.Rollback()
var expires int64
if err = tx.QueryRow(`SELECT expires_at FROM agent_enrollment_tokens WHERE token_hash=? AND used_at IS NULL`, tokenHash(req.Token)).Scan(&expires); err != nil || expires < time.Now().Unix() {
return agentConfig{}, errors.New("одноразовый токен недействителен или истёк")
}
id, err := randomHex(12)
if err != nil {
return agentConfig{}, err
}
secret, err := randomHex(32)
if err != nil {
return agentConfig{}, err
}
name := strings.TrimSpace(req.Name)
if name == "" {
name = req.Hostname
}
now := time.Now().Unix()
if _, err = tx.Exec(`INSERT INTO agents(id,name,vmid,hostname,machine_id,secret_hash,enrolled_at,last_seen,report_json,remote_address) VALUES(?,?,?,?,?,?,?,?,?,?)`, id, name, req.VMID, req.Hostname, req.MachineID, tokenHash(secret), now, now, `{}`, ""); err != nil {
return agentConfig{}, err
}
if _, err = tx.Exec(`UPDATE agent_enrollment_tokens SET used_at=? WHERE token_hash=?`, now, tokenHash(req.Token)); err != nil {
return agentConfig{}, err
}
if err = tx.Commit(); err != nil {
return agentConfig{}, err
}
return agentConfig{ID: id, Secret: secret, Name: name, VMID: req.VMID}, nil
}
func (s *Store) SaveAgentReport(id, secret, remote string, report AgentReport) error {
if !s.AuthenticateAgent(id, secret) {
return errors.New("агент не авторизован")
}
data, _ := json.Marshal(report)
_, err := s.db.Exec(`UPDATE agents SET hostname=?,version=?,last_seen=?,report_json=?,remote_address=? WHERE id=?`, report.Hostname, report.Version, time.Now().Unix(), string(data), remote, id)
return err
}
func (s *Store) AuthenticateAgent(id, secret string) bool {
var expected string
return s.db.QueryRow(`SELECT secret_hash FROM agents WHERE id=?`, id).Scan(&expected) == nil && hmac.Equal([]byte(expected), []byte(tokenHash(secret)))
}
func (s *Store) Agents() ([]ManagedAgent, error) {
rows, err := s.db.Query(`SELECT id,name,vmid,hostname,version,enrolled_at,last_seen,report_json,remote_address FROM agents ORDER BY name`)
if err != nil {
return nil, err
}
defer rows.Close()
result := []ManagedAgent{}
for rows.Next() {
var a ManagedAgent
var raw string
if err = rows.Scan(&a.ID, &a.Name, &a.VMID, &a.Hostname, &a.Version, &a.EnrolledAt, &a.LastSeen, &raw, &a.RemoteAddress); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(raw), &a.Report)
a.Online = time.Since(time.Unix(a.LastSeen, 0)) < agentOfflineAfter
result = append(result, a)
}
return result, rows.Err()
}
func (s *Store) DeleteAgent(id string) error {
_, err := s.db.Exec(`DELETE FROM agents WHERE id=?`, id)
return err
}
func runGuestAgent(opts agentOptions) error {
config, err := loadAgentConfig(opts.ConfigPath)
if err != nil && !os.IsNotExist(err) {
return err
}
if opts.Server != "" {
config.Server = strings.TrimRight(opts.Server, "/")
}
if opts.Name != "" {
config.Name = opts.Name
}
if opts.VMID > 0 {
config.VMID = opts.VMID
}
if config.Server == "" {
return errors.New("укажите --agent-server")
}
if config.ID == "" {
if opts.EnrollmentToken == "" {
return errors.New("для первого запуска нужен --agent-enroll-token")
}
config, err = enrollRemote(config, opts.EnrollmentToken)
if err != nil {
return err
}
if err = saveAgentConfig(opts.ConfigPath, config); err != nil {
return err
}
}
client := &http.Client{Timeout: 12 * time.Second}
for {
report := collectAgentReport()
data, _ := json.Marshal(report)
req, _ := http.NewRequest(http.MethodPost, config.Server+"/api/agent/report", bytes.NewReader(data))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+config.ID+"."+config.Secret)
response, requestErr := client.Do(req)
if requestErr == nil {
responseData, _ := io.ReadAll(io.LimitReader(response.Body, 64<<10))
response.Body.Close()
if response.StatusCode == http.StatusUnauthorized && opts.EnrollmentToken != "" {
fresh, enrollmentErr := enrollRemote(config, opts.EnrollmentToken)
if enrollmentErr == nil {
config = fresh
if saveErr := saveAgentConfig(opts.ConfigPath, config); saveErr != nil {
return saveErr
}
continue
}
requestErr = fmt.Errorf("повторная регистрация: %v", enrollmentErr)
}
if response.StatusCode >= 300 {
if requestErr == nil {
requestErr = fmt.Errorf("сервер вернул %s", response.Status)
}
}
if response.StatusCode == http.StatusOK {
var result agentReportResponse
if json.Unmarshal(responseData, &result) == nil && result.Update != nil && newerAgentVersion(result.Update.Version, version) {
if updateErr := applyAgentUpdate(client, config, *result.Update); updateErr != nil {
requestErr = fmt.Errorf("автообновление: %v", updateErr)
} else {
return nil
}
}
}
}
if requestErr != nil {
fmt.Fprintf(os.Stderr, "agent: %v\n", requestErr)
}
time.Sleep(10 * time.Second)
}
}
func newerAgentVersion(candidate, current string) bool {
parse := func(value string) [3]int {
value = strings.TrimPrefix(strings.TrimSpace(value), "v")
value = strings.SplitN(value, "-", 2)[0]
parts := strings.Split(value, ".")
var result [3]int
for i := 0; i < len(parts) && i < 3; i++ {
result[i], _ = strconv.Atoi(parts[i])
}
return result
}
if candidate == "" || current == "" || current == "dev" {
return false
}
next, installed := parse(candidate), parse(current)
for i := 0; i < 3; i++ {
if next[i] > installed[i] {
return true
}
if next[i] < installed[i] {
return false
}
}
return false
}
func applyAgentUpdate(client *http.Client, config agentConfig, update AgentUpdateManifest) error {
if !strings.HasPrefix(update.URL, "/api/agent/") || len(update.SHA256) != 64 {
return errors.New("сервер вернул некорректный манифест")
}
request, err := http.NewRequest(http.MethodGet, config.Server+update.URL, nil)
if err != nil {
return err
}
request.Header.Set("Authorization", "Bearer "+config.ID+"."+config.Secret)
updateClient := *client
updateClient.Timeout = 2 * time.Minute
response, err := updateClient.Do(request)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return fmt.Errorf("загрузка вернула %s", response.Status)
}
executable, err := os.Executable()
if err != nil {
return err
}
temporary := executable + ".update"
_ = os.Remove(temporary)
file, err := os.OpenFile(temporary, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0755)
if err != nil {
return err
}
keep := false
defer func() {
file.Close()
if !keep {
_ = os.Remove(temporary)
}
}()
hash := sha256.New()
written, err := io.Copy(io.MultiWriter(file, hash), io.LimitReader(response.Body, (64<<20)+1))
if err != nil {
return err
}
if written > 64<<20 {
return errors.New("бинарник превышает допустимый размер")
}
if err = file.Sync(); err != nil {
return err
}
if err = file.Close(); err != nil {
return err
}
actual := hex.EncodeToString(hash.Sum(nil))
if !hmac.Equal([]byte(actual), []byte(strings.ToLower(update.SHA256))) {
return errors.New("SHA-256 обновления не совпал")
}
if err = os.Chmod(temporary, 0755); err != nil {
return err
}
if err = os.Rename(temporary, executable); err != nil {
return err
}
keep = true
return nil
}
func enrollRemote(config agentConfig, token string) (agentConfig, error) {
hostname, _ := os.Hostname()
machineID, _ := os.ReadFile("/etc/machine-id")
payload := agentEnrollmentRequest{Token: token, Name: config.Name, VMID: config.VMID, Hostname: hostname, MachineID: strings.TrimSpace(string(machineID))}
data, _ := json.Marshal(payload)
response, err := http.Post(config.Server+"/api/agent/enroll", "application/json", bytes.NewReader(data))
if err != nil {
return config, err
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
message, _ := io.ReadAll(io.LimitReader(response.Body, 4096))
return config, fmt.Errorf("регистрация: %s", strings.TrimSpace(string(message)))
}
var credentials agentConfig
if err = json.NewDecoder(response.Body).Decode(&credentials); err != nil {
return config, err
}
credentials.Server = config.Server
return credentials, nil
}
func loadAgentConfig(path string) (agentConfig, error) {
data, err := os.ReadFile(path)
if err != nil {
return agentConfig{}, err
}
var c agentConfig
err = json.Unmarshal(data, &c)
return c, err
}
func saveAgentConfig(path string, c agentConfig) error {
if err := os.MkdirAll(filepath.Dir(path), 0700); err != nil {
return err
}
data, _ := json.MarshalIndent(c, "", " ")
return os.WriteFile(path, data, 0600)
}
func collectAgentReport() AgentReport {
hostname, _ := os.Hostname()
machineID, _ := os.ReadFile("/etc/machine-id")
report := AgentReport{Hostname: hostname, MachineID: strings.TrimSpace(string(machineID)), Version: version, OS: runtime.GOOS, Arch: runtime.GOARCH, CollectedAt: time.Now().Unix()}
if data, err := os.ReadFile("/proc/uptime"); err == nil {
fields := strings.Fields(string(data))
if len(fields) > 0 {
value, _ := strconv.ParseFloat(fields[0], 64)
report.UptimeSeconds = int64(value)
}
}
if data, err := os.ReadFile("/proc/loadavg"); err == nil {
fields := strings.Fields(string(data))
if len(fields) > 0 {
report.Load1, _ = strconv.ParseFloat(fields[0], 64)
}
}
report.CPUPercent = sampleAgentCPU()
report.Interfaces = collectAgentInterfaces()
if data, err := os.ReadFile("/proc/meminfo"); err == nil {
values := map[string]uint64{}
for _, line := range strings.Split(string(data), "\n") {
f := strings.Fields(line)
if len(f) >= 2 {
v, _ := strconv.ParseUint(f[1], 10, 64)
values[strings.TrimSuffix(f[0], ":")] = v * 1024
}
}
report.MemoryTotal = values["MemTotal"]
available := values["MemAvailable"]
if report.MemoryTotal > available {
report.MemoryUsed = report.MemoryTotal - available
}
}
var stat syscall.Statfs_t
if syscall.Statfs("/", &stat) == nil {
report.RootTotal = stat.Blocks * uint64(stat.Bsize)
free := stat.Bavail * uint64(stat.Bsize)
if report.RootTotal > free {
report.RootUsed = report.RootTotal - free
}
}
report.Containers, report.DockerError = collectDockerContainers()
report.DockerAvailable = report.DockerError == ""
inventory := cachedAgentInventory()
report.OSName, report.Kernel, report.Services = inventory.OSName, inventory.Kernel, inventory.Services
report.Issues = cachedAgentIssues()
report.UpdatesAvailable, report.RebootRequired, report.InventoryAt = inventory.UpdatesAvailable, inventory.RebootRequired, inventory.CollectedAt.Unix()
return report
}
func collectAgentInterfaces() []AgentInterface {
interfaces, err := net.Interfaces()
if err != nil {
return nil
}
result := []AgentInterface{}
for _, iface := range interfaces {
if iface.Flags&net.FlagLoopback != 0 {
continue
}
addresses, _ := iface.Addrs()
item := AgentInterface{Name: iface.Name}
for _, address := range addresses {
value := address.String()
if host, _, err := net.ParseCIDR(value); err == nil {
value = host.String()
}
item.Addresses = append(item.Addresses, value)
}
if len(item.Addresses) > 0 {
result = append(result, item)
}
}
return result
}
func cachedAgentInventory() agentInventory {
guestInventory.Lock()
defer guestInventory.Unlock()
if time.Since(guestInventory.value.CollectedAt) < 15*time.Minute {
return guestInventory.value
}
value := agentInventory{CollectedAt: time.Now()}
value.OSName = readOSPrettyName()
value.Kernel = strings.TrimSpace(runAgentCommand(3*time.Second, "uname", "-r"))
value.Services = collectAgentServices()
value.UpdatesAvailable = countAgentUpdates()
_, err := os.Stat("/var/run/reboot-required")
value.RebootRequired = err == nil
guestInventory.value = value
return value
}
func cachedAgentIssues() []AgentIssue {
guestIssues.Lock()
defer guestIssues.Unlock()
if time.Since(guestIssues.collectedAt) >= time.Minute {
guestIssues.value = collectAgentIssues()
guestIssues.collectedAt = time.Now()
}
return guestIssues.value
}
func readOSPrettyName() string {
data, err := os.ReadFile("/etc/os-release")
if err != nil {
return runtime.GOOS
}
for _, line := range strings.Split(string(data), "\n") {
if strings.HasPrefix(line, "PRETTY_NAME=") {
return strings.Trim(strings.TrimPrefix(line, "PRETTY_NAME="), "\"")
}
}
return runtime.GOOS
}
func runAgentCommand(timeout time.Duration, name string, args ...string) string {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
output, err := exec.CommandContext(ctx, name, args...).Output()
if err != nil && len(output) == 0 {
return ""
}
return string(output)
}
func collectAgentServices() []AgentService {
output := runAgentCommand(5*time.Second, "systemctl", "list-units", "--type=service", "--state=running,failed", "--no-legend", "--plain", "--no-pager")
result := []AgentService{}
for _, line := range strings.Split(output, "\n") {
fields := strings.Fields(line)
offset := 0
if len(fields) > 0 && fields[0] == "●" {
offset = 1
}
if len(fields) < offset+4 {
continue
}
description := ""
if len(fields) > offset+4 {
description = strings.Join(fields[offset+4:], " ")
}
result = append(result, AgentService{Name: fields[offset], State: fields[offset+2], SubState: fields[offset+3], Description: description})
if len(result) >= 300 {
break
}
}
return enrichAgentServices(result)
}
func enrichAgentServices(services []AgentService) []AgentService {
if len(services) == 0 {
return services
}
args := []string{"show", "--no-pager", "--property=Id", "--property=FragmentPath", "--property=NRestarts", "--property=ExecMainStatus", "--property=ActiveEnterTimestampUSec", "--property=MemoryCurrent", "--property=CPUUsageNSec"}
for _, service := range services {
args = append(args, service.Name)
}
show := runAgentCommand(8*time.Second, "systemctl", args...)
properties := map[string]map[string]string{}
paths := map[string]string{}
for _, record := range strings.Split(show, "\n\n") {
values := map[string]string{}
for _, line := range strings.Split(record, "\n") {
key, value, ok := strings.Cut(line, "=")
if ok {
values[key] = value
}
}
if id := values["Id"]; id != "" {
properties[id] = values
paths[id] = values["FragmentPath"]
}
}
pathArgs := []string{"-S"}
for _, service := range services {
if path := paths[service.Name]; path != "" {
pathArgs = append(pathArgs, path)
}
}
owners := map[string]string{}
packages := []string{}
if len(pathArgs) > 1 {
output := runAgentCommand(8*time.Second, "dpkg-query", pathArgs...)
for _, line := range strings.Split(output, "\n") {
left, path, ok := strings.Cut(line, ": ")
if !ok {
continue
}
pkg := strings.TrimSpace(strings.Split(left, ",")[0])
path = strings.TrimSpace(path)
if pkg != "" && path != "" {
owners[path] = pkg
packages = append(packages, pkg)
}
}
}
priorities := map[string]string{}
if len(packages) > 0 {
packageArgs := []string{"-W", "-f=${binary:Package}\t${Priority}\n"}
packageArgs = append(packageArgs, packages...)
output := runAgentCommand(8*time.Second, "dpkg-query", packageArgs...)
for _, line := range strings.Split(output, "\n") {
fields := strings.Fields(line)
if len(fields) >= 2 {
priorities[fields[0]] = fields[1]
}
}
}
for i := range services {
service := &services[i]
service.UnitPath = paths[service.Name]
service.Package = owners[service.UnitPath]
service.Origin = classifyAgentService(service.UnitPath, service.Package, priorities[service.Package])
values := properties[service.Name]
service.Restarts, _ = strconv.ParseUint(values["NRestarts"], 10, 64)
exit, _ := strconv.ParseInt(values["ExecMainStatus"], 10, 32)
service.ExitCode = int(exit)
started, _ := strconv.ParseInt(values["ActiveEnterTimestampUSec"], 10, 64)
service.StartedAt = started / 1_000_000
service.MemoryBytes, _ = strconv.ParseUint(values["MemoryCurrent"], 10, 64)
cpu, _ := strconv.ParseUint(values["CPUUsageNSec"], 10, 64)
service.CPUSeconds = float64(cpu) / 1_000_000_000
}
return services
}
func classifyAgentService(path, pkg, priority string) string {
switch {
case strings.HasPrefix(path, "/etc/systemd/") || strings.HasPrefix(path, "/usr/local/"):
return "local"
case priority == "required" || priority == "important" || priority == "standard":
return "system"
case pkg != "":
return "added"
case strings.HasPrefix(path, "/run/systemd/"):
return "runtime"
default:
return "system"
}
}
func collectAgentIssues() []AgentIssue {
output := runAgentCommand(8*time.Second, "journalctl", "-p", "err..alert", "--since", "24 hours ago", "--reverse", "-n", "50", "--no-pager", "-o", "json")
return parseAgentIssues(output)
}
func parseAgentIssues(output string) []AgentIssue {
result := []AgentIssue{}
for _, line := range strings.Split(output, "\n") {
var entry struct {
Message any `json:"MESSAGE"`
Unit string `json:"_SYSTEMD_UNIT"`
Timestamp string `json:"__REALTIME_TIMESTAMP"`
}
if json.Unmarshal([]byte(line), &entry) != nil {
continue
}
message, ok := entry.Message.(string)
if !ok || strings.TrimSpace(message) == "" {
continue
}
microseconds, _ := strconv.ParseInt(entry.Timestamp, 10, 64)
unit := entry.Unit
if unit == "" {
unit = "system"
}
result = append(result, AgentIssue{Time: microseconds / 1_000_000, Unit: unit, Message: message})
}
return result
}
func countAgentUpdates() int {
output := runAgentCommand(45*time.Second, "apt-get", "-s", "-o", "Debug::NoLocking=1", "upgrade")
count := 0
for _, line := range strings.Split(output, "\n") {
if strings.HasPrefix(line, "Inst ") {
count++
}
}
return count
}
func sampleAgentCPU() float64 {
total1, idle1 := readAgentCPU()
time.Sleep(100 * time.Millisecond)
total2, idle2 := readAgentCPU()
if total2 <= total1 {
return 0
}
totalDelta, idleDelta := total2-total1, idle2-idle1
if idleDelta > totalDelta {
return 0
}
return float64(totalDelta-idleDelta) / float64(totalDelta) * 100
}
func readAgentCPU() (uint64, uint64) {
data, err := os.ReadFile("/proc/stat")
if err != nil {
return 0, 0
}
line := strings.SplitN(string(data), "\n", 2)[0]
fields := strings.Fields(line)
if len(fields) < 5 {
return 0, 0
}
var total uint64
values := make([]uint64, 0, len(fields)-1)
for _, field := range fields[1:] {
v, _ := strconv.ParseUint(field, 10, 64)
values = append(values, v)
total += v
}
idle := values[3]
if len(values) > 4 {
idle += values[4]
}
return total, idle
}
func collectDockerContainers() ([]AgentContainer, string) {
if _, err := os.Stat("/var/run/docker.sock"); err != nil {
return []AgentContainer{}, "Docker socket недоступен"
}
transport := &http.Transport{DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
return (&net.Dialer{}).DialContext(ctx, "unix", "/var/run/docker.sock")
}}
defer transport.CloseIdleConnections()
client := &http.Client{Transport: transport, Timeout: 5 * time.Second}
response, err := client.Get("http://docker/containers/json?all=1")
if err != nil {
return nil, err.Error()
}
defer response.Body.Close()
if response.StatusCode != 200 {
return nil, response.Status
}
var raw []struct {
ID, Image, State, Status string
Names []string
Ports []struct {
IP string
PrivatePort, PublicPort int
Type string
}
}
if err = json.NewDecoder(response.Body).Decode(&raw); err != nil {
return nil, err.Error()
}
result := make([]AgentContainer, 0, len(raw))
for _, v := range raw {
name := strings.TrimPrefix(firstString(v.Names), "/")
item := AgentContainer{ID: shortID(v.ID), Name: name, Image: v.Image, State: v.State, Status: v.Status}
for _, p := range v.Ports {
if p.PublicPort > 0 {
item.Ports = append(item.Ports, fmt.Sprintf("%d:%d/%s", p.PublicPort, p.PrivatePort, p.Type))
}
}
inspect, inspectErr := client.Get("http://docker/containers/" + v.ID + "/json")
if inspectErr == nil {
var detail struct {
RestartCount int64
State struct{ Health *struct{ Status string } }
}
if json.NewDecoder(inspect.Body).Decode(&detail) == nil {
item.Restarts = detail.RestartCount
if detail.State.Health != nil {
item.Health = detail.State.Health.Status
}
}
inspect.Body.Close()
}
statsResponse, statsErr := client.Get("http://docker/containers/" + v.ID + "/stats?stream=false")
if statsErr == nil {
var stats struct {
CPUStats struct {
CPUUsage struct {
TotalUsage uint64 `json:"total_usage"`
} `json:"cpu_usage"`
SystemCPUUsage uint64 `json:"system_cpu_usage"`
OnlineCPUs uint64 `json:"online_cpus"`
} `json:"cpu_stats"`
PreCPUStats struct {
CPUUsage struct {
TotalUsage uint64 `json:"total_usage"`
} `json:"cpu_usage"`
SystemCPUUsage uint64 `json:"system_cpu_usage"`
} `json:"precpu_stats"`
MemoryStats struct {
Usage uint64 `json:"usage"`
Stats map[string]uint64 `json:"stats"`
} `json:"memory_stats"`
}
if json.NewDecoder(statsResponse.Body).Decode(&stats) == nil {
cpuDelta, systemDelta := uint64(0), uint64(0)
if stats.CPUStats.CPUUsage.TotalUsage >= stats.PreCPUStats.CPUUsage.TotalUsage {
cpuDelta = stats.CPUStats.CPUUsage.TotalUsage - stats.PreCPUStats.CPUUsage.TotalUsage
}
if stats.CPUStats.SystemCPUUsage >= stats.PreCPUStats.SystemCPUUsage {
systemDelta = stats.CPUStats.SystemCPUUsage - stats.PreCPUStats.SystemCPUUsage
}
cpus := stats.CPUStats.OnlineCPUs
if cpus == 0 {
cpus = 1
}
if systemDelta > 0 {
item.CPUPercent = float64(cpuDelta) / float64(systemDelta) * float64(cpus) * 100
}
item.MemoryUsage = stats.MemoryStats.Usage
if cache := stats.MemoryStats.Stats["inactive_file"]; item.MemoryUsage > cache {
item.MemoryUsage -= cache
}
}
statsResponse.Body.Close()
}
result = append(result, item)
}
return result, ""
}
func firstString(v []string) string {
if len(v) > 0 {
return v[0]
}
return ""
}
func shortID(v string) string {
if len(v) > 12 {
return v[:12]
}
return v
}

112
agent_test.go Normal file
View File

@@ -0,0 +1,112 @@
package main
import (
"os"
"testing"
"time"
)
func TestAgentEnrollmentTokenIsSingleUse(t *testing.T) {
store, err := openStore(":memory:")
if err != nil {
t.Fatal(err)
}
defer store.Close()
token, err := store.CreateAgentEnrollmentToken()
if err != nil {
t.Fatal(err)
}
credentials, err := store.EnrollAgent(agentEnrollmentRequest{Token: token, Name: "docker-01", Hostname: "vm105", VMID: 105})
if err != nil {
t.Fatal(err)
}
if credentials.ID == "" || credentials.Secret == "" {
t.Fatal("credentials were not generated")
}
if _, err = store.EnrollAgent(agentEnrollmentRequest{Token: token, Hostname: "duplicate"}); err == nil {
t.Fatal("token was accepted twice")
}
}
func TestCurrentAgentUpdateManifestMatchesExecutable(t *testing.T) {
manifest, path, err := currentAgentUpdate()
if err != nil {
t.Fatal(err)
}
if manifest.URL != "/api/agent/binary" || len(manifest.SHA256) != 64 {
t.Fatalf("unexpected manifest: %#v", manifest)
}
if _, err = os.Stat(path); err != nil {
t.Fatal(err)
}
}
func TestAgentOnlyInstallsNewerVersion(t *testing.T) {
if !newerAgentVersion("v0.14.0", "v0.13.0") {
t.Fatal("new version was not detected")
}
if newerAgentVersion("v0.12.0", "v0.13.0") {
t.Fatal("downgrade was allowed")
}
if newerAgentVersion("v0.13.0", "v0.13.0") {
t.Fatal("same version was offered")
}
}
func TestParseAgentIssues(t *testing.T) {
issues := parseAgentIssues(`{"MESSAGE":"disk error","_SYSTEMD_UNIT":"worker.service","__REALTIME_TIMESTAMP":"1722960000000000"}` + "\n" + `{"MESSAGE":"network error","__REALTIME_TIMESTAMP":"1722960001000000"}`)
if len(issues) != 2 || issues[0].Unit != "worker.service" || issues[1].Unit != "system" || issues[0].Time != 1722960000 {
t.Fatalf("unexpected issues: %#v", issues)
}
}
func TestClassifyAgentService(t *testing.T) {
cases := []struct{ path, pkg, priority, want string }{{"/lib/systemd/system/cron.service", "cron", "important", "system"}, {"/lib/systemd/system/jellyfin.service", "jellyfin", "optional", "added"}, {"/etc/systemd/system/my.service", "", "", "local"}, {"/run/systemd/system/transient.service", "", "", "runtime"}}
for _, tc := range cases {
if got := classifyAgentService(tc.path, tc.pkg, tc.priority); got != tc.want {
t.Errorf("%s: got %s want %s", tc.path, got, tc.want)
}
}
}
func TestAgentReportAuthenticationAndOfflineState(t *testing.T) {
store, err := openStore(":memory:")
if err != nil {
t.Fatal(err)
}
defer store.Close()
token, _ := store.CreateAgentEnrollmentToken()
credentials, _ := store.EnrollAgent(agentEnrollmentRequest{Token: token, Name: "docker", Hostname: "vm", VMID: 101})
report := AgentReport{Hostname: "vm", Version: "test", DockerAvailable: true, Containers: []AgentContainer{{ID: "abc", Name: "web", State: "running"}}}
if err = store.SaveAgentReport(credentials.ID, "wrong", "127.0.0.1", report); err == nil {
t.Fatal("invalid secret accepted")
}
if err = store.SaveAgentReport(credentials.ID, credentials.Secret, "127.0.0.1", report); err != nil {
t.Fatal(err)
}
agents, err := store.Agents()
if err != nil {
t.Fatal(err)
}
if len(agents) != 1 || !agents[0].Online || len(agents[0].Report.Containers) != 1 {
t.Fatalf("unexpected agents: %#v", agents)
}
_, _ = store.db.Exec(`UPDATE agents SET last_seen=? WHERE id=?`, time.Now().Add(-time.Minute).Unix(), credentials.ID)
agents, _ = store.Agents()
if agents[0].Online {
t.Fatal("stale agent is online")
}
}
func TestAgentProblemsBecomeMaintenanceAwareAlerts(t *testing.T) {
metrics := dashboardMetrics{Agents: []ManagedAgent{{ID: "node", Name: "app", VMID: 105, Online: true, Report: AgentReport{
Services: []AgentService{{Name: "postgres.service", State: "failed", SubState: "failed"}},
Containers: []AgentContainer{{ID: "dead", Name: "web", State: "exited", Status: "Exited (1)"}}, DockerAvailable: true,
}}}}
alerts := evaluateAlerts(metrics, defaultThresholds())
window := MaintenanceWindow{Active: true, VMIDs: []int{105}}
active, planned := splitMaintenanceAlerts(alerts, window)
if len(planned) != 2 || len(active) != 0 {
t.Fatalf("active=%d planned=%d alerts=%#v", len(active), len(planned), alerts)
}
}

View File

@@ -6,16 +6,18 @@ import (
)
type AlertEvent struct {
EventID int64 `json:"eventId"`
ID string `json:"id"`
Severity string `json:"severity"`
Source string `json:"source"`
Title string `json:"title"`
Message string `json:"message"`
FirstSeen int64 `json:"firstSeen"`
LastSeen int64 `json:"lastSeen"`
ResolvedAt *int64 `json:"resolvedAt,omitempty"`
AcknowledgedAt *int64 `json:"acknowledgedAt,omitempty"`
EventID int64 `json:"eventId"`
ID string `json:"id"`
Severity string `json:"severity"`
Source string `json:"source"`
Title string `json:"title"`
Message string `json:"message"`
FirstSeen int64 `json:"firstSeen"`
LastSeen int64 `json:"lastSeen"`
ResolvedAt *int64 `json:"resolvedAt,omitempty"`
AcknowledgedAt *int64 `json:"acknowledgedAt,omitempty"`
Planned bool `json:"planned"`
MaintenanceReason string `json:"maintenanceReason,omitempty"`
}
type RecoveryEvent struct {
@@ -24,13 +26,17 @@ type RecoveryEvent struct {
}
func (s *Store) SyncAlerts(alerts []Alert) ([]Alert, error) {
return s.SyncAlertsWithMaintenance(alerts, nil, "")
}
func (s *Store) SyncAlertsWithMaintenance(alerts, plannedAlerts []Alert, maintenanceReason string) ([]Alert, error) {
tx, err := s.db.Begin()
if err != nil {
return alerts, err
}
defer tx.Rollback()
now := time.Now().Unix()
rows, err := tx.Query(`SELECT id,alert_key FROM alert_events WHERE resolved_at IS NULL`)
rows, err := tx.Query(`SELECT id,alert_key,planned FROM alert_events WHERE resolved_at IS NULL`)
if err != nil {
return alerts, err
}
@@ -38,28 +44,43 @@ func (s *Store) SyncAlerts(alerts []Alert) ([]Alert, error) {
for rows.Next() {
var id int64
var key string
if err := rows.Scan(&id, &key); err != nil {
var planned bool
if err := rows.Scan(&id, &key, &planned); err != nil {
rows.Close()
return alerts, err
}
active[key] = id
active[alertEventKey(key, planned)] = id
}
rows.Close()
seen := map[string]bool{}
for _, alert := range alerts {
seen[alert.ID] = true
if id, ok := active[alert.ID]; ok {
_, err = tx.Exec(`UPDATE alert_events SET severity=?,source=?,title=?,message=?,last_seen=? WHERE id=?`, alert.Severity, alert.Source, alert.Title, alert.Message, now, id)
} else {
result, insertErr := tx.Exec(`INSERT INTO alert_events(alert_key,severity,source,title,message,first_seen,last_seen) VALUES(?,?,?,?,?,?,?)`, alert.ID, alert.Severity, alert.Source, alert.Title, alert.Message, now, now)
if insertErr != nil {
return alerts, insertErr
syncGroup := func(group []Alert, planned bool) error {
for _, alert := range group {
key := alertEventKey(alert.ID, planned)
seen[key] = true
reason := ""
if planned {
reason = maintenanceReason
}
if id, ok := active[key]; ok {
_, err = tx.Exec(`UPDATE alert_events SET severity=?,source=?,title=?,message=?,last_seen=?,maintenance_reason=? WHERE id=?`, alert.Severity, alert.Source, alert.Title, alert.Message, now, reason, id)
} else {
result, insertErr := tx.Exec(`INSERT INTO alert_events(alert_key,severity,source,title,message,first_seen,last_seen,planned,maintenance_reason) VALUES(?,?,?,?,?,?,?,?,?)`, alert.ID, alert.Severity, alert.Source, alert.Title, alert.Message, now, now, planned, reason)
if insertErr != nil {
return insertErr
}
active[key], _ = result.LastInsertId()
}
if err != nil {
return err
}
active[alert.ID], _ = result.LastInsertId()
}
if err != nil {
return alerts, err
}
return nil
}
if err = syncGroup(alerts, false); err != nil {
return alerts, err
}
if err = syncGroup(plannedAlerts, true); err != nil {
return alerts, err
}
for key, id := range active {
if !seen[key] {
@@ -74,7 +95,7 @@ func (s *Store) SyncAlerts(alerts []Alert) ([]Alert, error) {
if err = tx.Commit(); err != nil {
return alerts, err
}
rows, err = s.db.Query(`SELECT id,alert_key,severity,source,title,message,first_seen,last_seen FROM alert_events WHERE resolved_at IS NULL AND acknowledged_at IS NULL ORDER BY CASE severity WHEN 'critical' THEN 0 WHEN 'warning' THEN 1 ELSE 2 END,first_seen DESC`)
rows, err = s.db.Query(`SELECT id,alert_key,severity,source,title,message,first_seen,last_seen FROM alert_events WHERE resolved_at IS NULL AND acknowledged_at IS NULL AND planned=0 ORDER BY CASE severity WHEN 'critical' THEN 0 WHEN 'warning' THEN 1 ELSE 2 END,first_seen DESC`)
if err != nil {
return alerts, err
}
@@ -90,6 +111,13 @@ func (s *Store) SyncAlerts(alerts []Alert) ([]Alert, error) {
return visible, rows.Err()
}
func alertEventKey(id string, planned bool) string {
if planned {
return id + "\x00planned"
}
return id + "\x00active"
}
func (s *Store) AcknowledgeAlert(eventID int64) error {
_, err := s.db.Exec(`UPDATE alert_events SET acknowledged_at=? WHERE id=? AND acknowledged_at IS NULL`, time.Now().Unix(), eventID)
return err
@@ -99,7 +127,7 @@ func (s *Store) AlertHistory(limit int) ([]AlertEvent, error) {
if limit < 1 || limit > 500 {
limit = 200
}
rows, err := s.db.Query(`SELECT id,alert_key,severity,source,title,message,first_seen,last_seen,resolved_at,acknowledged_at FROM alert_events ORDER BY first_seen DESC LIMIT ?`, limit)
rows, err := s.db.Query(`SELECT id,alert_key,severity,source,title,message,first_seen,last_seen,resolved_at,acknowledged_at,planned,maintenance_reason FROM alert_events ORDER BY first_seen DESC LIMIT ?`, limit)
if err != nil {
return nil, err
}
@@ -108,7 +136,7 @@ func (s *Store) AlertHistory(limit int) ([]AlertEvent, error) {
for rows.Next() {
var e AlertEvent
var resolved, acked sql.NullInt64
if err = rows.Scan(&e.EventID, &e.ID, &e.Severity, &e.Source, &e.Title, &e.Message, &e.FirstSeen, &e.LastSeen, &resolved, &acked); err != nil {
if err = rows.Scan(&e.EventID, &e.ID, &e.Severity, &e.Source, &e.Title, &e.Message, &e.FirstSeen, &e.LastSeen, &resolved, &acked, &e.Planned, &e.MaintenanceReason); err != nil {
return nil, err
}
if resolved.Valid {
@@ -123,7 +151,7 @@ func (s *Store) AlertHistory(limit int) ([]AlertEvent, error) {
}
func (s *Store) PendingEmailAlerts() ([]Alert, error) {
rows, err := s.db.Query(`SELECT id,alert_key,severity,source,title,message FROM alert_events WHERE resolved_at IS NULL AND acknowledged_at IS NULL AND email_sent_at IS NULL ORDER BY first_seen`)
rows, err := s.db.Query(`SELECT id,alert_key,severity,source,title,message FROM alert_events WHERE resolved_at IS NULL AND acknowledged_at IS NULL AND email_sent_at IS NULL AND planned=0 ORDER BY first_seen`)
if err != nil {
return nil, err
}
@@ -155,7 +183,7 @@ func (s *Store) MarkAlertsEmailed(alerts []Alert) error {
}
func (s *Store) PendingImmediateAlerts() ([]Alert, error) {
rows, err := s.db.Query(`SELECT id,alert_key,severity,source,title,message FROM alert_events WHERE resolved_at IS NULL AND acknowledged_at IS NULL AND email_sent_at IS NULL AND (severity='critical' OR alert_key='ups-battery') ORDER BY first_seen`)
rows, err := s.db.Query(`SELECT id,alert_key,severity,source,title,message FROM alert_events WHERE resolved_at IS NULL AND acknowledged_at IS NULL AND email_sent_at IS NULL AND planned=0 AND (severity='critical' OR alert_key='ups-battery') ORDER BY first_seen`)
if err != nil {
return nil, err
}
@@ -171,7 +199,7 @@ func (s *Store) PendingImmediateAlerts() ([]Alert, error) {
return out, rows.Err()
}
func (s *Store) PendingRecoveries() ([]RecoveryEvent, error) {
rows, err := s.db.Query(`SELECT id,alert_key,severity,source,title,message FROM alert_events WHERE resolved_at IS NOT NULL AND email_sent_at IS NOT NULL AND recovery_sent_at IS NULL ORDER BY resolved_at`)
rows, err := s.db.Query(`SELECT id,alert_key,severity,source,title,message FROM alert_events WHERE resolved_at IS NOT NULL AND email_sent_at IS NOT NULL AND recovery_sent_at IS NULL AND planned=0 ORDER BY resolved_at`)
if err != nil {
return nil, err
}

View File

@@ -197,6 +197,36 @@ func evaluateAlerts(metrics dashboardMetrics, thresholds AlertThresholds) []Aler
add("guests-stopped", "info", "VM/LXC", "Есть остановленные гости", fmt.Sprintf("Остановлено VM/LXC: %d. Это уведомление информационное.", stopped))
}
}
for _, agent := range metrics.Agents {
vmSuffix := fmt.Sprintf("-%d", agent.VMID)
if !agent.Online {
add("agent-offline-"+agent.ID+vmSuffix, "critical", "Агенты", agent.Name+": агент не отвечает", fmt.Sprintf("Последний heartbeat: %s назад. VMID %d.", time.Since(time.Unix(agent.LastSeen, 0)).Round(time.Second), agent.VMID))
continue
}
if agent.Report.DockerAvailable {
for _, container := range agent.Report.Containers {
if container.State != "running" {
add("agent-container-stopped-"+agent.ID+"-"+container.ID+vmSuffix, "warning", "Docker", agent.Name+": контейнер "+container.Name+" остановлен", fmt.Sprintf("%s · VMID %d.", container.Status, agent.VMID))
} else if container.Health == "unhealthy" {
add("agent-container-health-"+agent.ID+"-"+container.ID+vmSuffix, "critical", "Docker", agent.Name+": контейнер "+container.Name+" unhealthy", fmt.Sprintf("Docker health-check не проходит · VMID %d.", agent.VMID))
}
}
}
for _, service := range agent.Report.Services {
if service.State == "failed" || service.SubState == "failed" {
add("agent-systemd-"+agent.ID+"-"+service.Name+vmSuffix, "critical", "Агенты", agent.Name+": упала служба "+service.Name, fmt.Sprintf("%s · VMID %d.", firstNonEmpty(service.Description, service.SubState), agent.VMID))
}
}
if agent.Report.RebootRequired {
add("agent-reboot-"+agent.ID+vmSuffix, "info", "Агенты", agent.Name+": требуется перезагрузка", fmt.Sprintf("Обновления системы запросили перезагрузку · VMID %d.", agent.VMID))
}
if agent.Report.MemoryTotal > 0 && float64(agent.Report.MemoryUsed)/float64(agent.Report.MemoryTotal)*100 >= 95 {
add("agent-memory-"+agent.ID+vmSuffix, "warning", "Агенты", agent.Name+": почти закончилась память", fmt.Sprintf("Используется %.1f%% RAM · VMID %d.", float64(agent.Report.MemoryUsed)/float64(agent.Report.MemoryTotal)*100, agent.VMID))
}
if agent.Report.RootTotal > 0 && float64(agent.Report.RootUsed)/float64(agent.Report.RootTotal)*100 >= 90 {
add("agent-storage-"+agent.ID+vmSuffix, "critical", "Агенты", agent.Name+": мало места на системном разделе", fmt.Sprintf("Раздел / заполнен на %.1f%% · VMID %d.", float64(agent.Report.RootUsed)/float64(agent.Report.RootTotal)*100, agent.VMID))
}
}
for _, service := range metrics.Services.Services {
for _, endpoint := range service.Endpoints {
if !endpoint.Up {
@@ -250,9 +280,6 @@ func evaluateAlerts(metrics dashboardMetrics, thresholds AlertThresholds) []Aler
if !metrics.Monitor.LastSuccessfulCollection.IsZero() && time.Since(metrics.Monitor.LastSuccessfulCollection) > 30*time.Second {
add("monitor-stale", "critical", "Dashboard", "Данные мониторинга устарели", "Последний успешный сбор был "+time.Since(metrics.Monitor.LastSuccessfulCollection).Round(time.Second).String()+" назад.")
}
if metrics.Monitor.SlowCollector {
add("monitor-slow", "warning", "Dashboard", "Сбор метрик выполняется слишком долго", fmt.Sprintf("Последний цикл занял %d ms.", metrics.Monitor.CollectionDurationMs))
}
if metrics.Monitor.HistoryWriteError != "" {
add("monitor-history", "critical", "Dashboard", "Ошибка записи истории", metrics.Monitor.HistoryWriteError)
}

View File

@@ -16,3 +16,12 @@ func TestEvaluateAlerts(t *testing.T) {
t.Fatalf("неверная сортировка уведомлений: %+v", alerts)
}
}
func TestSlowCollectionDoesNotCreateAlert(t *testing.T) {
metrics := dashboardMetrics{Monitor: MonitorHealth{SlowCollector: true, CollectionDurationMs: 12308}}
for _, alert := range evaluateAlerts(metrics, defaultThresholds()) {
if alert.ID == "monitor-slow" {
t.Fatalf("медленный цикл не должен создавать уведомление: %+v", alert)
}
}
}

12
main.go
View File

@@ -20,11 +20,23 @@ var version = "dev"
func main() {
showVersion := flag.Bool("version", false, "show version")
agentMode := flag.Bool("agent", false, "run as a read-only guest agent")
agentServer := flag.String("agent-server", "", "dashboard URL used by the guest agent")
agentEnrollToken := flag.String("agent-enroll-token", "", "one-time enrollment token")
agentName := flag.String("agent-name", "", "agent display name")
agentVMID := flag.Int("agent-vmid", 0, "Proxmox VM/LXC ID")
agentConfig := flag.String("agent-config", "/var/lib/proxmox-dashboard-agent/agent.json", "agent configuration path")
flag.Parse()
if *showVersion {
fmt.Println(version)
return
}
if *agentMode {
if err := runGuestAgent(agentOptions{Server: *agentServer, EnrollmentToken: *agentEnrollToken, Name: *agentName, VMID: *agentVMID, ConfigPath: *agentConfig}); err != nil {
log.Fatalf("Ошибка агента: %v", err)
}
return
}
address := os.Getenv("DASHBOARD_ADDR")
if address == "" {

125
maintenance_window.go Normal file
View File

@@ -0,0 +1,125 @@
package main
import (
"encoding/json"
"fmt"
"strconv"
"strings"
"time"
)
type MaintenanceWindow struct {
Enabled bool `json:"enabled"`
Active bool `json:"active"`
StartedAt int64 `json:"startedAt"`
EndsAt int64 `json:"endsAt"`
Reason string `json:"reason"`
AllAlerts bool `json:"allAlerts"`
ServiceIDs []int64 `json:"serviceIds"`
VMIDs []int `json:"vmids"`
}
func (s *Store) MaintenanceWindow() MaintenanceWindow {
var value string
var window MaintenanceWindow
if s.db.QueryRow(`SELECT value FROM settings WHERE key='maintenance_window'`).Scan(&value) == nil {
_ = json.Unmarshal([]byte(value), &window)
}
now := time.Now().Unix()
window.Active = window.Enabled && window.StartedAt <= now && window.EndsAt > now
if window.Enabled && window.EndsAt <= now {
window.Enabled = false
data, _ := json.Marshal(window)
_, _ = s.db.Exec(`INSERT INTO settings(key,value) VALUES('maintenance_window',?) ON CONFLICT(key) DO UPDATE SET value=excluded.value`, string(data))
}
return window
}
func (s *Store) SaveMaintenanceWindow(window MaintenanceWindow) (MaintenanceWindow, error) {
now := time.Now()
window.Reason = strings.TrimSpace(window.Reason)
if window.Reason == "" {
return window, fmt.Errorf("укажите причину плановых работ")
}
if window.EndsAt <= now.Unix() || window.EndsAt > now.Add(30*24*time.Hour).Unix() {
return window, fmt.Errorf("окончание должно быть в будущем, максимум через 30 дней")
}
if !window.AllAlerts && len(window.ServiceIDs) == 0 && len(window.VMIDs) == 0 {
return window, fmt.Errorf("выберите сервис, VM/LXC или все алерты")
}
window.Enabled = true
window.Active = true
window.StartedAt = now.Unix()
window.ServiceIDs = uniqueInt64(window.ServiceIDs)
window.VMIDs = uniqueInt(window.VMIDs)
data, _ := json.Marshal(window)
_, err := s.db.Exec(`INSERT INTO settings(key,value) VALUES('maintenance_window',?) ON CONFLICT(key) DO UPDATE SET value=excluded.value`, string(data))
return window, err
}
func (s *Store) StopMaintenanceWindow() (MaintenanceWindow, error) {
window := s.MaintenanceWindow()
window.Enabled = false
window.Active = false
window.EndsAt = time.Now().Unix()
data, _ := json.Marshal(window)
_, err := s.db.Exec(`INSERT INTO settings(key,value) VALUES('maintenance_window',?) ON CONFLICT(key) DO UPDATE SET value=excluded.value`, string(data))
return window, err
}
func splitMaintenanceAlerts(alerts []Alert, window MaintenanceWindow) (active, planned []Alert) {
if !window.Active {
return alerts, nil
}
for _, alert := range alerts {
if maintenanceMatchesAlert(window, alert) {
planned = append(planned, alert)
} else {
active = append(active, alert)
}
}
return active, planned
}
func maintenanceMatchesAlert(window MaintenanceWindow, alert Alert) bool {
if window.AllAlerts {
return true
}
for _, id := range window.ServiceIDs {
value := strconv.FormatInt(id, 10)
if strings.HasPrefix(alert.ID, "service-"+value+"-") || strings.HasPrefix(alert.ID, "service-tls-"+value+"-") {
return true
}
}
for _, vmid := range window.VMIDs {
value := strconv.Itoa(vmid)
if strings.HasSuffix(alert.ID, "-"+value) || strings.Contains(alert.Message, "VMID "+value) {
return true
}
}
return false
}
func uniqueInt64(values []int64) []int64 {
seen := map[int64]bool{}
result := make([]int64, 0, len(values))
for _, value := range values {
if value > 0 && !seen[value] {
seen[value] = true
result = append(result, value)
}
}
return result
}
func uniqueInt(values []int) []int {
seen := map[int]bool{}
result := make([]int, 0, len(values))
for _, value := range values {
if value > 0 && !seen[value] {
seen[value] = true
result = append(result, value)
}
}
return result
}

View File

@@ -0,0 +1,60 @@
package main
import (
"testing"
"time"
)
func TestMaintenanceFiltersSelectedTargets(t *testing.T) {
window := MaintenanceWindow{Active: true, ServiceIDs: []int64{7}, VMIDs: []int{105}}
alerts := []Alert{
{ID: "service-7-local"},
{ID: "service-tls-8-public"},
{ID: "backup-old-105"},
{ID: "task-failed-x", Message: "VMID 105, status error"},
{ID: "cpu-usage"},
}
active, planned := splitMaintenanceAlerts(alerts, window)
if len(active) != 2 || len(planned) != 3 {
t.Fatalf("неверное разделение: active=%+v planned=%+v", active, planned)
}
}
func TestMaintenanceExpiresAutomatically(t *testing.T) {
store, err := openStore(":memory:")
if err != nil {
t.Fatal(err)
}
defer store.Close()
window, err := store.SaveMaintenanceWindow(MaintenanceWindow{EndsAt: time.Now().Add(time.Hour).Unix(), Reason: "Обновление", AllAlerts: true})
if err != nil || !window.Active {
t.Fatalf("режим не включён: %+v %v", window, err)
}
if _, err = store.db.Exec(`UPDATE settings SET value=? WHERE key='maintenance_window'`, `{"enabled":true,"startedAt":1,"endsAt":2,"reason":"test","allAlerts":true}`); err != nil {
t.Fatal(err)
}
if expired := store.MaintenanceWindow(); expired.Active || expired.Enabled {
t.Fatalf("просроченный режим остался активным: %+v", expired)
}
}
func TestPlannedAlertOnlyAppearsInHistory(t *testing.T) {
store, err := openStore(":memory:")
if err != nil {
t.Fatal(err)
}
defer store.Close()
alert := Alert{ID: "service-7-local", Severity: "critical", Source: "Сервисы", Title: "Недоступен"}
visible, err := store.SyncAlertsWithMaintenance(nil, []Alert{alert}, "Обновление")
if err != nil || len(visible) != 0 {
t.Fatalf("плановый алерт стал активным: %+v %v", visible, err)
}
history, err := store.AlertHistory(10)
if err != nil || len(history) != 1 || !history[0].Planned || history[0].MaintenanceReason != "Обновление" {
t.Fatalf("плановое событие не сохранено: %+v %v", history, err)
}
pending, err := store.PendingEmailAlerts()
if err != nil || len(pending) != 0 {
t.Fatalf("плановое событие попало в почту: %+v %v", pending, err)
}
}

65
scripts/install-agent.sh Executable file
View File

@@ -0,0 +1,65 @@
#!/bin/sh
set -eu
if [ "$(id -u)" -ne 0 ]; then
echo "Запустите команду установки от root." >&2
exit 1
fi
if [ -z "${DASHBOARD_URL:-}" ] || [ -z "${ENROLL_TOKEN:-}" ]; then
echo "Не заданы DASHBOARD_URL и ENROLL_TOKEN. Создайте команду в настройках Dashboard." >&2
exit 1
fi
GITEA_URL=${GITEA_URL:-https://git.myown.center}
GITEA_REPOSITORY=${GITEA_REPOSITORY:-maxim/ProxmoxDash}
VMID=${VMID:-0}
AGENT_NAME=${AGENT_NAME:-$(hostname)}
release_json=$(curl -fsSL "$GITEA_URL/api/v1/repos/$GITEA_REPOSITORY/releases/latest")
download_url=$(printf '%s' "$release_json" | sed -n 's/.*"browser_download_url":"\([^"]*\/proxmox-dashboard-linux-amd64\)".*/\1/p' | head -n 1)
if [ -z "$download_url" ]; then
echo "В последнем релизе не найден Linux-бинарник." >&2
exit 1
fi
install -d -m 0700 /var/lib/proxmox-dashboard-agent
temporary_binary=$(mktemp /tmp/proxmox-dashboard-agent.XXXXXX)
trap 'rm -f "$temporary_binary"' EXIT
curl -fsSL "$download_url" -o "$temporary_binary"
install -m 0755 "$temporary_binary" /var/lib/proxmox-dashboard-agent/proxmox-dashboard-agent
umask 077
{
printf 'DASHBOARD_URL=%s\n' "$DASHBOARD_URL"
printf 'ENROLL_TOKEN=%s\n' "$ENROLL_TOKEN"
printf 'AGENT_NAME=%s\n' "$AGENT_NAME"
printf 'VMID=%s\n' "$VMID"
} > /etc/proxmox-dashboard-agent.env
cat > /etc/systemd/system/proxmox-dashboard-agent.service <<'UNIT'
[Unit]
Description=Proxmox Dashboard read-only guest agent
After=network-online.target docker.service
Wants=network-online.target
[Service]
Type=simple
EnvironmentFile=/etc/proxmox-dashboard-agent.env
ExecStart=/var/lib/proxmox-dashboard-agent/proxmox-dashboard-agent --agent --agent-server ${DASHBOARD_URL} --agent-enroll-token ${ENROLL_TOKEN} --agent-name ${AGENT_NAME} --agent-vmid ${VMID}
Restart=always
RestartSec=5
NoNewPrivileges=true
ProtectHome=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/var/lib/proxmox-dashboard-agent
[Install]
WantedBy=multi-user.target
UNIT
systemctl daemon-reload
systemctl enable proxmox-dashboard-agent.service
systemctl restart proxmox-dashboard-agent.service
rm -f /usr/local/bin/proxmox-dashboard-agent
echo "Агент установлен. Через несколько секунд он появится в Dashboard."

16
scripts/uninstall-agent.sh Executable file
View File

@@ -0,0 +1,16 @@
#!/bin/sh
set -eu
if [ "$(id -u)" -ne 0 ]; then
echo "Запустите удаление от root." >&2
exit 1
fi
systemctl disable --now proxmox-dashboard-agent.service 2>/dev/null || true
rm -f /etc/systemd/system/proxmox-dashboard-agent.service
rm -f /etc/proxmox-dashboard-agent.env
rm -f /usr/local/bin/proxmox-dashboard-agent
rm -rf /var/lib/proxmox-dashboard-agent
systemctl daemon-reload
systemctl reset-failed proxmox-dashboard-agent.service 2>/dev/null || true
echo "Агент и его локальные учётные данные удалены."

View File

@@ -138,6 +138,8 @@ func openStore(path string) (*Store, error) {
`CREATE INDEX IF NOT EXISTS service_checks_service_ts_idx ON service_checks(service_id,ts)`,
`CREATE TABLE IF NOT EXISTS entity_history (kind TEXT NOT NULL,name TEXT NOT NULL,ts INTEGER NOT NULL,value1 REAL,value2 REAL,PRIMARY KEY(kind,name,ts))`,
`CREATE INDEX IF NOT EXISTS entity_history_kind_ts_idx ON entity_history(kind,ts)`,
`CREATE TABLE IF NOT EXISTS agent_enrollment_tokens (token_hash TEXT PRIMARY KEY, created_at INTEGER NOT NULL, expires_at INTEGER NOT NULL, used_at INTEGER)`,
`CREATE TABLE IF NOT EXISTS agents (id TEXT PRIMARY KEY, name TEXT NOT NULL, vmid INTEGER NOT NULL DEFAULT 0, hostname TEXT NOT NULL DEFAULT '', machine_id TEXT NOT NULL DEFAULT '', secret_hash TEXT NOT NULL, enrolled_at INTEGER NOT NULL, last_seen INTEGER NOT NULL, version TEXT NOT NULL DEFAULT '', report_json TEXT NOT NULL DEFAULT '{}', remote_address TEXT NOT NULL DEFAULT '')`,
}
for _, statement := range statements {
if _, err := db.Exec(statement); err != nil {
@@ -147,6 +149,9 @@ func openStore(path string) (*Store, error) {
}
// Older databases get this column without requiring a separate migration tool.
_, _ = db.Exec(`ALTER TABLE alert_events ADD COLUMN recovery_sent_at INTEGER`)
_, _ = db.Exec(`ALTER TABLE alert_events ADD COLUMN planned INTEGER NOT NULL DEFAULT 0`)
_, _ = db.Exec(`ALTER TABLE alert_events ADD COLUMN maintenance_reason TEXT NOT NULL DEFAULT ''`)
_, _ = db.Exec(`DELETE FROM alert_events WHERE alert_key='monitor-slow'`)
_, _ = db.Exec(`ALTER TABLE service_checks ADD COLUMN endpoint TEXT NOT NULL DEFAULT 'public'`)
_, _ = db.Exec(`PRAGMA foreign_keys=ON`)
return &Store{db: db, path: path}, nil
@@ -230,7 +235,7 @@ func (s *Store) SaveThresholds(value AlertThresholds) error {
}
func defaultEmailSettings() EmailSettings {
return EmailSettings{Port: 587, IntervalMinutes: 60, Severities: []string{"critical", "warning"}, Sources: []string{"CPU", "Память", "Storage", "Диски", "ZFS", "UPS", "VM/LXC", "Proxmox", "Dashboard", "Сервисы", "Система", "Сеть"}}
return EmailSettings{Port: 587, IntervalMinutes: 60, Severities: []string{"critical", "warning"}, Sources: []string{"CPU", "Память", "Storage", "Диски", "ZFS", "UPS", "VM/LXC", "Proxmox", "Dashboard", "Сервисы", "Система", "Сеть", "Агенты", "Docker"}}
}
func (s *Store) EmailSettings(includePassword bool) EmailSettings {

234
web.go
View File

@@ -11,26 +11,28 @@ import (
)
type dashboardMetrics struct {
CPU CPUMetrics `json:"cpu"`
Memory MemoryMetrics `json:"memory"`
Storage StorageMetrics `json:"storage"`
Disks []DiskMetrics `json:"disks"`
MissingDisks []string `json:"missingDisks"`
Network NetworkMetrics `json:"network"`
ZFS ZFSMetrics `json:"zfs"`
UPS UPSMetrics `json:"ups"`
Guests GuestsMetrics `json:"guests"`
Backups BackupMetrics `json:"backups"`
Alerts []Alert `json:"alerts"`
Update UpdateStatus `json:"update"`
Maintenance MaintenanceMetrics `json:"maintenance"`
Services ServicesMetrics `json:"services"`
SystemHealth SystemHealthMetrics `json:"systemHealth"`
Activity ProxmoxActivity `json:"activity"`
Monitor MonitorHealth `json:"monitor"`
UPSShutdown UPSShutdownSettings `json:"upsShutdown"`
Trends []TrendAnomaly `json:"trends"`
Timestamp time.Time `json:"timestamp"`
CPU CPUMetrics `json:"cpu"`
Memory MemoryMetrics `json:"memory"`
Storage StorageMetrics `json:"storage"`
Disks []DiskMetrics `json:"disks"`
MissingDisks []string `json:"missingDisks"`
Network NetworkMetrics `json:"network"`
ZFS ZFSMetrics `json:"zfs"`
UPS UPSMetrics `json:"ups"`
Guests GuestsMetrics `json:"guests"`
Backups BackupMetrics `json:"backups"`
Alerts []Alert `json:"alerts"`
Update UpdateStatus `json:"update"`
Maintenance MaintenanceMetrics `json:"maintenance"`
MaintenanceWindow MaintenanceWindow `json:"maintenanceWindow"`
Services ServicesMetrics `json:"services"`
SystemHealth SystemHealthMetrics `json:"systemHealth"`
Activity ProxmoxActivity `json:"activity"`
Monitor MonitorHealth `json:"monitor"`
UPSShutdown UPSShutdownSettings `json:"upsShutdown"`
Trends []TrendAnomaly `json:"trends"`
Agents []ManagedAgent `json:"agents"`
Timestamp time.Time `json:"timestamp"`
}
func collectDashboard(cpuCollector *cpuCollector, networkCollector *networkCollector, diskCollector *diskCollector, zfsCollector *zfsCollector, upsCollector *upsCollector, guestCollector *guestCollector, backupCollector *backupCollector, maintenanceCollector *maintenanceCollector, serviceCollector *serviceCollector, systemHealth *systemHealthCollector, activity *activityCollector, ioAttribution *ioAttributionCollector, trends *trendCollector, store *Store, monitor *monitorTracker) (dashboardMetrics, error) {
@@ -63,9 +65,12 @@ func collectDashboard(cpuCollector *cpuCollector, networkCollector *networkColle
metrics.Activity = activity.collect(metrics.Guests)
attributeDiskIO(&metrics, ioAttribution.collect(metrics.Guests))
metrics.Trends = trends.collect(metrics, store)
metrics.Agents, _ = store.Agents()
decorateNetworkInterfaces(&metrics.Network, metrics.Guests)
metrics.Alerts = correlateAlerts(metrics, evaluateAlerts(metrics, thresholds))
metrics.Alerts, err = store.SyncAlerts(metrics.Alerts)
metrics.MaintenanceWindow = store.MaintenanceWindow()
allAlerts := correlateAlerts(metrics, evaluateAlerts(metrics, thresholds))
activeAlerts, plannedAlerts := splitMaintenanceAlerts(allAlerts, metrics.MaintenanceWindow)
metrics.Alerts, err = store.SyncAlertsWithMaintenance(activeAlerts, plannedAlerts, metrics.MaintenanceWindow.Reason)
if err != nil {
return dashboardMetrics{}, err
}
@@ -188,6 +193,118 @@ func routes(collector *cpuCollector, store *Store) http.Handler {
w.WriteHeader(http.StatusMethodNotAllowed)
}
})
mux.HandleFunc("/api/settings/maintenance", func(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
writeJSON(w, store.MaintenanceWindow(), nil)
case http.MethodPut:
var window MaintenanceWindow
if err := json.NewDecoder(io.LimitReader(r.Body, 64<<10)).Decode(&window); err != nil {
http.Error(w, "Некорректные настройки", http.StatusBadRequest)
return
}
value, err := store.SaveMaintenanceWindow(window)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
writeJSON(w, value, nil)
case http.MethodDelete:
value, err := store.StopMaintenanceWindow()
writeJSON(w, value, err)
default:
w.WriteHeader(http.StatusMethodNotAllowed)
}
})
mux.HandleFunc("/api/agents", func(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
agents, err := store.Agents()
writeJSON(w, agents, err)
case http.MethodDelete:
id := strings.TrimSpace(r.URL.Query().Get("id"))
if id == "" {
http.Error(w, "Не указан агент", http.StatusBadRequest)
return
}
writeJSON(w, map[string]bool{"deleted": true}, store.DeleteAgent(id))
default:
w.WriteHeader(http.StatusMethodNotAllowed)
}
})
mux.HandleFunc("/api/agents/enrollment-token", func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
token, err := store.CreateAgentEnrollmentToken()
writeJSON(w, map[string]any{"token": token, "expiresInSeconds": 900}, err)
})
mux.HandleFunc("/api/agent/enroll", func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
var request agentEnrollmentRequest
if json.NewDecoder(io.LimitReader(r.Body, 16<<10)).Decode(&request) != nil || strings.TrimSpace(request.Token) == "" {
http.Error(w, "Некорректная регистрация", http.StatusBadRequest)
return
}
credentials, err := store.EnrollAgent(request)
if err != nil {
http.Error(w, err.Error(), http.StatusUnauthorized)
return
}
writeJSON(w, credentials, nil)
})
mux.HandleFunc("/api/agent/report", func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
id, secret, ok := agentCredentials(r)
if !ok || id == "" || secret == "" {
http.Error(w, "Агент не авторизован", http.StatusUnauthorized)
return
}
var report AgentReport
if json.NewDecoder(io.LimitReader(r.Body, 2<<20)).Decode(&report) != nil {
http.Error(w, "Некорректный отчёт", http.StatusBadRequest)
return
}
if err := store.SaveAgentReport(id, secret, r.RemoteAddr, report); err != nil {
http.Error(w, err.Error(), http.StatusUnauthorized)
return
}
result := agentReportResponse{Accepted: true}
if report.Version != version && version != "dev" {
manifest, _, manifestErr := currentAgentUpdate()
if manifestErr == nil {
result.Update = &manifest
}
}
writeJSON(w, result, nil)
})
mux.HandleFunc("/api/agent/binary", func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
id, secret, ok := agentCredentials(r)
if !ok || !store.AuthenticateAgent(id, secret) {
http.Error(w, "Агент не авторизован", http.StatusUnauthorized)
return
}
manifest, path, err := currentAgentUpdate()
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("X-Checksum-SHA256", manifest.SHA256)
w.Header().Set("Content-Disposition", `attachment; filename="proxmox-dashboard-agent"`)
http.ServeFile(w, r, path)
})
mux.HandleFunc("/api/alerts/history", func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
w.WriteHeader(http.StatusMethodNotAllowed)
@@ -358,6 +475,12 @@ func writeJSON(w http.ResponseWriter, value any, err error) {
_ = json.NewEncoder(w).Encode(value)
}
func agentCredentials(r *http.Request) (string, string, bool) {
credentials := strings.TrimPrefix(r.Header.Get("Authorization"), "Bearer ")
id, secret, ok := strings.Cut(credentials, ".")
return id, secret, ok && id != "" && secret != ""
}
const dashboardHTML = `<!doctype html>
<html lang="ru">
<head>
@@ -385,6 +508,7 @@ dialog{width:min(760px,calc(100% - 28px));max-height:calc(100vh - 32px);padding:
.alert-list{display:grid;gap:10px;padding:16px}.alert-item{display:grid;grid-template-columns:auto 1fr;gap:13px;padding:17px 18px;border:1px solid var(--line);border-radius:14px;background:#0c141a}.alert-icon{width:10px;height:10px;border-radius:50%;margin-top:6px;background:#6ba8ff;box-shadow:0 0 14px currentColor}.alert-item.warning .alert-icon{background:#ffb35c}.alert-item.critical .alert-icon{background:#ff665c}.alert-title{font-weight:650}.alert-source{color:var(--muted);font-size:11px;text-transform:uppercase;letter-spacing:.08em;margin-bottom:4px}.alert-message{color:var(--muted);font-size:13px;margin-top:4px}.all-good{padding:45px 24px;text-align:center}.all-good strong{display:block;font-size:22px;margin-bottom:7px}.all-good span{color:var(--muted)}
html[data-theme="light"] body{background:radial-gradient(circle at 78% -8%,#cde9da 0,transparent 34%),var(--bg)}html[data-theme="light"] .card{background:linear-gradient(145deg,#fff,#f7f9fa)}html[data-theme="light"] dialog{background:#fff;box-shadow:0 30px 80px #4e64752b}html[data-theme="light"] .modal-head{background:rgba(255,255,255,.94)}html[data-theme="light"] .close{background:#f1f4f6}html[data-theme="light"] .disk-item,html[data-theme="light"] .alert-item{background:#f7f9fa}html[data-theme="light"] .bar{background:#e2e8ec}html[data-theme="light"] dialog::backdrop{background:rgba(30,45,55,.35)}html[data-theme="light"] .badge[data-reason]:hover::after,html[data-theme="light"] .badge[data-reason]:focus::after,html[data-theme="light"] .chart-tooltip{background:#fff;box-shadow:0 12px 32px #4e647533}
footer{color:#657581;font-size:12px;margin-top:17px}
.maintenance-banner{display:flex;align-items:center;justify-content:space-between;gap:16px;margin:-7px 0 18px;padding:12px 16px;border:1px solid #6d4c2c;border-radius:13px;background:rgba(255,179,92,.08);color:#ffb35c}.maintenance-banner[hidden]{display:none}.maintenance-banner strong{color:var(--text)}.maintenance-banner small{display:block;color:var(--muted);margin-top:2px}.maintenance-targets{display:grid;grid-template-columns:1fr 1fr;gap:8px;margin-top:10px;max-height:240px;overflow:auto}.maintenance-targets .setting-item{background:var(--panel)}
@media(max-width:700px){main{padding:28px 0}.cards{grid-template-columns:1fr}header{align-items:flex-start;flex-direction:column;gap:14px}.detail-grid{grid-template-columns:repeat(2,1fr)}.detail:nth-child(3n){border-right:1px solid var(--line)}.detail:nth-child(2n){border-right:0}.name{max-width:220px}.disk-stats,.setting-list,.threshold-grid{grid-template-columns:repeat(2,1fr)}.service-form,.maintenance-grid{grid-template-columns:1fr}.maintenance-grid .detail{border-right:0}.chart-head{align-items:flex-start;flex-direction:column}.detail-tabs,.settings-tabs,.system-tabs{overflow-x:auto}}
.save-settings:disabled{cursor:wait;opacity:.55}[data-settings-panel="email"] .threshold-item input{width:min(210px,52vw)}.ack-alert{margin-top:11px;padding:6px 10px;border:1px solid var(--line);border-radius:8px;background:var(--panel);color:var(--text);font-size:11px;cursor:pointer}.history-state{display:inline-block;margin-top:8px;color:var(--muted);font-size:11px}.alert-panel[hidden]{display:none}
.maintenance .fill{background:linear-gradient(90deg,#55d58a,#43c6d9)}.card-order{display:grid;gap:7px;margin-top:11px}.order-item{display:flex;align-items:center;gap:10px;padding:9px 10px;border:1px solid var(--line);border-radius:10px;background:var(--panel)}.order-item.dragging{opacity:.45}.order-handle{cursor:grab;color:var(--muted);font-size:17px}.order-item span:nth-child(2){flex:1}.order-button{width:30px;height:28px;border:1px solid var(--line);border-radius:7px;background:var(--bg);color:var(--text);cursor:pointer}
@@ -392,10 +516,13 @@ footer{color:#657581;font-size:12px;margin-top:17px}
.snapshots .fill{background:linear-gradient(90deg,#9d8cff,#ff78b8)}.tasks .fill{background:linear-gradient(90deg,#43c6d9,#6ba8ff)}.homelab-map .fill{background:linear-gradient(90deg,#55d58a,#9d8cff)}.map-board{display:grid;grid-template-columns:170px 1fr 1fr 1fr;gap:14px;padding:18px;min-width:680px}.map-column{display:grid;align-content:start;gap:9px}.map-column-title{color:var(--muted);font-size:10px;text-transform:uppercase;letter-spacing:.1em}.map-node{padding:12px;border:1px solid var(--line);border-radius:12px;background:var(--panel)}.map-node.good{border-color:#2d6245}.map-node.bad{border-color:#71362f}.map-node.warn{border-color:#6d4c2c}.map-node strong{display:block}.map-node small{display:block;color:var(--muted);margin-top:3px}.map-scroll{overflow-x:auto}.backup-check{margin-top:9px;color:var(--muted);font-size:12px}.task-filter{display:flex;gap:6px;padding:12px 16px;overflow-x:auto}
.topology-board{padding:18px;min-width:620px}.topology-host{margin-bottom:14px}.topology-grid{display:grid;grid-template-columns:repeat(2,minmax(270px,1fr));gap:12px}.topology-group{padding:14px;border:1px solid var(--line);border-radius:15px;background:rgba(100,130,150,.04)}.topology-group-head{display:flex;justify-content:space-between;gap:10px;padding-bottom:10px;border-bottom:1px solid var(--line)}.topology-section{margin-top:11px}.topology-section-title{color:var(--muted);font-size:10px;text-transform:uppercase;letter-spacing:.08em;margin-bottom:6px}.topology-items{display:flex;flex-wrap:wrap;gap:6px}.topology-chip{padding:6px 9px;border:1px solid var(--line);border-radius:9px;background:var(--panel);font-size:12px}.topology-chip.good{border-color:#2d6245}.topology-chip.bad{border-color:#71362f}.topology-unassigned{margin-top:14px}@media(max-width:700px){.topology-grid{grid-template-columns:1fr}}
.monitor .fill{background:linear-gradient(90deg,#55d58a,#43c6d9)}.shutdown-plan{display:grid;gap:8px;margin-top:12px}.shutdown-step{padding:10px 12px;border:1px solid var(--line);border-radius:10px}.infra-status{display:grid;grid-template-columns:repeat(4,1fr);gap:8px;margin:14px 0}.infra-status>div{padding:11px;border:1px solid var(--line);border-radius:10px}.infra-status b{display:block;margin-top:4px}@media(max-width:700px){.infra-status{grid-template-columns:1fr 1fr}}
.agents .fill{background:linear-gradient(90deg,#55d58a,#6ba8ff)}.agent-command{display:block;width:100%;padding:12px;border:1px solid var(--line);border-radius:10px;background:var(--bg);color:var(--text);font:12px/1.5 ui-monospace,monospace;overflow-wrap:anywhere;white-space:pre-wrap}.agent-containers{display:flex;flex-wrap:wrap;gap:7px;margin-top:13px}
.agent-detail-panel{padding:16px}.agent-detail-panel[hidden]{display:none}.agent-detail-summary{display:grid;grid-template-columns:repeat(3,1fr);gap:10px}.agent-detail-summary>div{padding:14px;border:1px solid var(--line);border-radius:12px}.agent-detail-summary b{display:block;margin-top:5px;font:600 16px ui-monospace,monospace}.agent-filter{display:flex;gap:6px;margin-bottom:12px}.agent-filter button{padding:7px 10px;border:1px solid var(--line);border-radius:8px;background:var(--panel);color:var(--text);cursor:pointer}.agent-filter button.active{border-color:var(--green);color:var(--green)}@media(max-width:700px){.agent-detail-summary{grid-template-columns:1fr 1fr}}
</style>
</head>
<body><main>
<header><div><div class="eyebrow">Proxmox host telemetry</div><h1>Host overview</h1></div><div class="header-actions"><div class="status"><i class="dot" id="dot"></i><span id="status">Подключение…</span></div><button class="settings-button" id="settingsButton" aria-label="Настройки">⚙</button></div></header>
<header><div><div class="eyebrow">Proxmox host telemetry</div><h1>Host overview</h1></div><div class="header-actions"><div class="status"><i class="dot" id="dot"></i><span id="status">Подключение…</span></div><button class="settings-button" id="managementButton" aria-label="Управление" title="Сервисы, агенты и плановые работы">⌘</button><button class="settings-button" id="settingsButton" aria-label="Настройки" title="Настройки Dashboard">⚙</button></div></header>
<div class="maintenance-banner" id="maintenanceBanner" hidden><div><strong>Плановые работы</strong><small id="maintenanceBannerReason"></small></div><span id="maintenanceBannerEnds"></span></div>
<section class="cards">
<button class="card alert-center" data-card="alerts" data-open="alertsDialog" aria-label="Открыть центр уведомлений">
<div class="card-head"><div><div class="label">Notification center</div><div class="name" id="alertHeadline">Проверка состояния…</div></div><div class="percent"><span id="alertCount">—</span><small> событий</small></div></div>
@@ -449,6 +576,12 @@ footer{color:#657581;font-size:12px;margin-top:17px}
<div class="quick"><div><span class="label">CPU VM и LXC</span><strong id="guestCPU">—</strong></div><div><span class="label">Память VM и LXC</span><strong id="guestMemory">—</strong></div></div>
<div class="hint">Нажмите, чтобы посмотреть подробности →</div>
</button>
<button class="card agents" data-card="agents" data-open="agentsDialog" aria-label="Открыть агенты VM и LXC">
<div class="card-head"><div><div class="label">Guest telemetry</div><div class="name">Агенты VM и LXC</div></div><div class="percent"><span id="agentOnline">—</span><small id="agentTotal"> / —</small></div></div>
<div class="bar"><div class="fill" id="agentBar"></div></div>
<div class="quick"><div><span class="label">Docker-контейнеры</span><strong id="agentContainers">—</strong></div><div><span class="label">Проблемы</span><strong id="agentProblems">—</strong></div></div>
<div class="hint">Системные метрики и Docker внутри VM/LXC →</div>
</button>
<button class="card maintenance" data-card="maintenance" data-open="maintenanceDialog" aria-label="Открыть сведения Proxmox">
<div class="card-head"><div><div class="label">Proxmox</div><div class="name" id="maintenanceVersion">Обслуживание хоста</div></div><div class="percent"><span id="maintenanceUpdates">—</span><small> updates</small></div></div>
<div class="bar"><div class="fill" id="maintenanceBar"></div></div>
@@ -597,10 +730,12 @@ footer{color:#657581;font-size:12px;margin-top:17px}
<dialog id="snapshotsDialog"><div class="modal-head"><div><div class="label">Snapshots</div><h2>Снимки VM и LXC</h2></div><button class="close" aria-label="Закрыть">×</button></div><div class="disk-list" id="snapshotList"><span class="empty">Получение snapshots…</span></div></dialog>
<dialog id="tasksDialog"><div class="modal-head"><div><div class="label">Proxmox activity</div><h2>События и задачи</h2></div><button class="close" aria-label="Закрыть">×</button></div><div class="task-filter"><button class="detail-tab active" data-task-filter="all">Все</button><button class="detail-tab" data-task-filter="vzdump">Backup</button><button class="detail-tab" data-task-filter="qm">VM/LXC</button><button class="detail-tab" data-task-filter="error">Ошибки</button></div><div class="disk-list" id="taskList"><span class="empty">Получение задач…</span></div></dialog>
<dialog id="mapDialog"><div class="modal-head"><div><div class="label">Homelab topology</div><h2>Карта homelab</h2></div><button class="close" aria-label="Закрыть">×</button></div><div class="map-scroll"><div class="topology-board" id="mapBoard"></div></div><div class="wide settings-note">Порт от локального адреса отбрасывается. Полученный IP сравнивается с адресами VM/LXC из Proxmox. Storage берётся из конфигурации каждой VM и LXC.</div></dialog>
<dialog id="agentsDialog"><div class="modal-head"><div><div class="label">Guest telemetry</div><h2>Агенты VM и LXC</h2></div><button class="close" aria-label="Закрыть">×</button></div><div class="disk-list" id="agentList"><span class="empty">Подключённые агенты появятся здесь.</span></div></dialog>
<dialog id="agentDetailDialog"><div class="modal-head"><div><div class="label">Guest details</div><h2 id="agentDetailTitle">Агент</h2><div class="disk-meta" id="agentDetailMeta"></div></div><button class="close" aria-label="Закрыть">×</button></div><div class="system-tabs"><button class="detail-tab active" data-agent-tab="overview">Обзор</button><button class="detail-tab" data-agent-tab="services">Службы</button><button class="detail-tab" data-agent-tab="issues">Ошибки</button><button class="detail-tab" data-agent-tab="docker">Docker</button></div><div class="agent-detail-panel" data-agent-panel="overview" id="agentOverview"></div><div class="agent-detail-panel" data-agent-panel="services" id="agentServiceDetails" hidden></div><div class="agent-detail-panel" data-agent-panel="issues" id="agentIssueDetails" hidden></div><div class="agent-detail-panel" data-agent-panel="docker" id="agentDockerDetails" hidden></div></dialog>
<dialog id="settingsDialog">
<div class="modal-head"><div><div class="label">Dashboard</div><h2>Настройки</h2></div><button class="close" aria-label="Закрыть">×</button></div>
<div class="settings-tabs"><button class="settings-tab active" data-settings-tab="appearance">Интерфейс</button><button class="settings-tab" data-settings-tab="alerts">Пороги</button><button class="settings-tab" data-settings-tab="services">Сервисы</button><button class="settings-tab" data-settings-tab="ups">UPS shutdown</button><button class="settings-tab" data-settings-tab="email">Почта</button><button class="settings-tab" data-settings-tab="updates">Обновления</button></div>
<div class="settings-tabs"><button class="settings-tab active" data-settings-tab="appearance">Интерфейс</button><button class="settings-tab" data-settings-tab="alerts">Пороги</button><button class="settings-tab" data-settings-tab="email">Почта</button><button class="settings-tab" data-settings-tab="updates">Обновления</button></div>
<div class="settings-body settings-panel" data-settings-panel="appearance">
<div class="settings-group"><div class="label">Тема оформления</div><div class="theme-options"><button class="theme-option" data-theme-choice="dark">Тёмная</button><button class="theme-option" data-theme-choice="light">Светлая</button></div></div>
<div class="settings-group"><div class="label">Видимые карточки</div><div class="setting-list">
@@ -608,7 +743,7 @@ footer{color:#657581;font-size:12px;margin-top:17px}
<label class="setting-item"><input type="checkbox" data-card-toggle="cpu"> CPU</label><label class="setting-item"><input type="checkbox" data-card-toggle="memory"> Память</label>
<label class="setting-item"><input type="checkbox" data-card-toggle="storage"> Диски</label><label class="setting-item"><input type="checkbox" data-card-toggle="network"> Сеть</label>
<label class="setting-item"><input type="checkbox" data-card-toggle="zfs"> ZFS</label><label class="setting-item"><input type="checkbox" data-card-toggle="ups"> UPS</label><label class="setting-item"><input type="checkbox" data-card-toggle="monitor"> Самоконтроль</label>
<label class="setting-item"><input type="checkbox" data-card-toggle="guests"> VM и LXC</label><label class="setting-item"><input type="checkbox" data-card-toggle="maintenance"> Proxmox</label><label class="setting-item"><input type="checkbox" data-card-toggle="backups"> Резервные копии</label><label class="setting-item"><input type="checkbox" data-card-toggle="services"> Сервисы</label><label class="setting-item"><input type="checkbox" data-card-toggle="system"> Система</label><label class="setting-item"><input type="checkbox" data-card-toggle="snapshots"> Snapshots</label><label class="setting-item"><input type="checkbox" data-card-toggle="tasks"> Задачи Proxmox</label><label class="setting-item"><input type="checkbox" data-card-toggle="map"> Карта homelab</label>
<label class="setting-item"><input type="checkbox" data-card-toggle="guests"> VM и LXC</label><label class="setting-item"><input type="checkbox" data-card-toggle="agents"> Агенты VM и LXC</label><label class="setting-item"><input type="checkbox" data-card-toggle="maintenance"> Proxmox</label><label class="setting-item"><input type="checkbox" data-card-toggle="backups"> Резервные копии</label><label class="setting-item"><input type="checkbox" data-card-toggle="services"> Сервисы</label><label class="setting-item"><input type="checkbox" data-card-toggle="system"> Система</label><label class="setting-item"><input type="checkbox" data-card-toggle="snapshots"> Snapshots</label><label class="setting-item"><input type="checkbox" data-card-toggle="tasks"> Задачи Proxmox</label><label class="setting-item"><input type="checkbox" data-card-toggle="map"> Карта homelab</label>
</div></div>
<div class="settings-group"><div class="label">Порядок карточек</div><div class="settings-note">Перетаскивайте строки или используйте стрелки.</div><div class="card-order" id="cardOrder"></div></div>
</div>
@@ -630,6 +765,13 @@ footer{color:#657581;font-size:12px;margin-top:17px}
</div>
<button class="save-settings" id="saveThresholds">Сохранить пороги</button><div class="settings-note" id="thresholdStatus">Настройки хранятся на сервере и применяются для всех браузеров.</div>
</div>
<div class="settings-body settings-panel" data-settings-panel="maintenance-mode" hidden>
<div class="settings-group"><div class="label">Окно плановых работ</div><div class="settings-note">Связанные сбои останутся в истории с пометкой «Плановые работы», но не попадут в активные алерты и письма.</div></div>
<div class="threshold-grid" style="margin-top:12px"><label class="field full-span"><span>Причина</span><input id="maintenanceReason" placeholder="Например, обновление Vaultwarden"></label><label class="threshold-item"><span>Продолжительность, минут</span><input id="maintenanceDuration" type="number" min="5" max="43200" value="60"></label><label class="setting-item"><input type="checkbox" id="maintenanceAll"> Отключить все алерты</label></div>
<div class="settings-group"><div class="label">Сервисы</div><div class="maintenance-targets" id="maintenanceServices"><span class="empty">Загрузка сервисов…</span></div></div>
<div class="settings-group"><div class="label">VM и LXC</div><div class="maintenance-targets" id="maintenanceGuests"><span class="empty">Данные VM/LXC ещё не получены.</span></div></div>
<button class="save-settings" id="startMaintenance">Включить Maintenance</button><button class="theme-option" id="stopMaintenance" style="margin-top:9px;width:100%" disabled>Завершить плановые работы</button><div class="settings-note" id="maintenanceStatus">Режим выключен.</div>
</div>
<div class="settings-body settings-panel" data-settings-panel="services" hidden>
<div class="settings-group"><div class="label">Новая или изменяемая проверка</div><div class="service-form">
<input type="hidden" id="serviceId"><label class="field"><span>Название сервиса</span><input id="serviceName" placeholder="Например, Gitea"></label><label class="field"><span>Способ проверки</span><select id="serviceType"><option value="http">Открыть HTTP / HTTPS</option><option value="tcp">Подключиться к TCP-порту</option><option value="ping">Проверить ping</option></select></label>
@@ -639,6 +781,11 @@ footer{color:#657581;font-size:12px;margin-top:17px}
</div><button class="save-settings" id="saveService">Сохранить сервис</button><div class="settings-note" id="serviceStatus">Проверка выполняется каждые 15 секунд, история хранится 7 дней.</div></div>
<div class="settings-group"><div class="label">Добавленные сервисы</div><div class="disk-list" id="serviceSettingsList"><span class="empty">Список пуст.</span></div></div>
</div>
<div class="settings-body settings-panel" data-settings-panel="agents" hidden>
<div class="settings-group"><div class="label">Подключить или обновить агент</div><div class="settings-note">Токен одноразовый и действует 15 минут. Для обновления уже подключённого агента создайте команду с тем же VMID и выполните её повторно — регистрация и история сохранятся.</div><div class="threshold-grid" style="margin-top:12px"><label class="field"><span>Адрес Dashboard из VM/LXC</span><input id="agentServerURL" placeholder="https://dashboard.example.lan"></label><label class="field"><span>VMID · необязательно</span><input id="agentVMID" type="number" min="0" placeholder="105"></label></div><button class="save-settings" id="createAgentToken">Создать команду установки или обновления</button></div>
<div class="settings-group" id="agentInstallBlock" hidden><div class="label">Выполните внутри VM/LXC от root</div><code class="agent-command" id="agentInstallCommand"></code><div class="settings-note">Для Docker-метрик служба получает доступ к /var/run/docker.sock. Сам агент не принимает входящие подключения.</div></div>
<div class="settings-group"><div class="label">Подключённые агенты</div><div id="agentSettingsList" class="disk-list" style="padding:12px 0 0"><span class="empty">Агентов пока нет.</span></div></div>
</div>
<div class="settings-body settings-panel" data-settings-panel="email" hidden>
<div class="settings-group"><div class="setting-list"><label class="setting-item"><input type="checkbox" id="emailEnabled"> Включить почтовые дайджесты</label><label class="setting-item"><input type="checkbox" id="emailSkipTLS"> Не проверять TLS-сертификат</label></div><div class="settings-note">Только для доверенного сервера во внутренней сети.</div></div>
<div class="settings-group"><div class="label">SMTP и получатель</div><div class="threshold-grid">
@@ -649,7 +796,7 @@ footer{color:#657581;font-size:12px;margin-top:17px}
</div></div>
<div class="settings-group"><div class="label">Важность</div><div class="setting-list"><label class="setting-item"><input type="checkbox" data-email-severity="critical"> Критические</label><label class="setting-item"><input type="checkbox" data-email-severity="warning"> Предупреждения</label><label class="setting-item"><input type="checkbox" data-email-severity="info"> Информация</label></div></div>
<div class="settings-group"><div class="label">Какие разделы отправлять</div><div class="setting-list">
<label class="setting-item"><input type="checkbox" data-email-source="CPU"> CPU</label><label class="setting-item"><input type="checkbox" data-email-source="Память"> Память</label><label class="setting-item"><input type="checkbox" data-email-source="Storage"> Storage</label><label class="setting-item"><input type="checkbox" data-email-source="Диски"> Диски</label><label class="setting-item"><input type="checkbox" data-email-source="ZFS"> ZFS</label><label class="setting-item"><input type="checkbox" data-email-source="UPS"> UPS</label><label class="setting-item"><input type="checkbox" data-email-source="VM/LXC"> VM и LXC</label><label class="setting-item"><input type="checkbox" data-email-source="Proxmox"> Proxmox</label><label class="setting-item"><input type="checkbox" data-email-source="Backups"> Резервные копии</label><label class="setting-item"><input type="checkbox" data-email-source="Dashboard"> Dashboard</label><label class="setting-item"><input type="checkbox" data-email-source="Сервисы"> Сервисы</label><label class="setting-item"><input type="checkbox" data-email-source="Система"> Система</label><label class="setting-item"><input type="checkbox" data-email-source="Сеть"> Сеть</label>
<label class="setting-item"><input type="checkbox" data-email-source="CPU"> CPU</label><label class="setting-item"><input type="checkbox" data-email-source="Память"> Память</label><label class="setting-item"><input type="checkbox" data-email-source="Storage"> Storage</label><label class="setting-item"><input type="checkbox" data-email-source="Диски"> Диски</label><label class="setting-item"><input type="checkbox" data-email-source="ZFS"> ZFS</label><label class="setting-item"><input type="checkbox" data-email-source="UPS"> UPS</label><label class="setting-item"><input type="checkbox" data-email-source="VM/LXC"> VM и LXC</label><label class="setting-item"><input type="checkbox" data-email-source="Агенты"> Агенты</label><label class="setting-item"><input type="checkbox" data-email-source="Docker"> Docker</label><label class="setting-item"><input type="checkbox" data-email-source="Proxmox"> Proxmox</label><label class="setting-item"><input type="checkbox" data-email-source="Backups"> Резервные копии</label><label class="setting-item"><input type="checkbox" data-email-source="Dashboard"> Dashboard</label><label class="setting-item"><input type="checkbox" data-email-source="Сервисы"> Сервисы</label><label class="setting-item"><input type="checkbox" data-email-source="Система"> Система</label><label class="setting-item"><input type="checkbox" data-email-source="Сеть"> Сеть</label>
</div></div>
<button class="save-settings" id="saveEmail">Сохранить настройки почты</button><button class="theme-option" id="testEmail" style="width:100%;margin-top:9px">Отправить тестовое письмо</button><div class="settings-note" id="emailStatus">Пароль хранится в локальной базе сервера и не возвращается в браузер.</div>
</div>
@@ -660,6 +807,11 @@ footer{color:#657581;font-size:12px;margin-top:17px}
<div class="wide"><div class="label">Состояние обновлений</div><div class="temps"><span class="temp" id="updateMessage">Проверка ещё не выполнялась</span></div><button class="save-settings" id="installUpdate">Проверить и установить обновление</button><div class="settings-note" id="updateChecked">Проверка запускается автоматически раз в сутки.</div></div>
</div>
</dialog>
<dialog id="managementDialog">
<div class="modal-head"><div><div class="label">Operations</div><h2>Управление</h2></div><button class="close" aria-label="Закрыть">×</button></div>
<div class="settings-tabs"><button class="settings-tab active" data-management-tab="services">Сервисы</button><button class="settings-tab" data-management-tab="agents">Агенты</button><button class="settings-tab" data-management-tab="maintenance-mode">Плановые работы</button><button class="settings-tab" data-management-tab="ups">UPS-сценарий</button></div>
<div id="managementPanels"></div>
</dialog>
<script>
const el=id=>document.getElementById(id), fixed=n=>Number(n).toFixed(1);
@@ -668,12 +820,16 @@ const rate=n=>size(n)+'/s';
const safe=v=>String(v??'').replace(/[&<>"']/g,c=>({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[c]));
const uptime=s=>{const d=Math.floor(s/86400),h=Math.floor(s%86400/3600),m=Math.floor(s%3600/60);return d?d+'д '+h+'ч':h?h+'ч '+m+'м':m+'м'};
let alertSettings={backupMaxAgeHours:48};
let maintenanceWindow={},maintenanceGuests=[],maintenanceTargetSignature='';
let lastDashboardEvent=0;
let currentServiceStatuses=[];
let currentSystemHealth={issues:[],failedServices:[]},systemSortNewest=true;
function renderSystemHealth(sh){currentSystemHealth=sh||{issues:[],failedServices:[]};const issues=[...(sh.issues||[])].sort((a,b)=>systemSortNewest?b.time-a.time:a.time-b.time);el('issueList').innerHTML=issues.length?issues.map(v=>'<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(v.unit)+'</div><div class="disk-meta">'+new Date(v.time*1000).toLocaleString('ru-RU')+'</div></div><span class="badge bad">'+(v.count>1?'Повторений: '+v.count:'1 раз')+'</span></div><div class="settings-note issue-message">'+safe(v.message)+'</div></article>').join(''):'<span class="empty">Критических сообщений за 24 часа нет.</span>';el('failedServiceList').innerHTML=(sh.failedServices||[]).length?sh.failedServices.map(v=>'<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(v.unit)+'</div><div class="disk-meta">'+safe(v.description||'Описание отсутствует')+'</div></div><span class="badge bad">'+safe(v.state)+'</span></div><div class="disk-stats"><div class="disk-stat"><span>Загрузка</span><b>'+safe(v.load)+'</b></div><div class="disk-stat"><span>Состояние</span><b>'+safe(v.active)+'</b></div><div class="disk-stat"><span>Подсостояние</span><b>'+safe(v.state)+'</b></div></div></article>').join(''):'<span class="empty">Упавших systemd-служб нет.</span>';el('systemSort').textContent=systemSortNewest?'Сначала свежие ↓':'Сначала старые ↑';el('systemSort').setAttribute('aria-label',systemSortNewest?'Сначала показаны свежие события':'Сначала показаны старые события')}
function endpointHTML(v){const tls=v.tlsDays==null?'Нет TLS':v.tlsDays<0?'Сертификат истёк':v.tlsDays+' дн. до окончания',cert=v.tlsSubject?'<div class="certificate"><div>Сертификат: <b>'+safe(v.tlsSubject)+'</b></div><div>Домены: <b>'+safe((v.tlsDnsNames||[]).join(', ')||'не указаны')+'</b></div><div>Издатель: <b>'+safe(v.tlsIssuer||'—')+'</b></div><div>Для этого адреса: <b class="'+(v.tlsHostnameValid?'status-up':'status-down')+'">'+(v.tlsHostnameValid?'подходит':'не подходит')+'</b></div></div>':'';return '<div class="endpoint"><div class="endpoint-head"><div><b>'+safe(v.label)+'</b><div class="endpoint-address">'+safe(v.address)+'</div></div><span class="badge '+(v.up?'':'bad')+'">'+(v.up?'Доступен':'Недоступен')+'</span></div><div class="disk-stats"><div class="disk-stat"><span>Отклик</span><b>'+fixed(v.latencyMs)+' ms</b></div><div class="disk-stat"><span>HTTP</span><b>'+(v.statusCode||'—')+'</b></div><div class="disk-stat"><span>TLS</span><b>'+tls+'</b></div><div class="disk-stat"><span>Uptime · сутки</span><b>'+fixed(v.uptimeDay)+'%</b></div><div class="disk-stat"><span>Uptime · неделя</span><b>'+fixed(v.uptimeWeek)+'%</b></div></div>'+cert+(v.error?'<div class="settings-note status-down">'+safe(v.error)+'</div>':'')+'</div>'}
function renderServices(svc){const items=svc.services||[];currentServiceStatuses=items;const avg=items.length?items.reduce((a,v)=>a+v.uptimeDay,0)/items.length:0,live=items.filter(v=>v.up),latency=live.length?live.reduce((a,v)=>a+v.latencyMs,0)/live.length:0;el('serviceUp').textContent=svc.up||0;el('serviceTotal').textContent=' / '+(svc.total||0);el('serviceBar').style.width=(svc.total?svc.up/svc.total*100:0)+'%';el('serviceUptime').textContent=svc.total?fixed(avg)+'%':'—';el('serviceLatency').textContent=svc.total?fixed(latency)+' ms':'—';el('serviceList').innerHTML=items.length?items.map(v=>'<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(v.name)+'</div><div class="disk-meta">'+(v.endpoints||[]).length+' '+((v.endpoints||[]).length===1?'адрес':'адреса')+' проверяются независимо</div></div><span class="badge '+(v.up?'':'bad')+'">'+(v.up?'Всё доступно':'Есть проблема')+'</span></div><div class="endpoint-list">'+(v.endpoints||[]).map(endpointHTML).join('')+'</div><div class="service-actions"><button data-service-open="'+v.id+'">Открыть сервис ↗</button></div></article>').join(''):'<span class="empty">Добавьте первую проверку в настройках.</span>'}
function renderMaintenanceTargets(){const signature=JSON.stringify([configuredServices.map(v=>[v.id,v.name]),maintenanceGuests.map(v=>[v.vmid,v.name,v.type]),maintenanceWindow.serviceIds||[],maintenanceWindow.vmids||[]]);if(signature===maintenanceTargetSignature)return;maintenanceTargetSignature=signature;const serviceIDs=new Set(maintenanceWindow.serviceIds||[]),vmids=new Set(maintenanceWindow.vmids||[]);el('maintenanceServices').innerHTML=configuredServices.length?configuredServices.map(v=>'<label class="setting-item"><input type="checkbox" data-maintenance-service="'+v.id+'" '+(serviceIDs.has(v.id)?'checked':'')+'> '+safe(v.name)+'</label>').join(''):'<span class="empty">Сервисы не добавлены.</span>';el('maintenanceGuests').innerHTML=maintenanceGuests.length?maintenanceGuests.map(v=>'<label class="setting-item"><input type="checkbox" data-maintenance-vm="'+v.vmid+'" '+(vmids.has(v.vmid)?'checked':'')+'> '+safe(v.name||('VMID '+v.vmid))+' · '+(v.type==='qemu'?'VM':'LXC')+'</label>').join(''):'<span class="empty">VM/LXC не найдены.</span>';toggleMaintenanceTargets()}
function toggleMaintenanceTargets(){const disabled=el('maintenanceAll').checked;document.querySelectorAll('[data-maintenance-service],[data-maintenance-vm]').forEach(v=>v.disabled=disabled)}
function applyMaintenanceState(window){maintenanceWindow=window||{};const active=!!maintenanceWindow.active;el('maintenanceBanner').hidden=!active;el('maintenanceBannerReason').textContent=maintenanceWindow.reason||'';el('maintenanceBannerEnds').textContent=active?'до '+new Date(maintenanceWindow.endsAt*1000).toLocaleString('ru-RU'):'';el('stopMaintenance').disabled=!active;el('maintenanceStatus').textContent=active?'Активен до '+new Date(maintenanceWindow.endsAt*1000).toLocaleString('ru-RU')+' · '+maintenanceWindow.reason:'Режим выключен.';if(active){el('maintenanceReason').value=maintenanceWindow.reason||'';el('maintenanceDuration').value=Math.max(5,Math.ceil((maintenanceWindow.endsAt-Date.now()/1000)/60));el('maintenanceAll').checked=!!maintenanceWindow.allAlerts}renderMaintenanceTargets()}
function renderNetworkGroups(interfaces){const groups=new Map();interfaces.forEach(v=>{const key=v.vmid?'guest-'+v.vmid:'host',group=groups.get(key)||{vmid:v.vmid,name:v.vmid?(v.guestName||('VMID '+v.vmid)):'Интерфейсы хоста',type:v.guestType||'Host',items:[]};group.items.push(v);groups.set(key,group)});el('networkInterfacesList').innerHTML=[...groups.values()].map(g=>{const primary=g.vmid?g.items.filter(v=>v.name.startsWith('tap')||v.name.startsWith('veth')):g.items,rx=primary.reduce((a,v)=>a+v.receivePerSec,0),tx=primary.reduce((a,v)=>a+v.transmitPerSec,0),errors=g.items.reduce((a,v)=>a+v.receiveErrors+v.transmitErrors+v.receiveDrops+v.transmitDrops,0),down=primary.some(v=>v.kind==='Physical'&&!v.carrier&&(v.master||v.receiveBytes+v.transmitBytes>0));return '<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(g.name)+'</div><div class="disk-meta">'+(g.vmid?'VMID '+g.vmid+' · ':'')+safe(g.type)+' · '+safe(g.items.map(v=>v.name+' ['+(v.kind||'Interface')+(v.speedMbps?' · '+v.speedMbps+'Mb/s':'')+(v.duplex?' · '+v.duplex:'')+(v.master?' → '+v.master:'')+']').join(', '))+'</div></div><span class="badge '+(errors||down?'warn':'')+'"'+(errors||down?' tabindex="0" data-reason="'+safe(down?'Нет carrier на используемом физическом интерфейсе':'Счётчики errors/drops больше нуля')+'"':'')+'>'+(down?'Нет линка':errors?'Есть потери':'Без ошибок')+'</span></div><div class="disk-stats"><div class="disk-stat"><span>Сейчас ↓</span><b>'+rate(rx)+'</b></div><div class="disk-stat"><span>Сейчас ↑</span><b>'+rate(tx)+'</b></div><div class="disk-stat"><span>Получено</span><b>'+size(primary.reduce((a,v)=>a+v.receiveBytes,0))+'</b></div><div class="disk-stat"><span>Передано</span><b>'+size(primary.reduce((a,v)=>a+v.transmitBytes,0))+'</b></div><div class="disk-stat"><span>Ошибки / drops</span><b>'+errors+'</b></div></div></article>'}).join('')}
function renderIOAttribution(disks){const active=disks.filter(d=>d.ioCause||(d.topConsumers||[]).length);el('ioAttributionList').innerHTML=active.length?active.map(d=>'<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(d.name)+' · '+safe(d.ioCause||'Источник не определён')+'</div><div class="disk-meta">'+safe(d.ioCauseDetail||'Собираем статистику процессов')+'</div></div><span class="badge '+(d.ioConfidence>=80?'':'warn')+'" tabindex="0" data-reason="Уверенность учитывает совпадение активной задачи Proxmox и долю I/O процессов">'+(d.ioConfidence||0)+'%</span></div><div class="topology-items">'+(d.topConsumers||[]).slice(0,5).map(v=>'<span class="topology-chip">'+safe(v.kind)+' · '+safe(v.name)+' · '+fixed(v.sharePercent)+'% · ↓'+rate(v.readBytesPerSec)+' ↑'+rate(v.writeBytesPerSec)+'</span>').join('')+'</div></article>').join(''):'<span class="empty">Для атрибуции нужно несколько realtime-циклов.</span>'}
const taskLabel=type=>({vzdump:'Backup',qmstart:'Запуск VM',qmstop:'Остановка VM',qmshutdown:'Выключение VM',qmsnapshot:'Snapshot VM',vzstart:'Запуск LXC',vzstop:'Остановка LXC',vzsnapshot:'Snapshot LXC',qmigrate:'Миграция',aptupdate:'Обновление пакетов'})[type]||type;
@@ -683,8 +839,17 @@ function renderTasks(tasks,filter='all'){currentTasks=tasks;const shown=tasks.fi
function renderActivity(activity,guests,services,zfs){const snapshots=activity.snapshots||[],tasks=activity.tasks||[],old=snapshots.filter(v=>v.ageSeconds>30*86400),guestCount=new Set(snapshots.map(v=>v.vmid)).size,failed=tasks.filter(v=>v.status&&v.status!=='OK');el('snapshotCount').textContent=snapshots.length;el('oldSnapshots').textContent=old.length;el('snapshotGuests').textContent=guestCount;el('snapshotBar').style.width=Math.min(old.length*20,100)+'%';el('snapshotList').innerHTML=snapshots.length?snapshots.map(v=>'<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(v.guestName||('VMID '+v.vmid))+' · '+safe(v.name)+'</div><div class="disk-meta">VMID '+v.vmid+' · '+safe(v.guestType)+(v.createdAt?' · '+new Date(v.createdAt*1000).toLocaleString('ru-RU'):'')+'</div></div><span class="badge '+(v.ageSeconds>30*86400?'warn':'')+'">'+(v.ageSeconds?uptime(v.ageSeconds):'Без даты')+'</span></div><div class="disk-stats"><div class="disk-stat"><span>Размер</span><b>'+(v.sizeBytes?size(v.sizeBytes):'Storage не сообщает')+'</b></div><div class="disk-stat"><span>Состояние VM</span><b>'+(v.running?'Запущена':'Остановлена')+'</b></div><div class="disk-stat"><span>Описание</span><b>'+safe(v.description||'—')+'</b></div></div></article>').join(''):'<span class="empty">Snapshots не найдены.</span>';el('taskCount').textContent=tasks.length;el('failedTasks').textContent=failed.length;el('lastTask').textContent=tasks.length?taskLabel(tasks[0].type):'—';el('taskBar').style.width=Math.min(failed.length*20,100)+'%';renderTasks(tasks,currentTaskFilter);renderHomelabMap(guests,services,zfs)}
function topologyHost(address){if(!address)return '';try{return new URL(address.includes('://')?address:'http://'+address).hostname.replace(/^\[|\]$/g,'')}catch(e){return address.replace(/^[a-z]+:\/\//i,'').split('/')[0].replace(/:\d+$/,'')}}
function renderHomelabMap(guests,services,zfs){const guestItems=(guests.guests||[]).filter(v=>!v.template),serviceItems=services.services||[],pools=zfs.pools||[],serviceOwners=new Map();serviceItems.forEach(s=>{const host=topologyHost(s.localAddress||'');const owner=guestItems.find(g=>(g.ipAddresses||[]).some(ip=>ip===host));serviceOwners.set(s.id,owner?.vmid||0)});const serviceChip=s=>{const healthy=(s.endpoints||[]).every(v=>v.up);return '<span class="topology-chip '+(healthy?'good':'bad')+'"'+(!healthy?' tabindex="0" data-reason="Один или несколько адресов сервиса недоступны"':'')+'>'+safe(s.name)+'</span>'},storageChip=name=>{const pool=pools.find(p=>p.name===name),healthy=!pool||pool.health==='ONLINE';return '<span class="topology-chip '+(healthy?'good':'bad')+'">'+safe(name)+(pool?' · '+fixed(pool.capacityPercent)+'%':'')+'</span>'};const groups=guestItems.map(g=>{const owned=serviceItems.filter(s=>serviceOwners.get(s.id)===g.vmid),storage=[...new Set(g.storage||[])],running=g.status==='running';return '<section class="topology-group"><div class="topology-group-head"><div><strong>'+safe(g.name||('VMID '+g.vmid))+'</strong><div class="disk-meta">VMID '+g.vmid+' · '+safe(g.type==='qemu'?'VM':'LXC')+' · '+safe((g.ipAddresses||[]).join(', ')||'IP не получен')+'</div></div><span class="badge '+(running?'':'warn')+'"'+(!running?' tabindex="0" data-reason="VM/LXC сейчас остановлена"':'')+'>'+(running?'Работает':'Остановлена')+'</span></div><div class="topology-section"><div class="topology-section-title">Сервисы</div><div class="topology-items">'+(owned.length?owned.map(serviceChip).join(''):'<span class="empty">Не сопоставлены</span>')+'</div></div><div class="topology-section"><div class="topology-section-title">Storage</div><div class="topology-items">'+(storage.length?storage.map(storageChip).join(''):'<span class="empty">Не определён</span>')+'</div></div></section>'}).join(''),unassigned=serviceItems.filter(s=>!serviceOwners.get(s.id)),usedStorage=new Set(guestItems.flatMap(g=>g.storage||[])),unassignedPools=pools.filter(p=>!usedStorage.has(p.name));el('mapBoard').innerHTML='<div class="topology-host map-node good"><strong>Proxmox host</strong><small>'+safe(location.hostname)+' · '+guestItems.length+' VM/LXC</small></div><div class="topology-grid">'+groups+'</div>'+((unassigned.length||unassignedPools.length)?'<section class="topology-group topology-unassigned"><div class="topology-group-head"><strong>Без автоматической привязки</strong><span class="badge warn" tabindex="0" data-reason="Для сервиса не найден совпадающий IP VM/LXC либо storage не используется в конфигурации VM/LXC">Проверьте связи</span></div><div class="topology-section"><div class="topology-section-title">Сервисы</div><div class="topology-items">'+unassigned.map(serviceChip).join('')+'</div></div><div class="topology-section"><div class="topology-section-title">Остальные ZFS-пулы</div><div class="topology-items">'+unassignedPools.map(p=>storageChip(p.name)).join('')+'</div></div></section>':'');const storage=new Set([...usedStorage,...pools.map(p=>p.name)]);el('mapGuests').textContent=guestItems.length;el('mapResources').textContent=serviceItems.length+' / '+storage.size;el('mapNodes').textContent=1+guestItems.length+serviceItems.length+storage.size;el('mapBar').style.width='100%'}
function agentHTML(a,settings=false){
const r=a.report||{},containers=r.containers||[],services=r.services||[],issues=r.issues||[],failed=services.filter(v=>v.state==='failed'||v.subState==='failed'),interfaces=r.interfaces||[],memoryPercent=r.memoryTotal?r.memoryUsed/r.memoryTotal*100:0,diskPercent=r.rootTotal?r.rootUsed/r.rootTotal*100:0;
return '<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(a.name||a.hostname)+'</div><div class="disk-meta">'+safe(a.hostname)+' · VMID '+(a.vmid||'не указан')+' · '+safe(r.osName||r.os||'Linux')+' · '+safe(r.kernel||'ядро не определено')+' · агент '+safe(a.version||'—')+'</div></div><span class="badge '+(a.online?'':'bad')+'">'+(a.online?'Онлайн':'Не отвечает')+'</span></div><div class="disk-stats"><div class="disk-stat"><span>CPU / Load</span><b>'+fixed(r.cpuPercent||0)+'% / '+fixed(r.load1||0)+'</b></div><div class="disk-stat"><span>Память</span><b>'+size(r.memoryUsed)+' · '+fixed(memoryPercent)+'%</b></div><div class="disk-stat"><span>Раздел /</span><b>'+size(r.rootUsed)+' · '+fixed(diskPercent)+'%</b></div><div class="disk-stat"><span>Uptime</span><b>'+uptime(r.uptimeSeconds||0)+'</b></div><div class="disk-stat"><span>Обновления</span><b>'+(r.updatesAvailable||0)+' пакетов'+(r.rebootRequired?' · нужен reboot':'')+'</b></div><div class="disk-stat"><span>Проблемы</span><b>'+failed.length+' служб · '+issues.length+' ошибок</b></div></div><div class="topology-section"><div class="topology-section-title">IP-адреса и интерфейсы</div><div class="topology-items">'+(interfaces.length?interfaces.map(v=>'<span class="topology-chip">'+safe(v.name)+' · '+safe((v.addresses||[]).join(', '))+'</span>').join(''):'<span class="empty">IP-адреса не найдены</span>')+'</div></div><div class="service-actions"><button data-agent-detail="'+safe(a.id)+'">Подробнее</button>'+(settings?'<button data-agent-delete="'+safe(a.id)+'">Удалить агент</button>':'')+'</div></article>'
}
let currentAgents=[];function renderAgents(items){const agents=currentAgents=items||[],online=agents.filter(v=>v.online),containers=agents.flatMap(v=>(v.report&&v.report.containers)||[]),failedServices=agents.flatMap(v=>(v.report&&v.report.services)||[]).filter(v=>v.state==='failed'||v.subState==='failed'),problems=(agents.length-online.length)+failedServices.length+containers.filter(v=>v.state!=='running'||v.health==='unhealthy').length;el('agentOnline').textContent=online.length;el('agentTotal').textContent=' / '+agents.length;el('agentBar').style.width=(agents.length?online.length/agents.length*100:0)+'%';el('agentContainers').textContent=containers.length;el('agentProblems').textContent=problems;el('agentList').innerHTML=agents.length?agents.map(v=>agentHTML(v)).join(''):'<span class="empty">Агенты пока не подключены. Создайте команду установки в разделе «Управление».</span>';el('agentSettingsList').innerHTML=agents.length?agents.map(v=>agentHTML(v,true)).join(''):'<span class="empty">Агентов пока нет.</span>'}
function renderAgentServices(services,filter='all'){const failed=v=>v.state==='failed'||v.subState==='failed',custom=v=>['added','local','runtime'].includes(v.origin),filtered=(services||[]).filter(v=>filter==='all'||filter==='failed'&&failed(v)||filter==='system'&&v.origin==='system'||filter==='custom'&&custom(v)),originLabel=v=>v.origin==='local'?'Локальная':v.origin==='added'?'Добавленный пакет':v.origin==='runtime'?'Временная':'Системная';el('agentServiceDetails').innerHTML='<div class="agent-filter"><button data-agent-service-filter="all" class="'+(filter==='all'?'active':'')+'">Все · '+services.length+'</button><button data-agent-service-filter="system" class="'+(filter==='system'?'active':'')+'">Системные · '+services.filter(v=>v.origin==='system').length+'</button><button data-agent-service-filter="custom" class="'+(filter==='custom'?'active':'')+'">Добавленные · '+services.filter(custom).length+'</button><button data-agent-service-filter="failed" class="'+(filter==='failed'?'active':'')+'">Упали · '+services.filter(failed).length+'</button></div><div class="disk-list" style="padding:0">'+(filtered.length?filtered.map(v=>'<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(v.name)+'</div><div class="disk-meta">'+safe(v.description||'Описание отсутствует')+'</div><div class="disk-meta">'+originLabel(v)+(v.package?' · пакет '+safe(v.package):'')+(v.unitPath?' · '+safe(v.unitPath):'')+'</div></div><span class="badge '+(failed(v)?'bad':'')+'">'+safe(v.subState||v.state)+'</span></div><div class="disk-stats"><div class="disk-stat"><span>Запущена</span><b>'+(v.startedAt?new Date(v.startedAt*1000).toLocaleString('ru-RU'):'—')+'</b></div><div class="disk-stat"><span>Рестарты</span><b>'+(v.restarts||0)+'</b></div><div class="disk-stat"><span>Exit code</span><b>'+(v.exitCode||0)+'</b></div><div class="disk-stat"><span>Память</span><b>'+(v.memoryBytes?size(v.memoryBytes):'Нет accounting')+'</b></div><div class="disk-stat"><span>CPU time</span><b>'+(v.cpuSeconds?fixed(v.cpuSeconds)+' сек.':'Нет accounting')+'</b></div></div></article>').join(''):'<span class="empty">Служб в этой группе нет.</span>')+'</div>';el('agentServiceDetails').querySelectorAll('[data-agent-service-filter]').forEach(button=>button.onclick=()=>renderAgentServices(services,button.dataset.agentServiceFilter))}
function openAgentDetail(agent){const r=agent.report||{},services=r.services||[],issues=r.issues||[],containers=r.containers||[],interfaces=r.interfaces||[],memoryPercent=r.memoryTotal?r.memoryUsed/r.memoryTotal*100:0,diskPercent=r.rootTotal?r.rootUsed/r.rootTotal*100:0;el('agentDetailTitle').textContent=agent.name||agent.hostname;el('agentDetailMeta').textContent=(agent.hostname||'')+' · VMID '+(agent.vmid||'не указан')+' · '+(r.osName||r.os||'Linux')+' · '+(r.kernel||'ядро не определено')+' · агент '+(agent.version||'—');el('agentOverview').innerHTML='<div class="agent-detail-summary"><div><span class="label">Состояние</span><b>'+(agent.online?'Онлайн':'Не отвечает')+'</b></div><div><span class="label">CPU / Load</span><b>'+fixed(r.cpuPercent||0)+'% / '+fixed(r.load1||0)+'</b></div><div><span class="label">Память</span><b>'+size(r.memoryUsed)+' · '+fixed(memoryPercent)+'%</b></div><div><span class="label">Раздел /</span><b>'+size(r.rootUsed)+' · '+fixed(diskPercent)+'%</b></div><div><span class="label">Uptime</span><b>'+uptime(r.uptimeSeconds||0)+'</b></div><div><span class="label">Обновления</span><b>'+(r.updatesAvailable||0)+' пакетов'+(r.rebootRequired?' · нужен reboot':'')+'</b></div></div><div class="topology-section"><div class="topology-section-title">IP-адреса</div><div class="topology-items">'+(interfaces.length?interfaces.map(v=>'<span class="topology-chip">'+safe(v.name)+' · '+safe((v.addresses||[]).join(', '))+'</span>').join(''):'<span class="empty">Не найдены</span>')+'</div></div><div class="settings-note">Последний heartbeat: '+new Date(agent.lastSeen*1000).toLocaleString('ru-RU')+' · системный инвентарь: '+(r.inventoryAt?new Date(r.inventoryAt*1000).toLocaleString('ru-RU'):'ещё не собран')+'</div>';renderAgentServices(services);el('agentIssueDetails').innerHTML='<div class="disk-list" style="padding:0">'+(issues.length?issues.map(v=>'<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(v.unit||'system')+'</div><div class="disk-meta">'+(v.time?new Date(v.time*1000).toLocaleString('ru-RU'):'Время не указано')+'</div></div><span class="badge bad">Ошибка</span></div><div class="settings-note issue-message">'+safe(v.message)+'</div></article>').join(''):'<span class="empty">Ошибок уровня err и выше за 24 часа нет.</span>')+'</div>';el('agentDockerDetails').innerHTML='<div class="disk-list" style="padding:0">'+(containers.length?containers.map(v=>'<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(v.name)+'</div><div class="disk-meta">'+safe(v.image)+' · '+safe(v.id)+'</div></div><span class="badge '+(v.state==='running'&&v.health!=='unhealthy'?'':'bad')+'">'+safe(v.health||v.state)+'</span></div><div class="disk-stats"><div class="disk-stat"><span>Статус</span><b>'+safe(v.status)+'</b></div><div class="disk-stat"><span>CPU</span><b>'+fixed(v.cpuPercent||0)+'%</b></div><div class="disk-stat"><span>Память</span><b>'+size(v.memoryUsage||0)+'</b></div><div class="disk-stat"><span>Рестарты</span><b>'+v.restarts+'</b></div><div class="disk-stat"><span>Порты</span><b>'+safe((v.ports||[]).join(', ')||'—')+'</b></div></div></article>').join(''):'<span class="empty">Docker-контейнеры не обнаружены.</span>')+'</div>';const first=document.querySelector('[data-agent-tab="overview"]');document.querySelectorAll('[data-agent-tab]').forEach(t=>t.classList.toggle('active',t===first));document.querySelectorAll('[data-agent-panel]').forEach(p=>p.hidden=p.dataset.agentPanel!=='overview');el('agentDetailDialog').showModal()}
function render(x){const c=x.cpu,m=x.memory,s=x.storage,n=x.network,z=x.zfs||{available:false,pools:[]},u=x.ups||{available:false},g=x.guests||{available:false,guests:[]},b=x.backups||{available:false,items:[]},mt=x.maintenance||{},alerts=x.alerts||[],disks=x.disks||[],avg=c.temperatures.length?c.temperatures.reduce((a,t)=>a+t.celsius,0)/c.temperatures.length:null;
lastDashboardEvent=Date.now();
maintenanceGuests=(g.guests||[]).filter(v=>!v.template);applyMaintenanceState(x.maintenanceWindow||{});
renderAgents(x.agents||[]);
const update=x.update||{},updating=['queued','installing'].includes(update.state);el('updateCurrent').textContent=update.currentVersion||'—';el('updateAvailable').textContent=update.availableVersion||'—';el('updateMode').textContent=update.mode==='auto'?'Автоматически':'Уведомлять';el('updateMessage').textContent=update.message||'Проверка ещё не выполнялась';el('updateChecked').textContent=update.checkedAt?'Последняя проверка: '+new Date(update.checkedAt).toLocaleString('ru-RU'):'Проверка запускается автоматически раз в сутки.';el('installUpdate').disabled=updating;el('installUpdate').textContent=updating?'Обновление выполняется…':'Проверить и установить обновление';
const critical=alerts.filter(a=>a.severity==='critical').length,warnings=alerts.filter(a=>a.severity==='warning').length,infos=alerts.filter(a=>a.severity==='info').length;el('alertCount').textContent=alerts.length;el('alertCritical').textContent=critical;el('alertWarnings').textContent=warnings;el('alertHeadline').textContent=critical?'Требуется срочное внимание':warnings?'Есть предупреждения':infos?'Есть информационные события':'Система в норме';el('alertBar').style.width=(critical?100:warnings?65:infos?30:0)+'%';
el('alertList').innerHTML=alerts.length?alerts.map(a=>'<article class="alert-item '+safe(a.severity)+'"><i class="alert-icon"></i><div><div class="alert-source">'+safe(a.source)+' · '+(a.severity==='critical'?'Критическое':a.severity==='warning'?'Предупреждение':'Информация')+(a.firstSeen?' · С '+new Date(a.firstSeen*1000).toLocaleString('ru-RU'):'')+'</div><div class="alert-title">'+safe(a.title)+'</div><div class="alert-message">'+safe(a.message)+'</div><button class="ack-alert" data-ack="'+a.eventId+'">Принято</button></div></article>').join(''):'<div class="all-good"><strong>Всё в порядке</strong><span>Новых неподтверждённых событий нет.</span></div>';
@@ -726,27 +891,38 @@ document.querySelectorAll('[data-open]').forEach(b=>b.onclick=()=>el(b.dataset.o
document.querySelectorAll('dialog').forEach(d=>{d.querySelector('.close').onclick=()=>d.close();d.onclick=e=>{if(e.target===d)d.close()}});
document.querySelectorAll('[data-system-tab]').forEach(tab=>tab.onclick=()=>{document.querySelectorAll('[data-system-tab]').forEach(v=>v.classList.toggle('active',v===tab));document.querySelectorAll('[data-system-panel]').forEach(v=>v.hidden=v.dataset.systemPanel!==tab.dataset.systemTab)});
el('systemSort').onclick=()=>{systemSortNewest=!systemSortNewest;renderSystemHealth(currentSystemHealth)};
document.querySelectorAll('[data-agent-tab]').forEach(tab=>tab.onclick=()=>{document.querySelectorAll('[data-agent-tab]').forEach(v=>v.classList.toggle('active',v===tab));document.querySelectorAll('[data-agent-panel]').forEach(v=>v.hidden=v.dataset.agentPanel!==tab.dataset.agentTab)});
el('agentList').onclick=event=>{const button=event.target.closest('[data-agent-detail]');if(!button)return;const agent=currentAgents.find(v=>v.id===button.dataset.agentDetail);if(agent)openAgentDetail(agent)};
el('serviceList').onclick=event=>{const button=event.target.closest('[data-service-open]');if(!button)return;const service=currentServiceStatuses.find(v=>v.id===Number(button.dataset.serviceOpen));if(!service)return;const choices=(service.endpoints||[]).filter(v=>/^https?:\/\//.test(v.address));if(choices.length===1){window.open(choices[0].address,'_blank','noopener');return}if(!choices.length)return;el('openServiceTitle').textContent=service.name;el('openServiceChoices').innerHTML=choices.map(v=>'<a class="theme-option" href="'+safe(v.address)+'" target="_blank" rel="noopener">'+safe(v.label)+'<small style="display:block;color:var(--muted);margin-top:4px">'+safe(v.address)+'</small></a>').join('');el('openServiceDialog').showModal()};
document.querySelectorAll('[data-task-filter]').forEach(button=>button.onclick=()=>{currentTaskFilter=button.dataset.taskFilter;document.querySelectorAll('[data-task-filter]').forEach(v=>v.classList.toggle('active',v===button));renderTasks(currentTasks,currentTaskFilter);explainWarningBadges()});
el('backupList').onclick=async event=>{const button=event.target.closest('[data-verify-backup]');if(!button)return;const target=el('backupList').querySelector('[data-backup-result="'+CSS.escape(button.dataset.verifyBackup)+'"]');button.disabled=true;target.textContent='Проверяем метаданные и читаемость…';try{const response=await fetch('/api/backups/verify',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({volid:button.dataset.verifyBackup})}),result=await response.json();target.textContent=(result.ok?'✓ ':'⚠ ')+result.message;target.className='backup-check '+(result.ok?'status-up':'status-down')}catch(e){target.textContent='Не удалось выполнить проверку.'}finally{button.disabled=false}};
el('alertList').onclick=async event=>{const button=event.target.closest('[data-ack]');if(!button)return;button.disabled=true;const response=await fetch('/api/alerts/acknowledge',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({eventId:Number(button.dataset.ack)})});if(response.ok)button.closest('.alert-item').remove();else button.disabled=false};
async function loadAlertHistory(){const response=await fetch('/api/alerts/history');if(!response.ok)return;const events=await response.json()||[];el('alertHistory').innerHTML=events.length?events.map(a=>{const state=a.acknowledgedAt?'Принято · '+new Date(a.acknowledgedAt*1000).toLocaleString('ru-RU'):a.resolvedAt?'Завершено · '+new Date(a.resolvedAt*1000).toLocaleString('ru-RU'):'Активно';return '<article class="alert-item '+safe(a.severity)+'"><i class="alert-icon"></i><div><div class="alert-source">'+safe(a.source)+' · '+new Date(a.firstSeen*1000).toLocaleString('ru-RU')+'</div><div class="alert-title">'+safe(a.title)+'</div><div class="alert-message">'+safe(a.message)+'</div><span class="history-state">'+state+'</span></div></article>'}).join(''):'<div class="all-good"><strong>История пока пуста</strong><span>Новые события появятся здесь автоматически.</span></div>'}
async function loadAlertHistory(){const response=await fetch('/api/alerts/history');if(!response.ok)return;const events=await response.json()||[];el('alertHistory').innerHTML=events.length?events.map(a=>{const state=a.planned?'Плановые работы · '+safe(a.maintenanceReason||'Причина не указана')+(a.resolvedAt?' · завершено '+new Date(a.resolvedAt*1000).toLocaleString('ru-RU'):''):a.acknowledgedAt?'Принято · '+new Date(a.acknowledgedAt*1000).toLocaleString('ru-RU'):a.resolvedAt?'Завершено · '+new Date(a.resolvedAt*1000).toLocaleString('ru-RU'):'Активно';return '<article class="alert-item '+safe(a.severity)+'"><i class="alert-icon"></i><div><div class="alert-source">'+safe(a.source)+' · '+new Date(a.firstSeen*1000).toLocaleString('ru-RU')+'</div><div class="alert-title">'+safe(a.title)+'</div><div class="alert-message">'+safe(a.message)+'</div><span class="history-state">'+state+'</span></div></article>'}).join(''):'<div class="all-good"><strong>История пока пуста</strong><span>Новые события появятся здесь автоматически.</span></div>'}
document.querySelectorAll('[data-alert-tab]').forEach(tab=>tab.onclick=()=>{document.querySelectorAll('[data-alert-tab]').forEach(t=>t.classList.toggle('active',t===tab));document.querySelectorAll('[data-alert-panel]').forEach(p=>p.hidden=p.dataset.alertPanel!==tab.dataset.alertTab);if(tab.dataset.alertTab==='history')loadAlertHistory()});
const cardNames=['alerts','cpu','memory','storage','network','zfs','ups','monitor','guests','maintenance','backups','services','system','snapshots','tasks','map'],cardLabels={alerts:'Центр уведомлений',cpu:'CPU',memory:'Память',storage:'Диски',network:'Сеть',zfs:'ZFS',ups:'UPS',monitor:'Самоконтроль',guests:'VM и LXC',maintenance:'Proxmox',backups:'Резервные копии',services:'Сервисы',system:'Система и ядро',snapshots:'Snapshots',tasks:'Задачи Proxmox',map:'Карта homelab'};let hiddenCards=[],cardOrder=[];try{hiddenCards=JSON.parse(localStorage.getItem('proxmox-hidden-cards')||'[]');cardOrder=JSON.parse(localStorage.getItem('proxmox-card-order')||'[]')}catch(e){}cardOrder=[...cardOrder.filter(x=>cardNames.includes(x)),...cardNames.filter(x=>!cardOrder.includes(x))];
const cardNames=['alerts','cpu','memory','storage','network','zfs','ups','monitor','guests','agents','maintenance','backups','services','system','snapshots','tasks','map'],cardLabels={alerts:'Центр уведомлений',cpu:'CPU',memory:'Память',storage:'Диски',network:'Сеть',zfs:'ZFS',ups:'UPS',monitor:'Самоконтроль',guests:'VM и LXC',agents:'Агенты VM и LXC',maintenance:'Proxmox',backups:'Резервные копии',services:'Сервисы',system:'Система и ядро',snapshots:'Snapshots',tasks:'Задачи Proxmox',map:'Карта homelab'};let hiddenCards=[],cardOrder=[];try{hiddenCards=JSON.parse(localStorage.getItem('proxmox-hidden-cards')||'[]');cardOrder=JSON.parse(localStorage.getItem('proxmox-card-order')||'[]')}catch(e){}cardOrder=[...cardOrder.filter(x=>cardNames.includes(x)),...cardNames.filter(x=>!cardOrder.includes(x))];
function renderCardOrder(){el('cardOrder').innerHTML=cardOrder.map(name=>'<div class="order-item" draggable="true" data-order-card="'+name+'"><span class="order-handle">⠿</span><span>'+cardLabels[name]+'</span><button class="order-button" data-move="up" aria-label="Выше">↑</button><button class="order-button" data-move="down" aria-label="Ниже">↓</button></div>').join('')}
function saveCardOrder(){localStorage.setItem('proxmox-card-order',JSON.stringify(cardOrder));applyCardSettings();renderCardOrder()}
function applyCardSettings(){const container=document.querySelector('.cards');cardOrder.forEach(name=>{const card=document.querySelector('[data-card="'+name+'"]');if(card)container.appendChild(card)});cardNames.forEach(name=>{const visible=!hiddenCards.includes(name),card=document.querySelector('[data-card="'+name+'"]'),toggle=document.querySelector('[data-card-toggle="'+name+'"]');if(card)card.hidden=!visible;if(toggle)toggle.checked=visible})}
document.querySelectorAll('[data-card-toggle]').forEach(toggle=>toggle.onchange=()=>{const name=toggle.dataset.cardToggle;hiddenCards=toggle.checked?hiddenCards.filter(v=>v!==name):[...new Set([...hiddenCards,name])];localStorage.setItem('proxmox-hidden-cards',JSON.stringify(hiddenCards));applyCardSettings()});renderCardOrder();applyCardSettings();
el('cardOrder').onclick=event=>{const button=event.target.closest('[data-move]'),item=event.target.closest('[data-order-card]');if(!button||!item)return;const index=cardOrder.indexOf(item.dataset.orderCard),next=button.dataset.move==='up'?index-1:index+1;if(next<0||next>=cardOrder.length)return;[cardOrder[index],cardOrder[next]]=[cardOrder[next],cardOrder[index]];saveCardOrder()};let draggedCard='';el('cardOrder').addEventListener('dragstart',event=>{const item=event.target.closest('[data-order-card]');if(!item)return;draggedCard=item.dataset.orderCard;item.classList.add('dragging')});el('cardOrder').addEventListener('dragend',event=>{event.target.closest('[data-order-card]')?.classList.remove('dragging');draggedCard=''});el('cardOrder').addEventListener('dragover',event=>event.preventDefault());el('cardOrder').addEventListener('drop',event=>{event.preventDefault();const target=event.target.closest('[data-order-card]');if(!target||!draggedCard||target.dataset.orderCard===draggedCard)return;const from=cardOrder.indexOf(draggedCard),to=cardOrder.indexOf(target.dataset.orderCard);cardOrder.splice(from,1);cardOrder.splice(to,0,draggedCard);saveCardOrder()});
function setTheme(theme){document.documentElement.dataset.theme=theme;localStorage.setItem('proxmox-theme',theme);document.querySelectorAll('[data-theme-choice]').forEach(b=>b.classList.toggle('active',b.dataset.themeChoice===theme));document.querySelectorAll('.chart-section:not([hidden])').forEach(showHistory)}document.querySelectorAll('[data-theme-choice]').forEach(b=>b.onclick=()=>setTheme(b.dataset.themeChoice));setTheme(document.documentElement.dataset.theme);el('settingsButton').onclick=()=>el('settingsDialog').showModal();
['services','agents','maintenance-mode','ups'].forEach((name,index)=>{const panel=document.querySelector('[data-settings-panel="'+name+'"]');if(panel){panel.dataset.managementPanel=name;panel.hidden=index!==0;el('managementPanels').appendChild(panel)}});
function selectManagement(tab){document.querySelectorAll('[data-management-tab]').forEach(t=>t.classList.toggle('active',t===tab));document.querySelectorAll('[data-management-panel]').forEach(p=>p.hidden=p.dataset.managementPanel!==tab.dataset.managementTab)}el('managementButton').onclick=()=>{selectManagement(document.querySelector('[data-management-tab].active')||document.querySelector('[data-management-tab]'));el('managementDialog').showModal()};document.querySelectorAll('[data-management-tab]').forEach(tab=>tab.onclick=()=>selectManagement(tab));
document.querySelectorAll('[data-settings-tab]').forEach(tab=>tab.onclick=()=>{document.querySelectorAll('[data-settings-tab]').forEach(t=>t.classList.toggle('active',t===tab));document.querySelectorAll('[data-settings-panel]').forEach(p=>p.hidden=p.dataset.settingsPanel!==tab.dataset.settingsTab)});
el('agentServerURL').value=location.origin;
el('createAgentToken').onclick=async()=>{const button=el('createAgentToken'),server=el('agentServerURL').value.trim().replace(/\/$/,''),vmid=Number(el('agentVMID').value)||0;if(!/^https?:\/\//.test(server)){alert('Укажите полный адрес Dashboard, доступный из VM/LXC.');return}button.disabled=true;button.textContent='Создаём токен…';try{const response=await fetch('/api/agents/enrollment-token',{method:'POST'});if(!response.ok)throw new Error(await response.text());const value=await response.json(),quote=v=>"'"+String(v).replaceAll("'","'\"'\"'")+"'";el('agentInstallCommand').textContent='curl -fsSL https://git.myown.center/maxim/ProxmoxDash/raw/branch/main/scripts/install-agent.sh | DASHBOARD_URL='+quote(server)+' ENROLL_TOKEN='+quote(value.token)+' VMID='+quote(vmid)+' sh';el('agentInstallBlock').hidden=false}catch(e){alert('Не удалось создать токен: '+e.message)}finally{button.disabled=false;button.textContent='Создать команду установки или обновления'}};
el('agentSettingsList').onclick=async event=>{const detail=event.target.closest('[data-agent-detail]');if(detail){const agent=currentAgents.find(v=>v.id===detail.dataset.agentDetail);if(agent)openAgentDetail(agent);return}const button=event.target.closest('[data-agent-delete]');if(!button||!confirm('Удалить агент? Для повторного подключения потребуется новый токен.'))return;button.disabled=true;const response=await fetch('/api/agents?id='+encodeURIComponent(button.dataset.agentDelete),{method:'DELETE'});if(response.ok)button.closest('.disk-item').remove();else{button.disabled=false;alert(await response.text())}};
async function loadThresholds(){const response=await fetch('/api/settings/alerts');if(!response.ok)return;alertSettings=await response.json();document.querySelectorAll('[data-threshold]').forEach(input=>input.value=alertSettings[input.dataset.threshold]??'')}
el('saveThresholds').onclick=async()=>{const values={};document.querySelectorAll('[data-threshold]').forEach(input=>values[input.dataset.threshold]=input.dataset.threshold==='diskIoExcluded'?input.value:Number(input.value));const status=el('thresholdStatus');status.textContent='Сохранение…';const response=await fetch('/api/settings/alerts',{method:'PUT',headers:{'Content-Type':'application/json'},body:JSON.stringify(values)});if(response.ok){alertSettings=await response.json();status.textContent='Пороги сохранены и уже применяются.'}else{status.textContent=await response.text()||'Не удалось сохранить настройки.'}};loadThresholds().catch(()=>{});
let configuredServices=[];
async function loadServices(){const response=await fetch('/api/services');if(!response.ok)return;configuredServices=await response.json()||[];el('serviceSettingsList').innerHTML=configuredServices.length?configuredServices.map(v=>'<article class="disk-item"><div class="disk-top"><div><div class="disk-model">'+safe(v.name)+'</div><div class="disk-meta">'+safe([v.url&&('Внешний: '+v.url),v.localAddress&&('Локальный: '+v.localAddress)].filter(Boolean).join(' · '))+'</div></div><span class="badge '+(v.enabled?'':'warn')+'">'+(v.enabled?'Включён':'Выключен')+'</span></div><div class="service-actions"><button data-service-edit="'+v.id+'">Изменить</button><button data-service-delete="'+v.id+'">Удалить</button></div></article>').join(''):'<span class="empty">Список пуст.</span>'}
function resetServiceForm(){el('serviceId').value='';el('serviceName').value='';el('serviceType').value='http';el('serviceURL').value='';el('serviceLocal').value='';el('serviceCode').value=200;el('serviceText').value='';el('serviceTimeout').value=5;el('serviceEnabled').checked=true}
el('saveService').onclick=async()=>{const value={id:Number(el('serviceId').value)||0,name:el('serviceName').value.trim(),checkType:el('serviceType').value,url:el('serviceURL').value.trim(),localAddress:el('serviceLocal').value.trim(),expectedCode:Number(el('serviceCode').value)||200,expectedText:el('serviceText').value,timeoutSeconds:Number(el('serviceTimeout').value)||5,enabled:el('serviceEnabled').checked},status=el('serviceStatus');status.textContent='Сохранение…';const response=await fetch('/api/services',{method:value.id?'PUT':'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(value)});status.textContent=response.ok?'Сервис сохранён. Первая проверка появится в течение 15 секунд.':await response.text();if(response.ok){resetServiceForm();loadServices()}};
el('serviceSettingsList').onclick=async event=>{const edit=event.target.closest('[data-service-edit]'),del=event.target.closest('[data-service-delete]');if(edit){const v=configuredServices.find(x=>x.id===Number(edit.dataset.serviceEdit));if(!v)return;el('serviceId').value=v.id;el('serviceName').value=v.name;el('serviceType').value=v.checkType;el('serviceURL').value=v.url;el('serviceLocal').value=v.localAddress||'';el('serviceCode').value=v.expectedCode;el('serviceText').value=v.expectedText||'';el('serviceTimeout').value=v.timeoutSeconds;el('serviceEnabled').checked=v.enabled;return}if(del&&confirm('Удалить сервис и его историю?')){const response=await fetch('/api/services?id='+del.dataset.serviceDelete,{method:'DELETE'});if(response.ok)loadServices()}};loadServices().catch(()=>{});
el('serviceSettingsList').onclick=async event=>{const edit=event.target.closest('[data-service-edit]'),del=event.target.closest('[data-service-delete]');if(edit){const v=configuredServices.find(x=>x.id===Number(edit.dataset.serviceEdit));if(!v)return;el('serviceId').value=v.id;el('serviceName').value=v.name;el('serviceType').value=v.checkType;el('serviceURL').value=v.url;el('serviceLocal').value=v.localAddress||'';el('serviceCode').value=v.expectedCode;el('serviceText').value=v.expectedText||'';el('serviceTimeout').value=v.timeoutSeconds;el('serviceEnabled').checked=v.enabled;return}if(del&&confirm('Удалить сервис и его историю?')){const response=await fetch('/api/services?id='+del.dataset.serviceDelete,{method:'DELETE'});if(response.ok)loadServices()}};loadServices().then(()=>{maintenanceTargetSignature='';renderMaintenanceTargets()}).catch(()=>{});
el('maintenanceAll').onchange=toggleMaintenanceTargets;
async function loadMaintenance(){const response=await fetch('/api/settings/maintenance');if(!response.ok)return;applyMaintenanceState(await response.json())}
el('startMaintenance').onclick=async()=>{const duration=Number(el('maintenanceDuration').value),value={endsAt:Math.floor(Date.now()/1000)+duration*60,reason:el('maintenanceReason').value.trim(),allAlerts:el('maintenanceAll').checked,serviceIds:[...document.querySelectorAll('[data-maintenance-service]:checked')].map(v=>Number(v.dataset.maintenanceService)),vmids:[...document.querySelectorAll('[data-maintenance-vm]:checked')].map(v=>Number(v.dataset.maintenanceVm))},status=el('maintenanceStatus');status.textContent='Включаем режим…';const response=await fetch('/api/settings/maintenance',{method:'PUT',headers:{'Content-Type':'application/json'},body:JSON.stringify(value)});if(response.ok){maintenanceTargetSignature='';applyMaintenanceState(await response.json())}else status.textContent=await response.text()};
el('stopMaintenance').onclick=async()=>{const response=await fetch('/api/settings/maintenance',{method:'DELETE'});if(response.ok){maintenanceTargetSignature='';applyMaintenanceState(await response.json())}else el('maintenanceStatus').textContent=await response.text()};loadMaintenance().catch(()=>{});
function upsShutdownForm(){return {enabled:el('upsShutdownEnabled').checked,chargePercent:Number(el('upsShutdownCharge').value),runtimeSeconds:Number(el('upsShutdownRuntime').value),graceSeconds:Number(el('upsShutdownGrace').value),guestOrder:el('upsShutdownOrder').value.split(',').map(v=>Number(v.trim())).filter(v=>v>0)}}
async function loadUPSShutdown(){const response=await fetch('/api/settings/ups-shutdown');if(!response.ok)return;const s=await response.json();el('upsShutdownEnabled').checked=!!s.enabled;el('upsShutdownCharge').value=s.chargePercent||15;el('upsShutdownRuntime').value=s.runtimeSeconds||600;el('upsShutdownGrace').value=s.graceSeconds||120;el('upsShutdownOrder').value=(s.guestOrder||[]).join(', ')}
el('saveUPSShutdown').onclick=async()=>{const status=el('upsShutdownStatus');status.textContent='Сохранение…';const response=await fetch('/api/settings/ups-shutdown',{method:'PUT',headers:{'Content-Type':'application/json'},body:JSON.stringify(upsShutdownForm())});status.textContent=response.ok?(upsShutdownForm().enabled?'Сценарий сохранён и разрешён.':'Сценарий сохранён, автоматическое выполнение отключено.'):await response.text()};