7 Commits

22 changed files with 1772 additions and 74 deletions

129
README.md
View File

@@ -586,3 +586,132 @@ HTTP-проверки остаются в истории и возвращают
новые правила сразу применяются и к старым записям. Срабатывания и решения новые правила сразу применяются и к старым записям. Срабатывания и решения
CrowdSec, а также общие счётчики firewall, никогда не скрываются этим фильтром. CrowdSec, а также общие счётчики firewall, никогда не скрываются этим фильтром.
Сетевые потоки по-прежнему хранятся только последним снимком, без истории. Сетевые потоки по-прежнему хранятся только последним снимком, без истории.
## Alpine LXC и Docker NPM/NPM+ (v0.23.0)
Установщик агента поддерживает Linux **x86_64/amd64** с systemd или OpenRC
(Alpine LXC). Для Alpine команда из настроек сначала устанавливает curl и
CA-сертификаты, если curl отсутствует. Бинарник статический, без glibc;
скачивание проверяется по SHA-256. OpenRC-служба включается в default runlevel
и использует supervise-daemon: перезапускает агент после ошибки и обновления.
Команды: `rc-service proxmox-dashboard-agent status/restart` (выберите одно
действие). Журнал: `/var/log/proxmox-dashboard-agent.log`; его ротацию настройте
средствами ОС. Удаление — штатным `scripts/uninstall-agent.sh`.
Обновление пакетов ОС и управление systemd-службами из интерфейса остаются
функциями Debian/Ubuntu/systemd; этот релиз добавляет установку и работу агента
на OpenRC, а не эмуляцию systemd. Включённый сетевой сбор на Alpine ставит
`conntrack-tools` через `apk add --no-cache`. Ограничения CAP_NET_ADMIN и
сетевого namespace LXC сохраняются; никаких изменений конфигурации Proxmox,
privileged/nesting, firewall или sysctl агент не делает.
После выбора агента Docker-хоста и включения карты трафика агент автоматически:
- проверяет стандартные каталоги NPM и NPM+;
- находит запущенные контейнеры с nginx-proxy-manager/npmplus в имени образа
и определяет каталоги логов по mounts `/data`, включая именованные volumes
и вложенные bind mounts;
- читает обычный формат NPM и формат NPM+ `access.log`, включая ротацию;
- находит единственный контейнер образа `crowdsecurity/crowdsec` и выполняет
только `cscli alerts list` через локальный Docker socket. Docker CLI, новый
bouncer key и опубликованный LAPI не нужны. Если контейнера нет, используется
прежний локальный `cscli`.
Поиск повторяется раз в минуту, поэтому пересоздание контейнеров подхватывается
автоматически. При нескольких CrowdSec задайте точный контейнер вручную:
`DASHBOARD_CROWDSEC_CONTAINER=имя` в `/etc/proxmox-dashboard-agent.env` и
перезапустите агент. Каталог логов можно переопределить переменной
`DASHBOARD_NPM_LOG_DIR=/путь/к/логам`; `off` выключает сбор трафика этого агента.
Используется локальный `/var/run/docker.sock`; rootless/удалённые Docker engines
и логи только в stdout этим обнаружением не поддерживаются.
NPM+ должен действительно писать access-логи: включите `LOGROTATE=true` в его
Compose и пересоздайте контейнер, если логирование выключено. Агент показывает
подсказку при отсутствии логов, но не меняет Compose приложений. Это настройка
[NPM+](https://github.com/ZoeyVid/NPMplus/blob/develop/README.md), а не Dashboard.
Stream-логи NPM+ не считаются HTTP-запросами; сетевые потоки собираются conntrack.
Как и раньше, первый запуск чтения начинается с конца существующих логов.
## Интерактивная карта (v0.24.0)
Карта трафика использует ту же настройку `proxmox-theme`, что и основной
Dashboard. Кнопка солнца/луны меняет тему; выбор сохраняется в браузере.
Светлая тема распространяется на карту, график, карточки и модальные окна.
Масштаб карты — от 100% до 600%: кнопки +/−, колесо мыши или жест двумя
пальцами. Увеличенную карту можно перетаскивать; ↺ возвращает весь мир.
При фокусе на карте доступны клавиши +/−, 0 и стрелки. Наведение подсвечивает
контур страны и показывает суточные счётчики, а нажатие открывает подробности.
Четыре карточки статистики открывают окна с итогом, объяснением метрики,
разбивкой по странам и примерами событий. Счётчики стран считаются сервером
по всей доступной истории за 24 часа с текущими фильтрами. Начиная с v0.25.0
история загружается целиком по страницам, без ограничения последними 100 событиями. Открытое
окно фиксирует данные на момент открытия; временная отметка показана внутри.
Сетевые потоки остаются отдельным текущим снимком, не суточной статистикой.
## Карта и полная история (v0.25.0)
Карта использует векторные границы стран: при зуме 100–600% они перерисовываются,
а маркеры и подписи сохраняют размер. Это обзорная карта стран, не уличная карта.
Карточки и окна стран загружают всю сохранённую историю за последние 24 часа
по 50 записей; фильтры учитываются сервером. В карточке IP показаны все уникальные
HTTP IP с количеством запросов. Переходы между страницами фиксируют снимок истории.
Для городов откройте настройки карты и нажмите «Установить базу городов».
Dashboard скачает DB-IP City Lite 2026-10 (~58 МБ сжатая, ~121 МБ на диске),
проверит контрольную сумму и сохранит `dbip-city.mmdb` рядом с `DASHBOARD_DB_PATH`.
IP посетителей не отправляются во внешние сервисы. Город и регион отображаются
в истории и списке IP; местоположение приблизительное и может относиться к VPN
или провайдеру. Для части IP город неизвестен. База устанавливается отдельно от
релиза, автоматически каждый месяц не обновляется.
## Фильтр истории и зеркало GeoIP (v0.25.1)
В окнах стран и событий есть локальный переключатель «Все события / HTTP /
Срабатывания / Решения CrowdSec». Он меняет тип событий только внутри окна,
сохраняя страну, домен, остальные фильтры и снимок времени; пагинация начинается
с первой страницы. Сводные карточки над историей сохраняют исходные итоги.
DB-IP City Lite загружается сначала из вложения релиза v0.25.1 в Gitea,
при ошибке — с официального сервера DB-IP. Для собственного адреса Gitea можно
задать `GITEA_URL` в окружении сервиса Dashboard. База хранится отдельно от Git-истории
и бинарника приложения; контрольная сумма проверяется независимо от источника.
Официальный файл: https://download.db-ip.com/free/dbip-city-lite-2026-10.mmdb.gz
## Города на карте (v0.26.0)
При установленной базе городов и масштабе от 200% на карте появляются города
с HTTP/CrowdSec-активностью. Маркеры используют приблизительные координаты DB-IP,
подписи не увеличиваются вместе с картой. Наведение показывает статистику,
нажатие открывает карточку города с итогами и ссылкой на всю историю страны.
HTTP/CrowdSec-импульсы также начинаются в городе, если он определён.
Счётчики городов учитывают выбранного агента и фильтры за 24 часа. На карте
показываются до 500 наиболее активных городов; остальные события и IP без города
остаются в точках стран. TCP/UDP/ICMP пока отображаются на уровне стран.
Город может относиться к провайдеру/VPN, а не фактическому местоположению человека.
## Подробности UPS (v0.27.0)
Карточка UPS показывает питание, батарею, пороги и диагностику, а также исходные
параметры `upsc` / `apcaccess`. Расчётная мощность помечается ≈ и вычисляется как
номинальная активная мощность в ваттах × процент нагрузки / 100. Если UPS передаёт
`ups.realpower`, используется измеренное значение. Без номинала оценка не строится.
Для NUT/CyberPower (CPS / vendor 0764) частота в диапазоне 450–650 Гц
корректируется делением на 10 с явной пометкой и сохранением исходного значения.
Корректные 49,8/50/60 Гц повторно не преобразуются. Настройки NUT не меняются.
Невалидные даты батареи вроде `CPS` остаются только в исходных параметрах.
История 1 час / 24 часа / 7 дней: заряд и нагрузка, мощность, входное/выходное
напряжение, напряжение батареи, автономность и частота. Запись раз в минуту,
новые ряды заполняются после обновления; при потере связи сохраняются пропуски.
Пороги UPS/NUT показаны отдельно от сценария выключения Dashboard.
## Компактная карточка UPS (v0.27.1)
Окно UPS разделено на вкладки «Обзор», «Батарея», «Диагностика», «Графики»
и «Исходные данные». В обзоре — компактные карточки основных показателей.
На графиках выбирается один показатель и период; технические пояснения
коррекции частоты и расчёта мощности находятся в исходных данных.

82
agent_installer_test.go Normal file
View File

@@ -0,0 +1,82 @@
package main
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// Run the real installer in a temporary tree with package/service/download
// commands stubbed. This must never touch the host's init system or packages.
func TestAgentInstallerOpenRC(t *testing.T) {
for _, init := range []string{"openrc", "systemd"} {
t.Run(init, func(t *testing.T) {
root := t.TempDir()
bin := filepath.Join(root, "bin")
os.MkdirAll(bin, 0755)
write := func(name, body string) {
t.Helper()
if err := os.WriteFile(filepath.Join(bin, name), []byte("#!/bin/sh\nset -eu\n"+body), 0755); err != nil {
t.Fatal(err)
}
}
write("id", "echo 0\n")
write("uname", "echo x86_64\n")
hash := strings.Repeat("a", 64)
write("curl", `case "$1" in -fsSL) ;; *) exit 4 ;; esac
if [ "$#" = 2 ]; then printf '%s' '{"browser_download_url":"https://example.test/proxmox-dashboard-linux-amd64"}'; exit; fi
case "$2" in *.sha256) printf '%s\n' '`+hash+` proxmox-dashboard-linux-amd64' > "$4" ;; *) printf '#!/bin/sh\nprintf "%%s\\n" "$@" > "$CAPTURE"\n' > "$4" ;; esac
`)
write("sha256sum", "printf '%s\\n' '"+hash+" binary'\n")
for _, name := range []string{"systemctl", "rc-service", "rc-update", "supervise-daemon"} {
write(name, `printf '%s %s\n' "$(basename "$0")" "$*" >> "$CALLS"`+"\n")
}
raw, err := os.ReadFile("scripts/install-agent.sh")
if err != nil {
t.Fatal(err)
}
// Redirect every persistent absolute path, including those in generated scripts.
s := string(raw)
for _, prefix := range []string{"/etc/", "/var/lib/", "/usr/local/", "/run/", "/var/log/"} {
s = strings.ReplaceAll(s, prefix, root+prefix)
}
for _, dir := range []string{"etc/systemd/system", "etc/init.d", "var/log"} {
os.MkdirAll(filepath.Join(root, dir), 0755)
}
if init == "systemd" {
os.MkdirAll(filepath.Join(root, "run/systemd/system"), 0755)
}
script := filepath.Join(root, "install.sh")
os.WriteFile(script, []byte(s), 0700)
env := append(os.Environ(), "PATH="+bin+":"+os.Getenv("PATH"), "DASHBOARD_URL=https://dashboard.test", "ENROLL_TOKEN=token", "AGENT_NAME=Alpine $(touch should-not-exist)", "VMID=102", "CALLS="+root+"/calls", "CAPTURE="+root+"/args")
cmd := exec.Command("sh", script)
cmd.Env = env
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("%v: %s", err, out)
}
calls, _ := os.ReadFile(root + "/calls")
if init == "openrc" {
if strings.Contains(string(calls), "systemctl") || !strings.Contains(string(calls), "rc-update add proxmox-dashboard-agent default") || !strings.Contains(string(calls), "rc-service proxmox-dashboard-agent restart") {
t.Fatalf("%s", calls)
}
cmd = exec.Command("sh", root+"/usr/local/sbin/proxmox-dashboard-agent-run")
cmd.Env = env
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("launcher: %v %s", err, out)
}
args, _ := os.ReadFile(root + "/args")
if !strings.Contains(string(args), "Alpine $(touch should-not-exist)\n") {
t.Fatalf("name was not passed literally: %s", args)
}
} else if !strings.Contains(string(calls), "systemctl restart proxmox-dashboard-agent.service") {
t.Fatalf("%s", calls)
}
info, _ := os.Stat(root + "/etc/proxmox-dashboard-agent.env")
if info.Mode().Perm() != 0600 {
t.Fatalf("credentials permissions: %v", info.Mode())
}
})
}
}

View File

@@ -60,6 +60,16 @@ func installNetworkDependency() error {
if os.Geteuid() != 0 { if os.Geteuid() != 0 {
return fmt.Errorf("для автоустановки нужны права root") return fmt.Errorf("для автоустановки нужны права root")
} }
if apk, err := exec.LookPath("apk"); err == nil {
out := runAgentCombinedCommand(10*time.Minute, apk, "add", "--no-cache", "conntrack-tools")
if strings.HasPrefix(out, "ERROR: ") {
return fmt.Errorf("%s", out)
}
if _, err := exec.LookPath("conntrack"); err != nil {
return fmt.Errorf("apk завершился, но conntrack не найден")
}
return nil
}
apt, err := exec.LookPath("apt-get") apt, err := exec.LookPath("apt-get")
if err != nil { if err != nil {
return fmt.Errorf("автоустановка поддерживается на Debian/Ubuntu с apt-get; установите conntrack вручную") return fmt.Errorf("автоустановка поддерживается на Debian/Ubuntu с apt-get; установите conntrack вручную")

View File

@@ -26,7 +26,7 @@ var npmTrafficPattern = regexp.MustCompile(`^\[([^]]+)\] .*? (\d{3}) - ([A-Z]+)
func parseNPMTraffic(line string) (TrafficEvent, bool) { func parseNPMTraffic(line string) (TrafficEvent, bool) {
m := npmTrafficPattern.FindStringSubmatch(line) m := npmTrafficPattern.FindStringSubmatch(line)
if m == nil { if m == nil {
return TrafficEvent{}, false return parseNPMPlusTraffic(line)
} }
ip := net.ParseIP(m[6]) ip := net.ParseIP(m[6])
if ip == nil { if ip == nil {
@@ -55,11 +55,23 @@ type trafficAgentState struct {
CrowdSeen map[string]int64 `json:"crowdSeen"` CrowdSeen map[string]int64 `json:"crowdSeen"`
} }
func (state *trafficAgentState) readLogs(dir string) (int, error) { func (state *trafficAgentState) readLogs(dirs ...string) (int, error) {
state.BacklogBytes = 0 state.BacklogBytes = 0
paths, err := filepath.Glob(filepath.Join(dir, "*_access.log*")) var paths []string
if err != nil { seenPaths := map[string]bool{}
return 0, err for _, dir := range dirs {
for _, pattern := range []string{"*_access.log*", "access.log*"} {
matches, err := filepath.Glob(filepath.Join(dir, pattern))
if err != nil {
return 0, err
}
for _, path := range matches {
if !seenPaths[path] {
paths = append(paths, path)
seenPaths[path] = true
}
}
}
} }
sort.Strings(paths) sort.Strings(paths)
files := 0 files := 0
@@ -85,7 +97,7 @@ func (state *trafficAgentState) readLogs(dir string) (int, error) {
f.Close() f.Close()
continue continue
} }
active := strings.HasSuffix(path, "_access.log") active := strings.HasSuffix(path, "_access.log") || filepath.Base(path) == "access.log"
if active { if active {
files++ files++
} }
@@ -229,21 +241,31 @@ func parseCrowdTraffic(data []byte) ([]TrafficEvent, error) {
// cscli uses the existing local machine credentials. No new bouncer key or // cscli uses the existing local machine credentials. No new bouncer key or
// public LAPI port is needed, and only the read-only alerts list is executed. // public LAPI port is needed, and only the read-only alerts list is executed.
func collectCrowdTraffic(ctx context.Context) ([]TrafficEvent, bool, error) { func collectCrowdTraffic(ctx context.Context, container ...string) ([]TrafficEvent, bool, error) {
ctx, cancel := context.WithTimeout(ctx, 6*time.Second) ctx, cancel := context.WithTimeout(ctx, 6*time.Second)
defer cancel() defer cancel()
cmd := exec.CommandContext(ctx, "cscli", "alerts", "list", "--since", "24h", "--limit", "50", "-o", "json") var data []byte
var output limitedTrafficOutput if len(container) > 0 && container[0] != "" {
cmd.Stdout = &output var err error
if err := cmd.Run(); err != nil { data, err = dockerCrowdOutput(ctx, container[0])
return nil, false, errors.New("cscli alerts list недоступен; проверьте права агента и локальную LAPI") if err != nil {
return nil, false, errors.New("CrowdSec в Docker недоступен; проверьте контейнер и Docker socket")
}
} else {
cmd := exec.CommandContext(ctx, "cscli", "alerts", "list", "--since", "24h", "--limit", "50", "-o", "json")
var output limitedTrafficOutput
cmd.Stdout = &output
if err := cmd.Run(); err != nil {
return nil, false, errors.New("cscli недоступен на хосте; контейнер CrowdSec не обнаружен")
}
data = output.Bytes()
} }
events, err := parseCrowdTraffic(output.Bytes()) events, err := parseCrowdTraffic(data)
if err != nil { if err != nil {
return nil, false, errors.New("Не удалось разобрать JSON CrowdSec") return nil, false, errors.New("Не удалось разобрать JSON CrowdSec")
} }
var raw []json.RawMessage var raw []json.RawMessage
json.Unmarshal(output.Bytes(), &raw) json.Unmarshal(data, &raw)
return events, len(raw) >= 50, nil return events, len(raw) >= 50, nil
} }
@@ -287,9 +309,6 @@ func runAgentTraffic(ctx context.Context, config agentConfig, stateDir string) {
if dir == "off" { if dir == "off" {
return return
} }
if dir == "" {
dir = "/opt/npm/data/logs"
}
statePath := filepath.Join(stateDir, "traffic-state.json") statePath := filepath.Join(stateDir, "traffic-state.json")
state := trafficAgentState{StartedAt: time.Now().Unix(), Cursors: map[string]trafficCursor{}, CrowdSeen: map[string]int64{}} state := trafficAgentState{StartedAt: time.Now().Unix(), Cursors: map[string]trafficCursor{}, CrowdSeen: map[string]int64{}}
@@ -310,6 +329,8 @@ func runAgentTraffic(ctx context.Context, config agentConfig, stateDir string) {
} }
client := &http.Client{Timeout: 10 * time.Second} client := &http.Client{Timeout: 10 * time.Second}
lastCrowd := time.Time{} lastCrowd := time.Time{}
lastDiscovery := time.Time{}
discovery := trafficDiscovery{}
crowdAt := int64(0) crowdAt := int64(0)
crowdError := "" crowdError := ""
crowdLimited := false crowdLimited := false
@@ -321,9 +342,21 @@ func runAgentTraffic(ctx context.Context, config agentConfig, stateDir string) {
return return
default: default:
} }
files, logErr := state.readLogs(dir) if time.Since(lastDiscovery) >= time.Minute {
discovery = discoverTraffic(ctx)
lastDiscovery = time.Now()
}
files, logErr := state.readLogs(discovery.Dirs...)
if logErr != nil {
logErr = fmt.Errorf("%s; проверьте access-логи и LOGROTATE=true у NPM+", logErr)
}
if len(state.Pending) < 400 && time.Since(lastCrowd) >= 15*time.Second { if len(state.Pending) < 400 && time.Since(lastCrowd) >= 15*time.Second {
events, limited, err := collectCrowdTraffic(ctx) events, limited, err := collectCrowdTraffic(ctx, discovery.CrowdContainer)
if discovery.Error != "" {
if _, localErr := exec.LookPath("cscli"); localErr != nil {
err = errors.New(discovery.Error)
}
}
crowdLimited = limited crowdLimited = limited
lastCrowd = time.Now() lastCrowd = time.Now()
if err != nil { if err != nil {

209
agent_traffic_discovery.go Normal file
View File

@@ -0,0 +1,209 @@
package main
import (
"bytes"
"context"
"encoding/binary"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"time"
)
type trafficDockerContainer struct {
ID string `json:"Id"`
Image string
Names []string
Mounts []struct{ Source, Destination string }
}
type trafficDiscovery struct {
Dirs []string
CrowdContainer string
Error string
}
func trafficDockerClient() *http.Client {
return &http.Client{Timeout: 8 * time.Second, Transport: &http.Transport{DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
return (&net.Dialer{}).DialContext(ctx, "unix", "/var/run/docker.sock")
}}}
}
func discoverTraffic(ctx context.Context) trafficDiscovery {
d := trafficDiscovery{Dirs: []string{"/opt/npm/data/logs", "/opt/npmplus/nginx/logs", "/opt/npmplus/nginx"}}
client := trafficDockerClient()
defer client.CloseIdleConnections()
req, _ := http.NewRequestWithContext(ctx, "GET", "http://docker/containers/json", nil)
res, err := client.Do(req)
if err == nil {
defer res.Body.Close()
var containers []trafficDockerContainer
if res.StatusCode == 200 && json.NewDecoder(io.LimitReader(res.Body, 2<<20)).Decode(&containers) == nil {
d = trafficSources(containers, d.Dirs)
} else {
d.Error = "Не удалось прочитать список Docker-контейнеров"
}
} else if _, e := os.Stat("/var/run/docker.sock"); e == nil {
d.Error = "Docker socket недоступен агенту"
}
if dir := os.Getenv("DASHBOARD_NPM_LOG_DIR"); dir != "" {
d.Dirs = []string{dir}
}
if container := os.Getenv("DASHBOARD_CROWDSEC_CONTAINER"); container != "" {
d.CrowdContainer = container
d.Error = ""
}
return d
}
func trafficSources(containers []trafficDockerContainer, defaults []string) trafficDiscovery {
d := trafficDiscovery{Dirs: append([]string{}, defaults...)}
var crowd []string
for _, c := range containers {
img := strings.ToLower(c.Image)
if strings.Contains(img, "crowdsecurity/crowdsec:") || strings.HasSuffix(img, "crowdsecurity/crowdsec") || strings.Contains(img, "crowdsecurity/crowdsec@") {
crowd = append(crowd, c.ID)
}
if !strings.Contains(img, "nginx-proxy-manager") && !strings.Contains(img, "npmplus") {
continue
}
for _, target := range []string{"/data/logs", "/data/nginx/logs", "/data/nginx"} {
// Use the most specific mount, including a nested bind or named volume.
best, source := "", ""
for _, m := range c.Mounts {
dst := filepath.Clean(m.Destination)
if filepath.IsAbs(m.Source) && (target == dst || strings.HasPrefix(target, dst+"/")) && len(dst) > len(best) {
best = dst
source = filepath.Join(m.Source, strings.TrimPrefix(target, dst))
}
}
if source != "" {
d.Dirs = append(d.Dirs, source)
}
}
}
if len(crowd) == 1 {
d.CrowdContainer = crowd[0]
} else if len(crowd) > 1 {
d.Error = "Найдено несколько CrowdSec: задайте DASHBOARD_CROWDSEC_CONTAINER"
}
unique := map[string]bool{}
dirs := []string{}
for _, dir := range d.Dirs {
dir = filepath.Clean(dir)
if !unique[dir] {
unique[dir] = true
dirs = append(dirs, dir)
}
}
d.Dirs = dirs
return d
}
// Docker exec uses a fixed read-only command; no shell, credential extraction or published LAPI.
func dockerCrowdOutput(ctx context.Context, container string) ([]byte, error) {
client := trafficDockerClient()
defer client.CloseIdleConnections()
return dockerCrowdWithClient(ctx, container, client)
}
func dockerCrowdWithClient(ctx context.Context, container string, client *http.Client) ([]byte, error) {
if !safeContainerName.MatchString(container) || strings.HasPrefix(container, "-") {
return nil, fmt.Errorf("некорректный контейнер CrowdSec")
}
call := func(path string, payload any) (*http.Response, error) {
raw, _ := json.Marshal(payload)
req, _ := http.NewRequestWithContext(ctx, "POST", "http://docker"+path, bytes.NewReader(raw))
req.Header.Set("Content-Type", "application/json")
return client.Do(req)
}
res, err := call("/containers/"+container+"/exec", map[string]any{"AttachStdout": true, "AttachStderr": true, "Tty": false, "Cmd": []string{"cscli", "alerts", "list", "--since", "24h", "--limit", "50", "-o", "json"}})
if err != nil {
return nil, err
}
var created struct{ ID string }
err = json.NewDecoder(io.LimitReader(res.Body, 4096)).Decode(&created)
res.Body.Close()
if err != nil || res.StatusCode != 201 || !safeContainerName.MatchString(created.ID) {
return nil, fmt.Errorf("Docker не создал cscli exec")
}
res, err = call("/exec/"+created.ID+"/start", map[string]any{"Detach": false, "Tty": false})
if err != nil {
return nil, err
}
defer res.Body.Close()
if res.StatusCode != 200 {
return nil, fmt.Errorf("Docker cscli exec: %s", res.Status)
}
output, err := readDockerTrafficOutput(res.Body)
if err != nil {
return nil, err
}
req, _ := http.NewRequestWithContext(ctx, "GET", "http://docker/exec/"+created.ID+"/json", nil)
check, err := client.Do(req)
if err != nil {
return nil, err
}
defer check.Body.Close()
var status struct {
Running bool
ExitCode int
}
if check.StatusCode != 200 || json.NewDecoder(io.LimitReader(check.Body, 4096)).Decode(&status) != nil || status.Running || status.ExitCode != 0 {
return nil, fmt.Errorf("cscli внутри Docker завершился с ошибкой")
}
return output, nil
}
func readDockerTrafficOutput(r io.Reader) ([]byte, error) {
var out bytes.Buffer
total := uint32(0)
for {
var header [8]byte
_, err := io.ReadFull(r, header[:])
if err == io.EOF {
return out.Bytes(), nil
}
if err != nil {
return nil, err
}
n := binary.BigEndian.Uint32(header[4:])
if n > 4<<20 || total+n > 4<<20 {
return nil, fmt.Errorf("Docker output too large")
}
total += n
if header[0] != 1 && header[0] != 2 {
return nil, fmt.Errorf("invalid Docker stream")
}
if header[0] == 1 {
_, err = io.CopyN(&out, r, int64(n))
} else {
_, err = io.CopyN(io.Discard, r, int64(n))
}
if err != nil {
return nil, err
}
}
}
var npmPlusTrafficPattern = regexp.MustCompile(`^\[([^]]+)\] (\S+) (\S+) [0-9.]+ "([A-Z]+) ([^" ]+) HTTP/[0-9.]+" (\d{3}) `)
func parseNPMPlusTraffic(line string) (TrafficEvent, bool) {
m := npmPlusTrafficPattern.FindStringSubmatch(line)
if m == nil {
return TrafficEvent{}, false
}
ip := net.ParseIP(m[3])
ts, err := time.Parse("02/Jan/2006:15:04:05 -0700", m[1])
if ip == nil || err != nil {
return TrafficEvent{}, false
}
host := m[2]
if h, _, err := net.SplitHostPort(host); err == nil {
host = h
}
status, _ := strconv.Atoi(m[6])
return TrafficEvent{Time: ts.Unix(), Kind: "request", IP: ip.String(), Domain: trafficText(host, 253), Method: m[4], Path: trafficPath(m[5]), Status: status}, true
}

View File

@@ -0,0 +1,134 @@
package main
import (
"bytes"
"context"
"encoding/binary"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func TestNPMPlusTraffic(t *testing.T) {
for _, ip := range []string{"8.8.8.8", "2001:4860:4860::8888"} {
line := `[14/Oct/2026:16:24:17 +0000] example.test:8443 ` + ip + ` 0.001 "GET /health?token=secret HTTP/3.0" 200 123 "-" "test" "-" "text/html" TLSv1.3 - - -`
e, ok := parseNPMTraffic(line)
if !ok || e.IP != ip || e.Domain != "example.test" || e.Path != "/health" || e.Status != 200 {
t.Fatalf("%+v %v", e, ok)
}
}
if _, ok := parseNPMTraffic(`[14/Oct/2026:16:24:17 +0000] 443 8.8.8.8 UDP - 200 10 20 0.001`); ok {
t.Fatal("stream counted as HTTP")
}
}
func TestDockerTrafficDiscovery(t *testing.T) {
var containers []trafficDockerContainer
json.Unmarshal([]byte(`[{"Id":"aaa","Image":"ghcr.io/zoeyvid/npmplus:latest","Mounts":[{"Source":"/var/lib/docker/volumes/npm/_data","Destination":"/data"},{"Source":"/srv/logs","Destination":"/data/nginx/logs"}]},{"Id":"bbb","Image":"crowdsecurity/crowdsec:latest"}]`), &containers)
d := trafficSources(containers, nil)
if d.CrowdContainer != "bbb" || d.Error != "" || strings.Join(d.Dirs, ",") != "/var/lib/docker/volumes/npm/_data/logs,/srv/logs,/var/lib/docker/volumes/npm/_data/nginx" {
t.Fatalf("%+v", d)
}
containers = append(containers, containers[1])
d = trafficSources(containers, nil)
if d.CrowdContainer != "" || d.Error == "" {
t.Fatal("ambiguous CrowdSec silently chosen")
}
}
func TestDockerTrafficDemux(t *testing.T) {
var b bytes.Buffer
for _, frame := range []struct {
stream byte
text string
}{{2, "diagnostic"}, {1, "["}, {1, "]"}} {
h := []byte{frame.stream, 0, 0, 0, 0, 0, 0, 0}
binary.BigEndian.PutUint32(h[4:], uint32(len(frame.text)))
b.Write(h)
b.WriteString(frame.text)
}
out, err := readDockerTrafficOutput(&b)
if err != nil || string(out) != "[]" {
t.Fatalf("%q %v", out, err)
}
if _, err := readDockerTrafficOutput(bytes.NewReader([]byte{1, 0, 0})); err == nil {
t.Fatal("truncation accepted")
}
if _, err := readDockerTrafficOutput(bytes.NewReader([]byte{1, 0, 0, 0, 255, 255, 255, 255})); err == nil {
t.Fatal("oversized frame accepted")
}
}
func TestTrafficMultipleDirectoriesAndNPMPlusRotation(t *testing.T) {
a, b := t.TempDir(), t.TempDir()
p := filepath.Join(b, "access.log")
os.WriteFile(filepath.Join(a, "proxy-host-1_access.log"), nil, 0600)
os.WriteFile(p, nil, 0600)
s := trafficAgentState{Cursors: map[string]trafficCursor{}}
if n, err := s.readLogs(a, b, a); err != nil || n != 2 {
t.Fatalf("files %d: %v", n, err)
}
line := "[" + time.Now().UTC().Format("02/Jan/2006:15:04:05 -0700") + `] example.test 8.8.8.8 0.1 "GET / HTTP/2.0" 200 123 "-"` + "\n"
os.WriteFile(p, []byte(line), 0600)
s.readLogs(a, b)
if len(s.Pending) != 1 {
t.Fatal("NPM+ event missing")
}
s.Pending = nil
os.Rename(p, p+".1")
os.WriteFile(p, []byte(line), 0600)
s.readLogs(a, b)
if len(s.Pending) != 1 {
t.Fatalf("rotation duplicated/lost events: %d", len(s.Pending))
}
}
type trafficRoundTrip func(*http.Request) (*http.Response, error)
func (f trafficRoundTrip) RoundTrip(r *http.Request) (*http.Response, error) { return f(r) }
func TestDockerCrowdExec(t *testing.T) {
for _, exitCode := range []int{0, 1} {
calls := 0
client := &http.Client{Transport: trafficRoundTrip(func(r *http.Request) (*http.Response, error) {
calls++
body := ""
status := 200
switch calls {
case 1:
if r.Method != "POST" || r.URL.Path != "/containers/crowdsec/exec" {
t.Fatalf("unexpected %s %s", r.Method, r.URL)
}
var payload struct {
Cmd []string
Tty bool
}
json.NewDecoder(r.Body).Decode(&payload)
if strings.Join(payload.Cmd, " ") != "cscli alerts list --since 24h --limit 50 -o json" || payload.Tty {
t.Fatalf("unexpected command %+v", payload)
}
status = 201
body = `{"Id":"exec123"}`
case 2:
if r.URL.Path != "/exec/exec123/start" {
t.Fatal(r.URL)
}
body = string([]byte{1, 0, 0, 0, 0, 0, 0, 2}) + "[]"
case 3:
if r.Method != "GET" || r.URL.Path != "/exec/exec123/json" {
t.Fatal(r.URL)
}
body = fmt.Sprintf(`{"Running":false,"ExitCode":%d}`, exitCode)
default:
t.Fatal("extra Docker request")
}
return &http.Response{StatusCode: status, Body: io.NopCloser(strings.NewReader(body)), Header: make(http.Header)}, nil
})}
out, err := dockerCrowdWithClient(context.Background(), "crowdsec", client)
if calls != 3 || (exitCode == 0 && (err != nil || string(out) != "[]")) || (exitCode != 0 && err == nil) {
t.Fatalf("calls=%d output=%s err=%v", calls, out, err)
}
}
}

View File

@@ -12,3 +12,9 @@ https://creativecommons.org/licenses/by/4.0/
https://db-ip.com/db/download/ip-to-country-lite https://db-ip.com/db/download/ip-to-country-lite
The database is unmodified. The traffic page displays the required DB-IP link. The database is unmodified. The traffic page displays the required DB-IP link.
Countries are approximate; plotted positions are country labels, not exact IP locations. Countries are approximate; plotted positions are country labels, not exact IP locations.
Optional DB-IP City Lite October 2026 is downloaded separately through Settings.
Source: https://download.db-ip.com/free/dbip-city-lite-2026-10.mmdb.gz
License: CC BY 4.0, https://creativecommons.org/licenses/by/4.0/
https://db-ip.com/db/download/ip-to-city-lite
The database is unmodified; city and region are approximate.

File diff suppressed because one or more lines are too long

View File

@@ -28,6 +28,31 @@ func runHistoryCollector(collect func() (dashboardMetrics, error), store *Store)
func (s *Store) AddEntityMetrics(metrics dashboardMetrics) { func (s *Store) AddEntityMetrics(metrics dashboardMetrics) {
ts := time.Now().Unix() ts := time.Now().Unix()
u := metrics.UPS
usable := func(v *float64) *float64 {
if !u.Available || !u.CommunicationOK {
return nil
}
return v
}
for _, item := range []struct {
kind, name string
value *float64
}{
{"ups-voltage", "Вход", u.InputVoltage}, {"ups-voltage", "Выход", u.OutputVoltage},
{"ups-battery", "Батарея", u.BatteryVoltage}, {"ups-runtime", "Автономность", u.RuntimeSeconds},
{"ups-frequency", "Частота", u.InputFrequency},
} {
s.AddEntityHistory(item.kind, item.name, ts, usable(item.value), nil)
}
var measured, estimated *float64
if u.PowerEstimated {
estimated = u.Power
} else {
measured = u.Power
}
s.AddEntityHistory("ups-power", "Измеренная", ts, usable(measured), nil)
s.AddEntityHistory("ups-power", "Расчётная ≈", ts, usable(estimated), nil)
for _, p := range metrics.ZFS.Pools { for _, p := range metrics.ZFS.Pools {
v := p.CapacityPercent v := p.CapacityPercent
s.AddEntityHistory("zfs", p.Name, ts, &v, nil) s.AddEntityHistory("zfs", p.Name, ts, &v, nil)
@@ -99,6 +124,10 @@ func historyPointFromMetrics(metrics dashboardMetrics) HistoryPoint {
NetworkTransmit: metrics.Network.TransmitPerSec, UPSCharge: metrics.UPS.ChargePercent, NetworkTransmit: metrics.Network.TransmitPerSec, UPSCharge: metrics.UPS.ChargePercent,
UPSLoad: metrics.UPS.LoadPercent, UPSLoad: metrics.UPS.LoadPercent,
} }
if !metrics.UPS.Available || !metrics.UPS.CommunicationOK {
point.UPSCharge = nil
point.UPSLoad = nil
}
if temperature := cpuTemperature(metrics.CPU.Temperatures); temperature != nil { if temperature := cpuTemperature(metrics.CPU.Temperatures); temperature != nil {
point.CPUTemperature = *temperature point.CPUTemperature = *temperature
} }

View File

@@ -10,11 +10,28 @@ if [ -z "${DASHBOARD_URL:-}" ] || [ -z "${ENROLL_TOKEN:-}" ]; then
exit 1 exit 1
fi fi
case "$(uname -m)" in x86_64|amd64) ;; *) echo "Поддерживается Linux amd64 (x86_64)." >&2; exit 1 ;; esac
if command -v systemctl >/dev/null 2>&1 && [ -d /run/systemd/system ]; then
agent_init=systemd
elif command -v rc-service >/dev/null 2>&1 && command -v supervise-daemon >/dev/null 2>&1; then
agent_init=openrc
else
echo "Нужен работающий systemd или OpenRC с supervise-daemon." >&2; exit 1
fi
if ! command -v curl >/dev/null 2>&1; then
if command -v apk >/dev/null 2>&1; then apk add --no-cache curl ca-certificates; else echo "Установите curl и ca-certificates." >&2; exit 1; fi
fi
GITEA_URL=${GITEA_URL:-https://git.myown.center} GITEA_URL=${GITEA_URL:-https://git.myown.center}
GITEA_REPOSITORY=${GITEA_REPOSITORY:-maxim/ProxmoxDash} GITEA_REPOSITORY=${GITEA_REPOSITORY:-maxim/ProxmoxDash}
VMID=${VMID:-0} VMID=${VMID:-0}
AGENT_NAME=${AGENT_NAME:-$(hostname)} AGENT_NAME=${AGENT_NAME:-$(hostname)}
case "$DASHBOARD_URL$ENROLL_TOKEN$AGENT_NAME$VMID" in
*'
'*) echo "Параметры установки не должны содержать переносы строк." >&2; exit 1 ;;
esac
release_json=$(curl -fsSL "$GITEA_URL/api/v1/repos/$GITEA_REPOSITORY/releases/latest") release_json=$(curl -fsSL "$GITEA_URL/api/v1/repos/$GITEA_REPOSITORY/releases/latest")
download_url=$(printf '%s' "$release_json" | sed -n 's/.*"browser_download_url":"\([^"]*\/proxmox-dashboard-linux-amd64\)".*/\1/p' | head -n 1) download_url=$(printf '%s' "$release_json" | sed -n 's/.*"browser_download_url":"\([^"]*\/proxmox-dashboard-linux-amd64\)".*/\1/p' | head -n 1)
if [ -z "$download_url" ]; then if [ -z "$download_url" ]; then
@@ -24,8 +41,12 @@ fi
install -d -m 0700 /var/lib/proxmox-dashboard-agent install -d -m 0700 /var/lib/proxmox-dashboard-agent
temporary_binary=$(mktemp /tmp/proxmox-dashboard-agent.XXXXXX) temporary_binary=$(mktemp /tmp/proxmox-dashboard-agent.XXXXXX)
trap 'rm -f "$temporary_binary"' EXIT trap 'rm -f "$temporary_binary" "$temporary_binary.sha256"' EXIT
curl -fsSL "$download_url" -o "$temporary_binary" curl -fsSL "$download_url" -o "$temporary_binary"
curl -fsSL "$download_url.sha256" -o "$temporary_binary.sha256"
expected_hash=$(awk 'NR==1 {print $1}' "$temporary_binary.sha256")
actual_hash=$(sha256sum "$temporary_binary" | awk '{print $1}')
if [ "${#expected_hash}" -ne 64 ] || [ "$expected_hash" != "$actual_hash" ]; then echo "SHA-256 загрузки не совпал." >&2; exit 1; fi
install -m 0755 "$temporary_binary" /var/lib/proxmox-dashboard-agent/proxmox-dashboard-agent install -m 0755 "$temporary_binary" /var/lib/proxmox-dashboard-agent/proxmox-dashboard-agent
umask 077 umask 077
@@ -36,6 +57,7 @@ umask 077
printf 'VMID=%s\n' "$VMID" printf 'VMID=%s\n' "$VMID"
} > /etc/proxmox-dashboard-agent.env } > /etc/proxmox-dashboard-agent.env
if [ "$agent_init" = systemd ]; then
cat > /etc/systemd/system/proxmox-dashboard-agent.service <<'UNIT' cat > /etc/systemd/system/proxmox-dashboard-agent.service <<'UNIT'
[Unit] [Unit]
Description=Proxmox Dashboard read-only guest agent Description=Proxmox Dashboard read-only guest agent
@@ -61,5 +83,38 @@ UNIT
systemctl daemon-reload systemctl daemon-reload
systemctl enable proxmox-dashboard-agent.service systemctl enable proxmox-dashboard-agent.service
systemctl restart proxmox-dashboard-agent.service systemctl restart proxmox-dashboard-agent.service
else
mkdir -p /usr/local/sbin /etc/init.d
cat > /usr/local/sbin/proxmox-dashboard-agent-run <<'RUN'
#!/bin/sh
set -eu
# Parse literal key=value records, never evaluate configuration as shell code.
while IFS='=' read -r key value; do
case "$key" in DASHBOARD_URL|ENROLL_TOKEN|AGENT_NAME|VMID|DASHBOARD_NPM_LOG_DIR|DASHBOARD_CROWDSEC_CONTAINER) export "$key=$value" ;; esac
done < /etc/proxmox-dashboard-agent.env
exec /var/lib/proxmox-dashboard-agent/proxmox-dashboard-agent --agent --agent-server "$DASHBOARD_URL" --agent-enroll-token "$ENROLL_TOKEN" --agent-name "$AGENT_NAME" --agent-vmid "$VMID"
RUN
chmod 0755 /usr/local/sbin/proxmox-dashboard-agent-run
cat > /etc/init.d/proxmox-dashboard-agent <<'OPENRC'
#!/sbin/openrc-run
description="Proxmox Dashboard guest agent"
command="/usr/local/sbin/proxmox-dashboard-agent-run"
supervisor="supervise-daemon"
respawn_delay=5
respawn_max=0
output_log="/var/log/proxmox-dashboard-agent.log"
error_log="/var/log/proxmox-dashboard-agent.log"
depend() {
need net
after docker
}
start_pre() {
checkpath --file --mode 0600 /var/log/proxmox-dashboard-agent.log
}
OPENRC
chmod 0755 /etc/init.d/proxmox-dashboard-agent
rc-update add proxmox-dashboard-agent default
rc-service proxmox-dashboard-agent restart
fi
rm -f /usr/local/bin/proxmox-dashboard-agent rm -f /usr/local/bin/proxmox-dashboard-agent
echo "Агент установлен. Через несколько секунд он появится в Dashboard." echo "Агент установлен. Через несколько секунд он появится в Dashboard."

View File

@@ -6,11 +6,16 @@ if [ "$(id -u)" -ne 0 ]; then
exit 1 exit 1
fi fi
if command -v rc-service >/dev/null 2>&1 && [ -f /etc/init.d/proxmox-dashboard-agent ]; then
rc-service proxmox-dashboard-agent stop || true
rc-update del proxmox-dashboard-agent default || true
fi
systemctl disable --now proxmox-dashboard-agent.service 2>/dev/null || true systemctl disable --now proxmox-dashboard-agent.service 2>/dev/null || true
rm -f /etc/init.d/proxmox-dashboard-agent /usr/local/sbin/proxmox-dashboard-agent-run
rm -f /etc/systemd/system/proxmox-dashboard-agent.service rm -f /etc/systemd/system/proxmox-dashboard-agent.service
rm -f /etc/proxmox-dashboard-agent.env rm -f /etc/proxmox-dashboard-agent.env
rm -f /usr/local/bin/proxmox-dashboard-agent rm -f /usr/local/bin/proxmox-dashboard-agent
rm -rf /var/lib/proxmox-dashboard-agent rm -rf /var/lib/proxmox-dashboard-agent
systemctl daemon-reload systemctl daemon-reload 2>/dev/null || true
systemctl reset-failed proxmox-dashboard-agent.service 2>/dev/null || true systemctl reset-failed proxmox-dashboard-agent.service 2>/dev/null || true
echo "Агент и его локальные учётные данные удалены." echo "Агент и его локальные учётные данные удалены."

View File

@@ -12,19 +12,22 @@ import (
) )
type TrafficEvent struct { type TrafficEvent struct {
Monitoring bool `json:"monitoring"` Location *TrafficCityLocation `json:"location,omitempty"`
ID string `json:"id"` City string `json:"city,omitempty"`
Sequence int64 `json:"sequence,omitempty"` Region string `json:"region,omitempty"`
Time int64 `json:"time"` Monitoring bool `json:"monitoring"`
Kind string `json:"kind"` ID string `json:"id"`
IP string `json:"ip"` Sequence int64 `json:"sequence,omitempty"`
Domain string `json:"domain,omitempty"` Time int64 `json:"time"`
Method string `json:"method,omitempty"` Kind string `json:"kind"`
Path string `json:"path,omitempty"` IP string `json:"ip"`
Status int `json:"status,omitempty"` Domain string `json:"domain,omitempty"`
Reason string `json:"reason,omitempty"` Method string `json:"method,omitempty"`
Country string `json:"country,omitempty"` Path string `json:"path,omitempty"`
Agent string `json:"agent,omitempty"` Status int `json:"status,omitempty"`
Reason string `json:"reason,omitempty"`
Country string `json:"country,omitempty"`
Agent string `json:"agent,omitempty"`
} }
type TrafficBatch struct { type TrafficBatch struct {
@@ -51,7 +54,21 @@ type TrafficCount struct {
Count int64 `json:"count"` Count int64 `json:"count"`
} }
type TrafficCountryDetail struct {
Country string `json:"country"`
Requests int64 `json:"requests"`
UniqueIPs int64 `json:"uniqueIps"`
Alerts int64 `json:"alerts"`
Decisions int64 `json:"decisions"`
LastSeen int64 `json:"lastSeen"`
}
type TrafficView struct { type TrafficView struct {
Cities []TrafficMapCity `json:"cities"`
CitiesLimited bool `json:"citiesLimited"`
CityReady bool `json:"cityReady"`
MaxSequence int64 `json:"maxSequence"`
CountryDetails []TrafficCountryDetail `json:"countryDetails"`
Network *NetworkMapView `json:"network,omitempty"` Network *NetworkMapView `json:"network,omitempty"`
Events []TrafficEvent `json:"events"` Events []TrafficEvent `json:"events"`
Sources []TrafficSource `json:"sources"` Sources []TrafficSource `json:"sources"`
@@ -160,8 +177,11 @@ func (s *Store) TrafficView(domain, kind string, external bool, hideMonitoring .
network := makeNetworkMap(snapshot, external) network := makeNetworkMap(snapshot, external)
v.Network = &network v.Network = &network
} }
where := ` WHERE ts>=? AND agent=?` if err := s.db.QueryRow(`SELECT COALESCE(MAX(seq),0) FROM traffic_events WHERE agent=?`, v.Settings.AgentID).Scan(&v.MaxSequence); err != nil {
args := []any{now - 86400, v.Settings.AgentID} return v, err
}
where := ` WHERE ts>=? AND agent=? AND ts<=? AND seq<=?`
args := []any{now - 86400, v.Settings.AgentID, now, v.MaxSequence}
if hide { if hide {
clause, params := monitoringSQL(v.Settings.MonitoringRules) clause, params := monitoringSQL(v.Settings.MonitoringRules)
where += clause where += clause
@@ -193,6 +213,11 @@ func (s *Store) TrafficView(domain, kind string, external bool, hideMonitoring .
return v, err return v, err
} }
e.Monitoring = isMonitoringRequest(e, v.Settings.MonitoringRules) e.Monitoring = isMonitoringRequest(e, v.Settings.MonitoringRules)
g := lookupTrafficCityLocation(e.IP)
e.City, e.Region = g.City, g.Region
if g.Located {
e.Location = &g
}
v.Events = append(v.Events, e) v.Events = append(v.Events, e)
} }
err = rows.Err() err = rows.Err()
@@ -222,6 +247,32 @@ func (s *Store) TrafficView(domain, kind string, external bool, hideMonitoring .
return v, e return v, e
} }
} }
detailRows, detailErr := s.db.Query(`SELECT country,SUM(kind='request'),COUNT(DISTINCT CASE WHEN kind='request' THEN ip END),SUM(kind='alert'),SUM(kind='decision'),MAX(ts) FROM traffic_events`+where+` GROUP BY country`, args...)
if detailErr != nil {
return v, detailErr
}
v.CountryDetails = []TrafficCountryDetail{}
for detailRows.Next() {
var d TrafficCountryDetail
if err = detailRows.Scan(&d.Country, &d.Requests, &d.UniqueIPs, &d.Alerts, &d.Decisions, &d.LastSeen); err != nil {
detailRows.Close()
return v, err
}
v.CountryDetails = append(v.CountryDetails, d)
}
detailErr = detailRows.Err()
detailRows.Close()
if detailErr != nil {
return v, detailErr
}
v.CityReady = trafficCityReady()
if v.CityReady {
var cityErr error
v.Cities, v.CitiesLimited, cityErr = s.trafficMapCities(where, args)
if cityErr != nil {
return v, cityErr
}
}
r, err := s.db.Query(`SELECT CAST((ts/60)*60 AS TEXT),COUNT(*) FROM traffic_events`+where+` AND kind='request' AND ts>=? GROUP BY ts/60 ORDER BY ts/60`, append(args, now-3600)...) r, err := s.db.Query(`SELECT CAST((ts/60)*60 AS TEXT),COUNT(*) FROM traffic_events`+where+` AND kind='request' AND ts>=? GROUP BY ts/60 ORDER BY ts/60`, append(args, now-3600)...)
if err != nil { if err != nil {
return v, err return v, err
@@ -263,6 +314,8 @@ func (s *Store) TrafficView(domain, kind string, external bool, hideMonitoring .
func trafficRoutes(mux *http.ServeMux, s *Store) { func trafficRoutes(mux *http.ServeMux, s *Store) {
trafficSettingsRoute(mux, s) trafficSettingsRoute(mux, s)
s.trafficHistoryRoute(mux)
trafficCityRoutes(mux)
networkRoutes(mux, s) networkRoutes(mux, s)
mux.HandleFunc("/traffic", func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("/traffic", func(w http.ResponseWriter, r *http.Request) {
if !s.TrafficSettings().Enabled { if !s.TrafficSettings().Enabled {

216
traffic_city.go Normal file
View File

@@ -0,0 +1,216 @@
package main
import (
"compress/gzip"
"crypto/sha1"
"encoding/hex"
"fmt"
"io"
"math"
"net"
"net/http"
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/oschwald/maxminddb-golang"
)
const cityDownloadURL = "https://download.db-ip.com/free/dbip-city-lite-2026-10.mmdb.gz"
const cityDatabaseSHA1 = "df17bd24390108ed1d24dcb7718dc60f6b9abd4f"
var trafficCity struct {
sync.Mutex
once sync.Once
db *maxminddb.Reader
running bool
message string
cache map[string]TrafficCityLocation
}
func trafficCityPath() string {
db := os.Getenv("DASHBOARD_DB_PATH")
if db == "" {
db = defaultDatabasePath
}
return filepath.Join(filepath.Dir(db), "dbip-city.mmdb")
}
func trafficCityReady() bool {
trafficCity.once.Do(func() {
trafficCity.Lock()
defer trafficCity.Unlock()
trafficCity.db, _ = maxminddb.Open(trafficCityPath())
if trafficCity.db != nil && !strings.Contains(strings.ToLower(trafficCity.db.Metadata.DatabaseType), "city") {
trafficCity.db.Close()
trafficCity.db = nil
}
})
trafficCity.Lock()
defer trafficCity.Unlock()
return trafficCity.db != nil
}
type TrafficCityLocation struct {
City string `json:"city"`
Region string `json:"region,omitempty"`
Latitude float64 `json:"latitude"`
Longitude float64 `json:"longitude"`
Located bool `json:"located"`
}
func lookupTrafficCity(text string) (string, string) {
g := lookupTrafficCityLocation(text)
return g.City, g.Region
}
func lookupTrafficCityLocation(text string) TrafficCityLocation {
ip := net.ParseIP(text)
if ip == nil || ip.IsPrivate() || ip.IsLoopback() || ip.IsLinkLocalUnicast() || !ip.IsGlobalUnicast() || !trafficCityReady() {
return TrafficCityLocation{}
}
trafficCity.Lock()
defer trafficCity.Unlock()
if g, ok := trafficCity.cache[text]; ok {
return g
}
var r struct {
Location struct {
Latitude *float64 `maxminddb:"latitude"`
Longitude *float64 `maxminddb:"longitude"`
} `maxminddb:"location"`
City struct {
Names map[string]string `maxminddb:"names"`
} `maxminddb:"city"`
Subdivisions []struct {
Names map[string]string `maxminddb:"names"`
} `maxminddb:"subdivisions"`
}
if trafficCity.db.Lookup(ip, &r) != nil {
return TrafficCityLocation{}
}
region := ""
if len(r.Subdivisions) > 0 {
region = cityName(r.Subdivisions[0].Names)
}
g := TrafficCityLocation{City: cityName(r.City.Names), Region: region}
if g.City != "" && r.Location.Latitude != nil && r.Location.Longitude != nil {
g.Latitude, g.Longitude = *r.Location.Latitude, *r.Location.Longitude
g.Located = !math.IsNaN(g.Latitude) && !math.IsNaN(g.Longitude) && math.Abs(g.Latitude) <= 90 && math.Abs(g.Longitude) <= 180
}
if trafficCity.cache == nil {
trafficCity.cache = map[string]TrafficCityLocation{}
}
if len(trafficCity.cache) < 32768 {
trafficCity.cache[text] = g
}
return g
}
func trafficCityRoutes(mux *http.ServeMux) {
mux.HandleFunc("/api/settings/traffic/city-database", func(w http.ResponseWriter, r *http.Request) {
if r.Method != "GET" && r.Method != "POST" {
w.WriteHeader(405)
return
}
trafficCityReady()
trafficCity.Lock()
if r.Method == "POST" && !trafficCity.running && trafficCity.db == nil {
trafficCity.running = true
trafficCity.message = "Загрузка базы городов (около 58 МБ)…"
go func() {
err := installTrafficCity()
trafficCity.Lock()
defer trafficCity.Unlock()
trafficCity.running = false
if err != nil {
trafficCity.message = "Не удалось установить базу: " + err.Error()
} else {
trafficCity.message = "База городов DB-IP City Lite 2026-10 установлена"
}
}()
}
status := map[string]any{"ready": trafficCity.db != nil, "running": trafficCity.running, "message": trafficCity.message, "date": "2026-10"}
trafficCity.Unlock()
writeJSON(w, status, nil)
})
}
func cityMirrorURL() string {
base := strings.TrimRight(os.Getenv("GITEA_URL"), "/")
if base == "" {
base = "https://git.myown.center"
}
return base + "/maxim/ProxmoxDash/releases/download/v0.25.1/dbip-city-lite-2026-10.mmdb.gz"
}
func installTrafficCity() error {
if err := installTrafficCityFrom(cityMirrorURL(), cityDatabaseSHA1); err == nil {
return nil
}
return installTrafficCityFrom(cityDownloadURL, cityDatabaseSHA1)
}
func installTrafficCityFrom(source, expectedHash string) error {
transport := http.DefaultTransport.(*http.Transport).Clone()
transport.ResponseHeaderTimeout = 20 * time.Second
transport.DialContext = (&net.Dialer{Timeout: 10 * time.Second, KeepAlive: 30 * time.Second}).DialContext
defer transport.CloseIdleConnections()
client := &http.Client{Timeout: 15 * time.Minute, Transport: transport}
response, err := client.Get(source)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode != 200 {
return fmt.Errorf("HTTP %d", response.StatusCode)
}
reader, err := gzip.NewReader(io.LimitReader(response.Body, 70<<20))
if err != nil {
return err
}
defer reader.Close()
path := trafficCityPath()
if err = os.MkdirAll(filepath.Dir(path), 0750); err != nil {
return err
}
file, err := os.CreateTemp(filepath.Dir(path), ".city-*.mmdb")
if err != nil {
return err
}
defer os.Remove(file.Name())
defer file.Close()
hash := sha1.New()
n, err := io.Copy(io.MultiWriter(file, hash), io.LimitReader(reader, (160<<20)+1))
if err != nil {
return err
}
if n > 160<<20 {
return fmt.Errorf("база превышает лимит размера")
}
if hex.EncodeToString(hash.Sum(nil)) != expectedHash {
return fmt.Errorf("контрольная сумма базы не совпала")
}
if err = file.Sync(); err != nil {
return err
}
if err = file.Close(); err != nil {
return err
}
db, err := maxminddb.Open(file.Name())
if err != nil {
return err
}
if !strings.Contains(strings.ToLower(db.Metadata.DatabaseType), "city") {
db.Close()
return fmt.Errorf("неверный тип GeoIP-базы")
}
if err = os.Rename(file.Name(), path); err != nil {
db.Close()
return err
}
trafficCity.Lock()
defer trafficCity.Unlock()
if trafficCity.db != nil {
trafficCity.db.Close()
}
trafficCity.db = db
trafficCity.cache = nil
return nil
}

82
traffic_city_test.go Normal file
View File

@@ -0,0 +1,82 @@
package main
import (
"compress/gzip"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"time"
)
func TestCityInstallRejectsBadDownload(t *testing.T) {
dir := t.TempDir()
t.Setenv("DASHBOARD_DB_PATH", filepath.Join(dir, "dashboard.db"))
existing := []byte("existing database")
if err := os.WriteFile(trafficCityPath(), existing, 0600); err != nil {
t.Fatal(err)
}
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
g := gzip.NewWriter(w)
g.Write([]byte("corrupt database"))
g.Close()
}))
defer server.Close()
if err := installTrafficCityFrom(server.URL, cityDatabaseSHA1); err == nil {
t.Fatal("bad checksum accepted")
}
got, err := os.ReadFile(trafficCityPath())
if err != nil || string(got) != string(existing) {
t.Fatal("existing database replaced on failure")
}
files, _ := filepath.Glob(filepath.Join(dir, ".city-*"))
if len(files) != 0 {
t.Fatal("partial file left behind")
}
}
func TestCityDatabaseDownloaded(t *testing.T) {
source := os.Getenv("DASHBOARD_CITY_TEST_GZIP")
if source == "" {
t.Skip("optional official database integration test")
}
t.Setenv("DASHBOARD_DB_PATH", filepath.Join(t.TempDir(), "dashboard.db"))
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { http.ServeFile(w, r, source) }))
defer server.Close()
trafficCityReady()
if err := installTrafficCityFrom(server.URL, cityDatabaseSHA1); err != nil {
t.Fatal(err)
}
g := lookupTrafficCityLocation("20.194.96.114")
if !g.Located || g.Latitude < 30 || g.Latitude > 40 || g.Longitude < 120 || g.Longitude > 130 {
t.Fatalf("invalid Seoul coordinates: %+v", g)
}
s, a := trafficFixture(t)
s.SaveTrafficSettings(TrafficSettings{Enabled: true, AgentID: a.ID})
now := time.Now().Unix()
if err := s.SaveTraffic(a.ID, TrafficBatch{Events: []TrafficEvent{
{ID: "1", Time: now, Kind: "request", IP: "20.194.96.114", Domain: "a.test"},
{ID: "2", Time: now, Kind: "request", IP: "20.194.96.114", Domain: "a.test"},
{ID: "3", Time: now, Kind: "alert", IP: "20.194.96.114", Domain: "b.test"},
{ID: "4", Time: now, Kind: "request", IP: "192.168.0.10"},
}}); err != nil {
t.Fatal(err)
}
view, err := s.TrafficView("", "", false)
if err != nil || len(view.Cities) != 1 {
t.Fatalf("cities %+v err %v", view.Cities, err)
}
c := view.Cities[0]
if c.Requests != 2 || c.UniqueIPs != 1 || c.Alerts != 1 || c.City != g.City {
t.Fatalf("bad aggregation %+v", c)
}
filtered, err := s.TrafficView("b.test", "alert", false)
if err != nil || len(filtered.Cities) != 1 || filtered.Cities[0].Requests != 0 || filtered.Cities[0].Alerts != 1 {
t.Fatalf("city filter %+v %v", filtered.Cities, err)
}
city, region := lookupTrafficCity("20.194.96.114")
t.Logf("city=%q region=%q", city, region)
if city == "" {
t.Fatal("public IP city not decoded")
}
}

44
traffic_country_test.go Normal file
View File

@@ -0,0 +1,44 @@
package main
import (
"fmt"
"testing"
"time"
)
func TestTrafficCountryDetailsUseHistoryAndFilters(t *testing.T) {
s, a := trafficFixture(t)
if _, err := s.SaveTrafficSettings(TrafficSettings{Enabled: true, AgentID: a.ID, MonitoringRules: []MonitoringRule{{IP: "8.8.8.8"}}}); err != nil {
t.Fatal(err)
}
events := []TrafficEvent{}
for i := 0; i < 120; i++ {
events = append(events, TrafficEvent{ID: fmt.Sprint("http-", i), Time: time.Now().Unix(), Kind: "request", IP: "8.8.8.8", Domain: "example.test", Path: "/", Status: 200})
}
events = append(events, TrafficEvent{ID: "alert", Time: time.Now().Unix(), Kind: "alert", IP: "8.8.8.8", Domain: "example.test", Reason: "test"})
if err := s.SaveTraffic(a.ID, TrafficBatch{Events: events}); err != nil {
t.Fatal(err)
}
v, err := s.TrafficView("", "", false)
if err != nil {
t.Fatal(err)
}
if len(v.Events) != 100 || len(v.CountryDetails) != 1 {
t.Fatalf("events=%d countries=%+v", len(v.Events), v.CountryDetails)
}
d := v.CountryDetails[0]
if d.Requests != 120 || d.UniqueIPs != 1 || d.Alerts != 1 || d.LastSeen == 0 {
t.Fatalf("%+v", d)
}
v, err = s.TrafficView("example.test", "", false, true)
if err != nil {
t.Fatal(err)
}
if len(v.CountryDetails) != 1 || v.CountryDetails[0].Requests != 0 || v.CountryDetails[0].Alerts != 1 {
t.Fatalf("filter: %+v", v.CountryDetails)
}
v, err = s.TrafficView("other.test", "", false)
if err != nil || len(v.CountryDetails) != 0 {
t.Fatalf("domain: %+v %v", v.CountryDetails, err)
}
}

144
traffic_history.go Normal file
View File

@@ -0,0 +1,144 @@
package main
import (
"net/http"
"strconv"
"strings"
"time"
)
type TrafficIPDetail struct {
IP string `json:"ip"`
Country string `json:"country"`
Requests int64 `json:"requests"`
LastSeen int64 `json:"lastSeen"`
City string `json:"city,omitempty"`
Region string `json:"region,omitempty"`
}
type TrafficHistory struct {
Events []TrafficEvent `json:"events"`
IPs []TrafficIPDetail `json:"ips"`
Total int64 `json:"total"`
Page int `json:"page"`
PageSize int `json:"pageSize"`
Snapshot int64 `json:"snapshot"`
MaxSequence int64 `json:"maxSequence"`
CityReady bool `json:"cityReady"`
}
func (s *Store) trafficHistoryRoute(mux *http.ServeMux) {
mux.HandleFunc("/api/traffic/history", func(w http.ResponseWriter, r *http.Request) {
if r.Method != "GET" {
w.WriteHeader(405)
return
}
if !s.TrafficSettings().Enabled {
http.Error(w, "Карта отключена", 404)
return
}
q := r.URL.Query()
page, _ := strconv.Atoi(q.Get("page"))
snapshot, _ := strconv.ParseInt(q.Get("snapshot"), 10, 64)
maxSeq, _ := strconv.ParseInt(q.Get("maxSequence"), 10, 64)
if q.Get("maxSequence") == "0" {
maxSeq = -1
}
if page < 0 || page > 10000 {
http.Error(w, "Invalid page", 400)
return
}
kind := q.Get("kind")
metricType := q.Get("type")
if metricType != "" && metricType != "request" && metricType != "alert" && metricType != "decision" {
http.Error(w, "Invalid type", 400)
return
}
if kind != "" && kind != "request" && kind != "alert" && kind != "decision" {
http.Error(w, "Invalid kind", 400)
return
}
result, err := s.TrafficHistory(q.Get("domain"), kind, q.Get("country"), q.Get("ip"), q.Get("type"), q.Get("external") == "1", q.Get("hideMonitoring") == "1", q.Get("group") == "ip", page, snapshot, maxSeq)
writeJSON(w, result, err)
})
}
func (s *Store) TrafficHistory(domain, kind, country, ip, metricType string, external, hide, group bool, page int, snapshot, maxSeq int64) (TrafficHistory, error) {
now := time.Now().Unix()
if snapshot <= 0 || snapshot > now {
snapshot = now
}
if snapshot < now-86400 {
snapshot = now
}
settings := s.TrafficSettings()
v := TrafficHistory{Events: []TrafficEvent{}, IPs: []TrafficIPDetail{}, Page: page, PageSize: 50, Snapshot: snapshot, CityReady: trafficCityReady()}
if maxSeq == 0 {
if err := s.db.QueryRow(`SELECT COALESCE(MAX(seq),0) FROM traffic_events WHERE agent=?`, settings.AgentID).Scan(&maxSeq); err != nil {
return v, err
}
}
v.MaxSequence = maxSeq
where := ` WHERE agent=? AND ts>=? AND ts<=? AND seq<=?`
args := []any{settings.AgentID, snapshot - 86400, snapshot, maxSeq}
if hide {
clause, values := monitoringSQL(settings.MonitoringRules)
where += clause
args = append(args, values...)
}
if external {
where += ` AND country!='LAN'`
}
for _, filter := range []struct{ column, value string }{{"domain", domain}, {"kind", kind}, {"kind", metricType}, {"country", country}, {"ip", ip}} {
if filter.value != "" {
where += " AND " + filter.column + "=?"
args = append(args, filter.value)
}
}
count := "COUNT(*)"
if group {
where += ` AND kind='request'`
count = "COUNT(DISTINCT ip)"
}
if err := s.db.QueryRow("SELECT "+count+" FROM traffic_events"+where, args...).Scan(&v.Total); err != nil {
return v, err
}
paged := append(append([]any{}, args...), v.PageSize, page*v.PageSize)
if group {
rows, err := s.db.Query(`SELECT ip,MIN(country),COUNT(*),MAX(ts) FROM traffic_events`+where+` GROUP BY ip ORDER BY COUNT(*) DESC,ip LIMIT ? OFFSET ?`, paged...)
if err != nil {
return v, err
}
defer rows.Close()
for rows.Next() {
var item TrafficIPDetail
if err = rows.Scan(&item.IP, &item.Country, &item.Requests, &item.LastSeen); err != nil {
return v, err
}
item.City, item.Region = lookupTrafficCity(item.IP)
v.IPs = append(v.IPs, item)
}
return v, rows.Err()
}
rows, err := s.db.Query(`SELECT seq,event_id,ts,kind,ip,domain,method,path,status,reason,country,agent FROM traffic_events`+where+` ORDER BY ts DESC,seq DESC LIMIT ? OFFSET ?`, paged...)
if err != nil {
return v, err
}
defer rows.Close()
for rows.Next() {
var e TrafficEvent
if err = rows.Scan(&e.Sequence, &e.ID, &e.Time, &e.Kind, &e.IP, &e.Domain, &e.Method, &e.Path, &e.Status, &e.Reason, &e.Country, &e.Agent); err != nil {
return v, err
}
e.Monitoring = isMonitoringRequest(e, settings.MonitoringRules)
e.City, e.Region = lookupTrafficCity(e.IP)
v.Events = append(v.Events, e)
}
return v, rows.Err()
}
func cityName(names map[string]string) string {
for _, lang := range []string{"ru", "en"} {
if s := strings.TrimSpace(names[lang]); s != "" {
return s
}
}
return ""
}

109
traffic_history_test.go Normal file
View File

@@ -0,0 +1,109 @@
package main
import (
"fmt"
"testing"
"time"
)
func TestTrafficHistoryFullPaginationAndSnapshot(t *testing.T) {
s, a := trafficFixture(t)
if _, err := s.SaveTrafficSettings(TrafficSettings{Enabled: true, AgentID: a.ID}); err != nil {
t.Fatal(err)
}
now := time.Now().Unix()
batch := TrafficBatch{}
for i := 0; i < 137; i++ {
batch.Events = append(batch.Events, TrafficEvent{ID: fmt.Sprint(i), Time: now - 60, Kind: "request", IP: "8.8.8.8", Domain: "example.test"})
}
batch.Events = append(batch.Events, TrafficEvent{ID: "old-alert", Time: now - 3600, Kind: "alert", IP: "1.1.1.1", Domain: "other.test"})
if err := s.SaveTraffic(a.ID, batch); err != nil {
t.Fatal(err)
}
first, err := s.TrafficHistory("", "request", "", "", "", false, false, false, 0, now, 0)
if err != nil || first.Total != 137 || len(first.Events) != 50 {
t.Fatalf("first: %+v %v", first, err)
}
if err := s.SaveTraffic(a.ID, TrafficBatch{Events: []TrafficEvent{{ID: "new", Time: now - 60, Kind: "request", IP: "8.8.8.8"}}}); err != nil {
t.Fatal(err)
}
seen := map[int64]bool{}
for page := 0; page < 3; page++ {
h, err := s.TrafficHistory("", "request", "", "", "", false, false, false, page, first.Snapshot, first.MaxSequence)
if err != nil || h.Total != 137 {
t.Fatalf("page %d: %v total %d", page, err, h.Total)
}
for _, e := range h.Events {
if seen[e.Sequence] {
t.Fatal("duplicate page event")
}
seen[e.Sequence] = true
}
}
if len(seen) != 137 {
t.Fatalf("missing history: %d", len(seen))
}
h, err := s.TrafficHistory("other.test", "alert", "", "", "", false, false, false, 0, now, 0)
if err != nil || h.Total != 1 || h.Events[0].ID != "old-alert" {
t.Fatalf("old alert missing: %+v %v", h, err)
}
h, err = s.TrafficHistory("", "alert", "", "", "request", false, false, false, 0, now, 0)
if err != nil || h.Total != 0 {
t.Fatal("metric must respect global kind filter")
}
h, err = s.TrafficHistory("example.test", "", "", "", "", false, false, true, 0, now, 0)
if err != nil || h.Total != 1 || len(h.IPs) != 1 || h.IPs[0].Requests != 137 {
t.Fatalf("IP summary %+v %v", h, err)
}
h, err = s.TrafficHistory("", "", "ZZ", "", "", false, false, false, 0, now, 0)
if err != nil || h.Total != 0 {
t.Fatal("country filter ignored")
}
h, err = s.TrafficHistory("", "", "", "1.1.1.1", "", false, false, false, 0, now, 0)
if err != nil || h.Total != 1 {
t.Fatal("IP filter ignored")
}
}
func TestCityNamesAndLocalAddresses(t *testing.T) {
if cityName(map[string]string{"ru": "Москва", "en": "Moscow"}) != "Москва" || cityName(map[string]string{"en": "Amsterdam"}) != "Amsterdam" {
t.Fatal("city localization")
}
for _, ip := range []string{"192.168.0.10", "127.0.0.1", "::1", "fe80::1", "invalid"} {
c, r := lookupTrafficCity(ip)
if c != "" || r != "" {
t.Fatal("local IP geolocated")
}
}
}
func TestTrafficHistoryMonitoringAndAgentIsolation(t *testing.T) {
s, a := trafficFixture(t)
s.SaveTrafficSettings(TrafficSettings{Enabled: true, AgentID: a.ID, MonitoringRules: []MonitoringRule{{IP: "192.168.0.10"}}})
now := time.Now().Unix()
batch := TrafficBatch{Events: []TrafficEvent{{ID: "check", Time: now, Kind: "request", IP: "192.168.0.10"}, {ID: "alert", Time: now, Kind: "alert", IP: "192.168.0.10"}, {ID: "public", Time: now, Kind: "request", IP: "8.8.8.8"}}}
if err := s.SaveTraffic(a.ID, batch); err != nil {
t.Fatal(err)
}
token, _ := s.CreateAgentEnrollmentToken()
other, err := s.EnrollAgent(agentEnrollmentRequest{Token: token, Name: "other", VMID: 103})
if err != nil {
t.Fatal(err)
}
if err = s.SaveTraffic(other.ID, batch); err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
external, hide bool
want int64
}{{false, false, 3}, {false, true, 2}, {true, false, 1}, {true, true, 1}} {
h, err := s.TrafficHistory("", "", "", "", "", tc.external, tc.hide, false, 0, now, 0)
if err != nil || h.Total != tc.want {
t.Fatalf("filters %+v total=%d err=%v", tc, h.Total, err)
}
for _, e := range h.Events {
if e.Agent != a.ID {
t.Fatal("foreign agent leaked")
}
}
}
}

71
traffic_map_cities.go Normal file
View File

@@ -0,0 +1,71 @@
package main
import (
"fmt"
"sort"
)
type TrafficMapCity struct {
TrafficCityLocation
Country string `json:"country"`
Requests int64 `json:"requests"`
UniqueIPs int64 `json:"uniqueIps"`
Alerts int64 `json:"alerts"`
Decisions int64 `json:"decisions"`
LastSeen int64 `json:"lastSeen"`
}
func (s *Store) trafficMapCities(where string, args []any) ([]TrafficMapCity, bool, error) {
rows, err := s.db.Query(`SELECT ip,country,SUM(kind='request'),SUM(kind='alert'),SUM(kind='decision'),MAX(ts) FROM traffic_events`+where+` GROUP BY ip,country`, args...)
if err != nil {
return nil, false, err
}
defer rows.Close()
groups := map[string]*TrafficMapCity{}
for rows.Next() {
var ip, country string
var requests, alerts, decisions, last int64
if err = rows.Scan(&ip, &country, &requests, &alerts, &decisions, &last); err != nil {
return nil, false, err
}
g := lookupTrafficCityLocation(ip)
if !g.Located {
continue
}
key := fmt.Sprintf("%s/%s/%s/%.5f/%.5f", country, g.Region, g.City, g.Latitude, g.Longitude)
c := groups[key]
if c == nil {
c = &TrafficMapCity{TrafficCityLocation: g, Country: country}
groups[key] = c
}
c.Requests += requests
c.Alerts += alerts
c.Decisions += decisions
if requests > 0 {
c.UniqueIPs++
}
if last > c.LastSeen {
c.LastSeen = last
}
}
if err = rows.Err(); err != nil {
return nil, false, err
}
result := make([]TrafficMapCity, 0, len(groups))
for _, c := range groups {
result = append(result, *c)
}
sort.Slice(result, func(i, j int) bool {
a, b := result[i], result[j]
x, y := a.Requests+a.Alerts+a.Decisions, b.Requests+b.Alerts+b.Decisions
if x != y {
return x > y
}
return fmt.Sprint(a.Country, a.Region, a.City, a.Latitude, a.Longitude) < fmt.Sprint(b.Country, b.Region, b.City, b.Latitude, b.Longitude)
})
limited := len(result) > 500
if limited {
result = result[:500]
}
return result, limited, nil
}

49
ups.go
View File

@@ -2,6 +2,7 @@ package main
import ( import (
"context" "context"
"math"
"os/exec" "os/exec"
"strconv" "strconv"
"strings" "strings"
@@ -10,6 +11,13 @@ import (
) )
type UPSMetrics struct { type UPSMetrics struct {
Raw map[string]string `json:"raw"`
NominalPower *float64 `json:"nominalPowerWatts"`
Power *float64 `json:"powerWatts"`
PowerEstimated bool `json:"powerEstimated"`
InputFrequencyRaw *float64 `json:"inputFrequencyRaw"`
FrequencyCorrected bool `json:"frequencyCorrected"`
Available bool `json:"available"` Available bool `json:"available"`
Source string `json:"source"` Source string `json:"source"`
Name string `json:"name"` Name string `json:"name"`
@@ -92,14 +100,16 @@ func readAPCUPSD() (UPSMetrics, bool) {
} }
raw := parseColonValues(string(output)) raw := parseColonValues(string(output))
values := map[string]string{ values := map[string]string{
"device.model": raw["MODEL"], "ups.status": raw["STATUS"], "device.model": raw["MODEL"], "ups.realpower.nominal": raw["NOMPOWER"], "ups.status": raw["STATUS"],
"battery.charge": raw["BCHARGE"], "battery.runtime": minutesToSeconds(raw["TIMELEFT"]), "battery.charge": raw["BCHARGE"], "battery.runtime": minutesToSeconds(raw["TIMELEFT"]),
"ups.load": raw["LOADPCT"], "input.voltage": raw["LINEV"], "ups.load": raw["LOADPCT"], "input.voltage": raw["LINEV"],
"output.voltage": raw["OUTPUTV"], "input.frequency": raw["LINEFREQ"], "output.voltage": raw["OUTPUTV"], "input.frequency": raw["LINEFREQ"],
"battery.voltage": raw["BATTV"], "ups.temperature": raw["ITEMP"], "battery.voltage": raw["BATTV"], "ups.temperature": raw["ITEMP"],
"battery.date": raw["BATTDATE"], "battery.charge.low": raw["MBATTCHG"], "battery.runtime.low": minutesToSeconds(raw["MINTIMEL"]), "battery.date": raw["BATTDATE"], "battery.charge.low": raw["MBATTCHG"], "battery.runtime.low": minutesToSeconds(raw["MINTIMEL"]),
} }
return upsFromValues("apcupsd", raw["UPSNAME"], values), true m := upsFromValues("apcupsd", raw["UPSNAME"], values)
m.Raw = raw
return m, true
} }
func parseColonValues(output string) map[string]string { func parseColonValues(output string) map[string]string {
@@ -116,7 +126,7 @@ func parseColonValues(output string) map[string]string {
func upsFromValues(source, name string, values map[string]string) UPSMetrics { func upsFromValues(source, name string, values map[string]string) UPSMetrics {
status := values["ups.status"] status := values["ups.status"]
metrics := UPSMetrics{ metrics := UPSMetrics{
Available: true, CommunicationOK: true, Source: source, Name: name, Model: values["device.model"], Available: true, CommunicationOK: true, Source: source, Name: name, Model: firstNonEmpty(values["device.model"], values["ups.model"]), Raw: values,
Status: status, StatusLabel: upsStatusLabel(status), Status: status, StatusLabel: upsStatusLabel(status),
ChargePercent: numberPointer(values["battery.charge"]), RuntimeSeconds: numberPointer(values["battery.runtime"]), ChargePercent: numberPointer(values["battery.charge"]), RuntimeSeconds: numberPointer(values["battery.runtime"]),
LoadPercent: numberPointer(values["ups.load"]), InputVoltage: numberPointer(values["input.voltage"]), LoadPercent: numberPointer(values["ups.load"]), InputVoltage: numberPointer(values["input.voltage"]),
@@ -124,13 +134,44 @@ func upsFromValues(source, name string, values map[string]string) UPSMetrics {
BatteryVoltage: numberPointer(values["battery.voltage"]), Temperature: numberPointer(firstNonEmpty(values["ups.temperature"], values["battery.temperature"])), BatteryVoltage: numberPointer(values["battery.voltage"]), Temperature: numberPointer(firstNonEmpty(values["ups.temperature"], values["battery.temperature"])),
BatteryDate: firstNonEmpty(values["battery.date"], values["battery.mfr.date"]), ReplaceBattery: strings.Contains(status, "RB"), LowBatteryCharge: numberPointer(values["battery.charge.low"]), LowBatteryRuntime: numberPointer(values["battery.runtime.low"]), BatteryDate: firstNonEmpty(values["battery.date"], values["battery.mfr.date"]), ReplaceBattery: strings.Contains(status, "RB"), LowBatteryCharge: numberPointer(values["battery.charge.low"]), LowBatteryRuntime: numberPointer(values["battery.runtime.low"]),
} }
metrics.NominalPower = numberPointer(values["ups.realpower.nominal"])
metrics.Power = numberPointer(values["ups.realpower"])
if metrics.Power != nil && *metrics.Power < 0 {
metrics.Power = nil
}
if metrics.NominalPower != nil && *metrics.NominalPower <= 0 {
metrics.NominalPower = nil
}
if metrics.Power == nil && metrics.NominalPower != nil && metrics.LoadPercent != nil && *metrics.LoadPercent >= 0 {
watts := *metrics.NominalPower * *metrics.LoadPercent / 100
if !math.IsInf(watts, 0) {
metrics.Power = &watts
metrics.PowerEstimated = true
}
}
metrics.InputFrequencyRaw = metrics.InputFrequency
mfr := strings.ToLower(firstNonEmpty(values["device.mfr"], values["ups.mfr"]))
cyberpower := mfr == "cps" || strings.Contains(mfr, "cyberpower") || values["ups.vendorid"] == "0764"
if source == "NUT" && cyberpower && metrics.InputFrequency != nil && *metrics.InputFrequency >= 450 && *metrics.InputFrequency <= 650 {
hz := *metrics.InputFrequency / 10
metrics.InputFrequency = &hz
metrics.FrequencyCorrected = true
}
if metrics.BatteryDate != "" { if metrics.BatteryDate != "" {
validDate := false
for _, layout := range []string{"2006/01/02", "2006-01-02", "01/02/2006", "02/01/2006"} { for _, layout := range []string{"2006/01/02", "2006-01-02", "01/02/2006", "02/01/2006"} {
if date, err := time.Parse(layout, metrics.BatteryDate); err == nil { if date, err := time.Parse(layout, metrics.BatteryDate); err == nil {
if date.After(time.Now()) {
continue
}
validDate = true
metrics.BatteryAgeDays = int(time.Since(date).Hours() / 24) metrics.BatteryAgeDays = int(time.Since(date).Hours() / 24)
break break
} }
} }
if !validDate {
metrics.BatteryDate = ""
}
} }
return metrics return metrics
} }
@@ -141,7 +182,7 @@ func numberPointer(value string) *float64 {
return nil return nil
} }
number, err := strconv.ParseFloat(strings.TrimSuffix(field[0], "%"), 64) number, err := strconv.ParseFloat(strings.TrimSuffix(field[0], "%"), 64)
if err != nil { if err != nil || math.IsNaN(number) || math.IsInf(number, 0) {
return nil return nil
} }
return &number return &number

68
ups_preview_test.go Normal file
View File

@@ -0,0 +1,68 @@
package main
import (
"encoding/json"
"fmt"
"net/http"
"os"
"strconv"
"strings"
"testing"
"time"
)
func TestUPSPreview(t *testing.T) {
if os.Getenv("DASHBOARD_UPS_PREVIEW") != "1" {
t.Skip("manual visual fixture")
}
s, a := trafficFixture(t)
_ = a
u := upsFromValues("NUT", "cyberpower", parseColonValues(cyberPowerSample))
m := dashboardMetrics{UPS: u, CPU: CPUMetrics{CPUStatic: CPUStatic{Model: "DEMO / искусственные данные"}, Temperatures: []Temperature{}}, Timestamp: time.Now()}
s.AddEntityMetrics(m)
for i := 0; i < 30; i++ {
ts := time.Now().Add(time.Duration(i-30) * time.Minute).Unix()
v := 220 + float64(i%6)
out := 225.0
power := 150 + float64(i)
runtime := 450.0
freq := 49.8
bat := 13.4
s.AddEntityHistory("ups-voltage", "Вход", ts, &v, nil)
s.AddEntityHistory("ups-voltage", "Выход", ts, &out, nil)
s.AddEntityHistory("ups-power", "Расчётная ≈", ts, &power, nil)
s.AddEntityHistory("ups-runtime", "Автономность", ts, &runtime, nil)
s.AddEntityHistory("ups-frequency", "Частота", ts, &freq, nil)
s.AddEntityHistory("ups-battery", "Батарея", ts, &bat, nil)
p := historyPointFromMetrics(m)
p.Timestamp = ts
s.AddHistory(p)
}
mux := http.NewServeMux()
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
writeJSON(w, map[string]any{}, nil)
return
}
w.Header().Set("Content-Type", "text/html")
w.Write([]byte(strings.Replace(dashboardHTML, "<title>", "<title>DEMO / ", 1)))
})
mux.HandleFunc("/api/events", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
b, _ := json.Marshal(m)
fmt.Fprintf(w, "data: %s\n\n", b)
w.(http.Flusher).Flush()
<-r.Context().Done()
})
mux.HandleFunc("/api/history", func(w http.ResponseWriter, r *http.Request) {
h, _ := strconv.Atoi(r.URL.Query().Get("hours"))
v, e := s.History(h)
writeJSON(w, v, e)
})
mux.HandleFunc("/api/entity-history", func(w http.ResponseWriter, r *http.Request) {
h, _ := strconv.Atoi(r.URL.Query().Get("hours"))
v, e := s.EntityHistory(r.URL.Query().Get("kind"), h)
writeJSON(w, v, e)
})
t.Fatal(http.ListenAndServe("127.0.0.1:19106", mux))
}

View File

@@ -12,3 +12,104 @@ func TestUPSParsing(t *testing.T) {
t.Fatalf("неожиданный статус: %s", metrics.StatusLabel) t.Fatalf("неожиданный статус: %s", metrics.StatusLabel)
} }
} }
const cyberPowerSample = `device.mfr: CPS
device.model: VP700ELCD
battery.charge: 100
battery.charge.low: 10
battery.charge.warning: 20
battery.mfr.date: CPS
battery.runtime: 450
battery.runtime.low: 300
battery.type: PbAcid
battery.voltage: 13.4
battery.voltage.nominal: 12
input.frequency: 498.0
input.transfer.high: 295
input.transfer.low: 167
input.voltage: 224.0
input.voltage.nominal: 230
output.voltage: 225.0
ups.beeper.status: disabled
ups.delay.shutdown: 20
ups.delay.start: 30
ups.load: 46
ups.realpower.nominal: 390
ups.status: OL
ups.test.result: No test initiated
driver.name: usbhid-ups
driver.version: 2.8.1
`
func TestCyberPowerFrequencyPowerAndRaw(t *testing.T) {
raw := parseColonValues(cyberPowerSample)
m := upsFromValues("NUT", "cyberpower", raw)
if !m.FrequencyCorrected || *m.InputFrequency != 49.8 || *m.InputFrequencyRaw != 498 || m.Raw["input.frequency"] != "498.0" {
t.Fatalf("frequency %+v", m)
}
if m.Power == nil || *m.Power != 179.4 || !m.PowerEstimated {
t.Fatalf("power %+v", m)
}
if m.BatteryDate != "" || m.BatteryAgeDays != 0 || m.Raw["battery.mfr.date"] != "CPS" {
t.Fatal("invalid date not preserved as raw only")
}
raw["input.frequency"] = "49.8"
raw["ups.realpower"] = "165"
m = upsFromValues("NUT", "ups", raw)
if m.FrequencyCorrected || *m.InputFrequency != 49.8 || m.PowerEstimated || *m.Power != 165 {
t.Fatal("double correction or ignored measured power")
}
raw["device.mfr"] = "Other"
raw["input.frequency"] = "498"
if m = upsFromValues("NUT", "ups", raw); m.FrequencyCorrected || *m.InputFrequency != 498 {
t.Fatal("other vendor corrected")
}
for _, v := range []string{"NaN", "Inf", "-Inf", ""} {
if numberPointer(v) != nil {
t.Fatal("nonfinite accepted")
}
}
if m = upsFromValues("NUT", "ups", map[string]string{"ups.load": "46"}); m.Power != nil {
t.Fatal("invented power without nominal")
}
}
func TestUPSHistoryRecordsCorrectedValuesAndGaps(t *testing.T) {
s, a := trafficFixture(t)
_ = a
u := upsFromValues("NUT", "cyberpower", parseColonValues(cyberPowerSample))
s.AddEntityMetrics(dashboardMetrics{UPS: u})
for _, tc := range []struct {
kind, name string
want float64
}{{"ups-frequency", "Частота", 49.8}, {"ups-power", "Расчётная ≈", 179.4}, {"ups-runtime", "Автономность", 450}} {
rows, err := s.EntityHistory(tc.kind, 1)
if err != nil {
t.Fatal(err)
}
found := false
for _, r := range rows {
if r.Name == tc.name {
found = true
if r.Points[0].Value1 == nil || *r.Points[0].Value1 != tc.want {
t.Fatalf("bad %s: %+v", tc.kind, r)
}
}
}
if !found {
t.Fatal("missing history")
}
}
u.Available = false
u.CommunicationOK = false
s.AddEntityMetrics(dashboardMetrics{UPS: u})
rows, _ := s.EntityHistory("ups-frequency", 1)
for _, r := range rows {
if r.Points[len(r.Points)-1].Value1 != nil {
t.Fatal("stale reading stored as live")
}
}
p := historyPointFromMetrics(dashboardMetrics{UPS: u})
if p.UPSCharge != nil || p.UPSLoad != nil {
t.Fatal("stale charge stored")
}
}

52
web.go

File diff suppressed because one or more lines are too long