diff --git a/CHANGELOG.md b/CHANGELOG.md index ab53869..2e2b3d8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,11 +9,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- **Bootstrap Never Repaired a Broken Sudoers File** - 2.1.1 generates valid rules, but `magebox bootstrap` skipped the whole step whenever `/etc/sudoers.d/magebox` already existed, so every machine upgrading from an older release kept its rejected wildcard rules and nothing improved. The step now always runs; the installer decides whether a rewrite is needed and leaves a correct file alone. +- **Bootstrap Aborted Before It Could Fix the PHP Repository** - `InstallPrerequisites` ran `apt update` first and returned on any error. A PPA with no packages for the running release makes apt exit 100, so bootstrap gave up before reaching the code that repairs exactly that repository. Both `apt update` calls are now warnings rather than failures, and `add-apt-repository`'s own exit code is ignored for the same reason. +- **One Missing Certificate Took Every Site Offline** - nginx refuses to start when a vhost references a certificate that is not on disk, so a single stale project stopped every other one. Bootstrap now checks the certificates every vhost references, regenerates the missing ones it manages, and names the file to remove for any it does not. +- **"Docker Is Not Running" When Docker Was Running** - A permission error on the Docker socket was reported as a stopped daemon, sending users to restart something that was already up. Bootstrap now distinguishes the two and, for the permission case, prints the command that adds the user to the docker group. +- **One Half-Configured Package Blocked Every Install** - A package left half-configured by an earlier failure makes `apt install` exit 100 whatever it is asked for. Bootstrap treated that as fatal while installing its base tools, so it never reached the repository configuration that would have made PHP installable, and every PHP version was then reported as unavailable. Bootstrap now finishes configuring broken packages with `dpkg --configure -a` before installing, and a failure to install base tools is a warning rather than the end of the run. +- **Every PHP Version Skipped Because apt Could Not Read a Source File** - MageBox filters out packages `apt-cache` does not know, but apt returns the same empty answer whether a package is missing or a sources file could not be read. A repository file MageBox itself wrote root-only therefore made every `php8.1` … `php8.4` package look unavailable, so bootstrap silently installed none and reported success. The check now treats a permission warning as "cannot tell" and lets apt decide, so an install either works or fails with a message worth reading. +- **"Local CA Already Installed" While Browsers Trusted a Different One** - The check only looked for `rootCA.pem` on disk, so on a machine restored from another install MageBox reported the certificate authority as installed while the browser still trusted the old one, and every local HTTPS site warned. Bootstrap now compares the authority's serial against the browser trust store and reinstalls it when they differ. +- **systemd-resolved Silently Ignored MageBox's Configuration** - MageBox writes config through a temp file, which `os.CreateTemp` creates as root-readable only, and `cp` keeps that mode. systemd-resolved reads its configuration after dropping privileges, so it refused the drop-in with "Permission denied" at every restart and `.test` names never resolved, while MageBox reported DNS as configured. Config files are now written world-readable, and callers that need something stricter, such as sudoers, still set their own mode. +- **Bootstrap Reported Working DNS When Only dnsmasq Answered** - The check queried dnsmasq directly, which proves dnsmasq works and nothing else. Everything on the machine goes through the system resolver, so browsers and curl kept failing after a "resolves" line. Bootstrap now checks the system resolver too, and says which of the two answered. +- **`.test` Names Failed Outright After a dnsmasq Fallback** - The systemd-resolved drop-in that sends `.test` lookups to MageBox's dnsmasq is written before dnsmasq is known to work. When dnsmasq could not be started, bootstrap fell back to `/etc/hosts` but left the drop-in in place, pointing every `.test` lookup at a resolver that was not running, so names failed instead of falling through to the hosts file. The fallback now removes the drop-in and restarts systemd-resolved. +- **Bootstrap Ran as Root Broke the Machine** - Run with `sudo`, bootstrap built the whole environment under `/root`: the config, the certificate authority, PHP-FPM pools and an nginx include pointing into a directory nginx cannot read, which stopped nginx from starting at all. Bootstrap now refuses to run as root and says to run it as the normal user, which asks for a password where it needs one. +- **php-fpm.conf Accumulated Includes Until PHP-FPM Would Not Start** - Every bootstrap run appended another MageBox pools include. Duplicates define the same pool twice, and an include left by a run under another user matches nothing; either stops PHP-FPM from starting, and a PHP-FPM that cannot start makes every later `dpkg` operation on the package fail, which cascaded into failed Blackfire and dnsmasq installs. Bootstrap now rewrites the file to hold exactly one include, dropping duplicates and includes naming another user's home. +- **dnsmasq Reported as Installed When Only the Library Was** - Ubuntu's `dnsmasq-base` provides the binary without a systemd unit, so MageBox skipped the install and then failed to start the service with a bare exit code. Installation now also requires the service unit on systemd machines. - **Passwordless Sudo Broken on Ubuntu 26.04** - Ubuntu 26.04 ships sudo-rs, which refuses to parse wildcards in command arguments. MageBox wrote rules such as `systemctl start php*-fpm`, `cp /tmp/magebox-* /etc/nginx/nginx.conf` and `apt install -y blackfire*`, so `/etc/sudoers.d/magebox` failed to parse entirely and every MageBox operation, including `sudo -s`, printed parse errors and asked for a password. Rules are now generated as complete commands, one line per PHP version and action, and are identical whether they come from the Go installers or the YAML installer definitions. Bootstrap validates the file with `visudo` before installing it and replaces an existing file that the local sudo rejects, so upgrading and re-running `magebox bootstrap` repairs a broken system. -- **PHP 8.1 to 8.4 Could Not Be Installed on Ubuntu 26.04** - Ondrej Sury's PPA publishes nothing for Ubuntu releases it has not caught up with, so on 26.04 apt had no `php8.1` … `php8.4` packages and only Ubuntu's own PHP 8.5 could be installed. Bootstrap now checks which suites the PPA publishes, pins the newest one available, and says so. Packages built for the previous release normally install cleanly; a version that fails is reported and bootstrap continues. - -### Changed - +- **PHP 8.1 to 8.4 Could Not Be Installed on Ubuntu 26.04** - Ondrej Sury's PPA publishes nothing past Ubuntu 24.04 and is being folded into packages.sury.org, which does cover 26.04. Bootstrap now picks the repository that covers the running release: the PPA where it still does, packages.sury.org where it does not, and neither with a clear warning when no repository covers the release at all. Pinning the PPA's older suite was tried first and is worse than useless, because those packages depend on library versions (libxml2, libicu74, libzip4t64) the newer release no longer ships, so every install fails on unsatisfiable dependencies. - **Ubuntu 26.04 is recognised as a supported release** - It no longer triggers the "not officially tested" warning. - **Passwordless sudo is limited to service control** - Only starting, stopping, reloading and restarting nginx, PHP-FPM and the Blackfire agent, plus `nginx -t` and `nginx -s reload`, run without a password. Commands that run during bootstrap or an explicit install ask for one, as do Xdebug and Blackfire configuration edits. This also removes the previous `sed -i *` and `ln -s *` rules, which allowed arbitrary arguments and amounted to unrestricted root for the MageBox user. diff --git a/cmd/magebox/bootstrap.go b/cmd/magebox/bootstrap.go index 733bdf6..0109adb 100644 --- a/cmd/magebox/bootstrap.go +++ b/cmd/magebox/bootstrap.go @@ -69,6 +69,18 @@ func init() { func runBootstrap(cmd *cobra.Command, args []string) error { verbose.Section("Bootstrap Starting") + // Run as root, bootstrap would build the whole environment under /root: + // the config, certificates, PHP-FPM pools and an nginx include pointing at + // a directory nginx cannot read, which stops nginx from starting at all. + if installer.ShouldRefuseRootBootstrap(os.Geteuid()) { + cli.PrintError("Do not run bootstrap as root.") + fmt.Println() + fmt.Println(" Run it as your own user; it asks for a sudo password where it needs one:") + fmt.Println(" magebox bootstrap") + fmt.Println() + return fmt.Errorf("bootstrap must not run as root") + } + p, err := getPlatform() if err != nil { verbose.Debug("Failed to detect platform: %v", err) @@ -165,12 +177,22 @@ func runBootstrap(cmd *cobra.Command, args []string) error { errors = append(errors, "Docker is not installed. Install: "+bootstrapper.DockerInstallInstructions()) } else { verbose.Debug("Docker binary found, checking daemon...") - // Check if Docker daemon is running - if !bootstrapper.CheckDockerRunning() { + // Check whether Docker answers, and why it does not + switch issue := bootstrapper.CheckDocker(); issue { + case bootstrap.DockerPermissionDenied: + verbose.Debug("Docker socket is not accessible for this user") + cli.PrintWarning("Docker is running but this user cannot reach its socket.") + cli.PrintInfo("Fix: %s", issue.Advice()) + errors = append(errors, "Docker socket is not accessible for this user") + case bootstrap.DockerNotRunning: verbose.Debug("Docker daemon is not running") cli.PrintWarning("Docker is installed but not running. Please start Docker.") errors = append(errors, "Docker daemon is not running") - } else { + case bootstrap.DockerUnknownFailure: + verbose.Debug("Docker did not answer") + cli.PrintWarning("Docker did not answer. %s", issue.Advice()) + errors = append(errors, "Docker is not answering") + default: verbose.Debug("Docker daemon is running") } } @@ -674,6 +696,38 @@ func runBootstrap(cmd *cobra.Command, args []string) error { } } + // Repair certificates referenced by vhosts but missing on disk. nginx + // refuses to start over a single one, taking every project offline. + fmt.Print(" Checking vhost certificates... ") + if missing, err := nginx.MissingCertificates(vhostsDir); err != nil { + fmt.Println(cli.Warning("skipped")) + } else if len(missing) == 0 { + fmt.Println(cli.Success("ok")) + } else { + fmt.Println(cli.Warning(fmt.Sprintf("%d vhost(s) reference missing certificates", len(missing)))) + for vhost, certs := range missing { + domain := "" + for _, cert := range certs { + if d := nginx.DomainFromCertPath(cert); d != "" { + domain = d + break + } + } + if domain == "" { + cli.PrintWarning("%s references a certificate MageBox does not manage: %s", filepath.Base(vhost), certs[0]) + continue + } + fmt.Printf(" Regenerating certificate for %s... ", domain) + if _, err := sslMgr.EnsureCert(domain, domain); err != nil { + fmt.Println(cli.Error("failed")) + cli.PrintWarning("Could not regenerate %s: %v", domain, err) + cli.PrintInfo("Remove %s or run 'magebox start' in that project to fix it.", vhost) + } else { + fmt.Println(cli.Success("done")) + } + } + } + // Test and reload nginx fmt.Print(" Testing nginx config... ") if err := nginxCtrl.Test(); err != nil { @@ -800,12 +854,37 @@ func runBootstrap(cmd *cobra.Command, args []string) error { fmt.Println(" Falling back to /etc/hosts mode") globalCfg.DNSMode = "hosts" _ = config.SaveGlobalConfig(homeDir, globalCfg) + + // The systemd-resolved drop-in is written before dnsmasq is known to + // work. Left behind, it sends every .test lookup to a resolver that is + // not running, so names fail instead of falling through to /etc/hosts. + if dns.NeedsResolvedCleanup(false, dns.ResolvedDropInPresent()) { + fmt.Print(" Removing the systemd-resolved override... ") + if err := dnsManager.RemoveSystemdResolvedConfig(); err != nil { + fmt.Println(cli.Error("failed")) + cli.PrintWarning("Remove %s by hand, or .test names will not resolve: %v", dns.ResolvedDropInPath, err) + } else { + fmt.Println(cli.Success("done")) + } + } + cli.PrintInfo("Domains will be added to /etc/hosts when you run %s", cli.Command("magebox start")) } // Test DNS resolution if dnsmasq was configured if dnsmasqConfigured { testDomain := fmt.Sprintf("test.%s", tld) + + // dnsmasq answering says nothing about the rest of the machine: the + // systemd-resolved drop-in may not have been read, in which case + // browsers and curl still fail. + if dns.SystemResolves(testDomain) { + fmt.Printf(" System resolver answers for %s %s\n", testDomain, cli.Success("✓")) + } else if dnsManager.TestResolution(testDomain) { + cli.PrintWarning("dnsmasq answers for %s but the system resolver does not.", testDomain) + cli.PrintInfo("Check that systemd-resolved can read %s, then restart it: sudo systemctl restart systemd-resolved", dns.ResolvedDropInPath) + } + fmt.Printf(" Testing DNS resolution for %s... ", testDomain) if dnsManager.TestResolution(testDomain) { fmt.Println(cli.Success("✓ resolves to 127.0.0.1")) @@ -916,17 +995,15 @@ func runBootstrap(cmd *cobra.Command, args []string) error { if p.Type == platform.Linux { fmt.Println(cli.Header("Step 10: Sudoers Configuration")) - sudoersFile := "/etc/sudoers.d/magebox" - if _, err := os.Stat(sudoersFile); err == nil { - fmt.Println(" Sudoers already configured " + cli.Success("✓")) + // Never skip because the file exists: a file written by an older + // MageBox contains wildcards that today's sudo rejects, which + // disables every rule in it. + fmt.Print(" Setting up passwordless nginx/php-fpm control... ") + if err := bootstrapper.ConfigureSudoers(); err != nil { + fmt.Println(cli.Error("failed")) + cli.PrintWarning("Failed to setup sudoers: %v", err) } else { - fmt.Print(" Setting up passwordless nginx/php-fpm control... ") - if err := bootstrapper.ConfigureSudoers(); err != nil { - fmt.Println(cli.Error("failed")) - cli.PrintWarning("Failed to setup sudoers: %v", err) - } else { - fmt.Println(cli.Success("done")) - } + fmt.Println(cli.Success("done")) } fmt.Println() } diff --git a/internal/bootstrap/docker.go b/internal/bootstrap/docker.go new file mode 100644 index 0000000..f6a393f --- /dev/null +++ b/internal/bootstrap/docker.go @@ -0,0 +1,61 @@ +package bootstrap + +import ( + "os/exec" + "strings" +) + +// DockerIssue explains why talking to Docker failed. +type DockerIssue int + +const ( + // DockerOK means the daemon answered. + DockerOK DockerIssue = iota + // DockerNotRunning means the daemon is not up. + DockerNotRunning + // DockerPermissionDenied means the daemon is up but this user may not + // reach its socket — usually a missing membership of the docker group. + DockerPermissionDenied + // DockerUnknownFailure means the command failed for another reason. + DockerUnknownFailure +) + +// Advice returns what the user should do about the issue. +func (d DockerIssue) Advice() string { + switch d { + case DockerNotRunning: + return "start Docker and run bootstrap again" + case DockerPermissionDenied: + return "add yourself to the docker group, then log out and back in: sudo usermod -aG docker $USER" + case DockerUnknownFailure: + return "run 'docker info' to see what Docker reports" + default: + return "" + } +} + +// ClassifyDockerFailure reads the output of a failed "docker info". +// +// A permission error means the daemon is running and reachable by others; +// telling the user to start Docker would send them the wrong way. +func ClassifyDockerFailure(output string) DockerIssue { + lower := strings.ToLower(output) + switch { + case strings.Contains(lower, "permission denied"): + return DockerPermissionDenied + case strings.Contains(lower, "cannot connect to the docker daemon"), + strings.Contains(lower, "is the docker daemon running"): + return DockerNotRunning + default: + return DockerUnknownFailure + } +} + +// CheckDocker reports whether Docker answers, and why it does not. +func (b *Bootstrapper) CheckDocker() DockerIssue { + output, err := exec.Command("docker", "info").CombinedOutput() + if err == nil { + return DockerOK + } + return ClassifyDockerFailure(string(output)) +} diff --git a/internal/bootstrap/docker_test.go b/internal/bootstrap/docker_test.go new file mode 100644 index 0000000..425ee41 --- /dev/null +++ b/internal/bootstrap/docker_test.go @@ -0,0 +1,72 @@ +package bootstrap + +import "testing" + +// "docker info" fails for two very different reasons, and telling the user to +// start a daemon that is already running sends them the wrong way. +func TestClassifyDockerFailure(t *testing.T) { + tests := []struct { + name string + output string + want DockerIssue + wantAdvice string + }{ + { + name: "user is not in the docker group", + output: "permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock", + want: DockerPermissionDenied, + wantAdvice: "docker group", + }, + { + name: "daemon is down", + output: "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?", + want: DockerNotRunning, + wantAdvice: "start Docker", + }, + { + name: "anything else", + output: "something unexpected", + want: DockerUnknownFailure, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := ClassifyDockerFailure(tt.output) + if got != tt.want { + t.Errorf("ClassifyDockerFailure() = %v, want %v", got, tt.want) + } + if tt.wantAdvice != "" && !containsFold(got.Advice(), tt.wantAdvice) { + t.Errorf("advice %q does not mention %q", got.Advice(), tt.wantAdvice) + } + }) + } +} + +func containsFold(haystack, needle string) bool { + for i := 0; i+len(needle) <= len(haystack); i++ { + if equalFold(haystack[i:i+len(needle)], needle) { + return true + } + } + return false +} + +func equalFold(a, b string) bool { + if len(a) != len(b) { + return false + } + for i := range a { + x, y := a[i], b[i] + if 'A' <= x && x <= 'Z' { + x += 'a' - 'A' + } + if 'A' <= y && y <= 'Z' { + y += 'a' - 'A' + } + if x != y { + return false + } + } + return true +} diff --git a/internal/bootstrap/installer/aptcache_test.go b/internal/bootstrap/installer/aptcache_test.go new file mode 100644 index 0000000..1ac7ada --- /dev/null +++ b/internal/bootstrap/installer/aptcache_test.go @@ -0,0 +1,40 @@ +package installer + +import "testing" + +// apt-cache answers "unknown package" the same way whether the package really +// is missing or apt could not read a sources file. Treating an unreadable +// source as "package unavailable" made bootstrap skip every PHP package and +// report success, leaving the machine with only the distribution's PHP. +func TestAptCacheAnswerReliable(t *testing.T) { + tests := []struct { + name string + stderr string + want bool + }{ + {name: "clean run", stderr: "", want: true}, + { + name: "sources file not readable", + stderr: "W: Unable to read /etc/apt/sources.list.d/magebox-php.list - open (13: Permission denied)", + want: false, + }, + { + name: "lists directory not readable", + stderr: "E: Could not open file /var/lib/apt/lists/… - open (13: Permission denied)", + want: false, + }, + { + name: "ordinary warning about a missing package", + stderr: "N: Unable to locate package php9.9-fpm", + want: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := aptCacheAnswerReliable(tt.stderr); got != tt.want { + t.Errorf("aptCacheAnswerReliable(%q) = %v, want %v", tt.stderr, got, tt.want) + } + }) + } +} diff --git a/internal/bootstrap/installer/base.go b/internal/bootstrap/installer/base.go index d21578b..6ed6c2f 100644 --- a/internal/bootstrap/installer/base.go +++ b/internal/bootstrap/installer/base.go @@ -71,7 +71,15 @@ func (b *BaseInstaller) WriteFile(path, content string) error { tmpFile.Close() // Copy to destination with sudo (use RunSudo to allow password prompt) - return b.RunSudo("cp", tmpPath, path) + if err := b.RunSudo("cp", tmpPath, path); err != nil { + return err + } + + // os.CreateTemp makes the file 0600, and cp keeps that mode. Daemons that + // read their configuration after dropping privileges (systemd-resolved, + // for one) then cannot open it and silently ignore the file. Callers that + // need something stricter, such as sudoers, set it afterwards. + return b.RunSudo("chmod", "0644", path) } // CommandExists checks if a command is available diff --git a/internal/bootstrap/installer/dpkgrepair.go b/internal/bootstrap/installer/dpkgrepair.go new file mode 100644 index 0000000..150c850 --- /dev/null +++ b/internal/bootstrap/installer/dpkgrepair.go @@ -0,0 +1,31 @@ +package installer + +import ( + "os/exec" + "strings" + + "qoliber/magebox/internal/verbose" +) + +// dpkgAuditReportsBroken reports whether dpkg found packages left in a broken +// state. Its output is empty when everything is configured. +func dpkgAuditReportsBroken(auditOutput string) bool { + return strings.TrimSpace(auditOutput) != "" +} + +// repairBrokenPackages finishes configuring packages left half-installed. +// +// One such package makes every apt install exit 100, whatever it was asked to +// install, so bootstrap could not install anything and could not repair the +// repository that would have let it. +func (u *UbuntuInstaller) repairBrokenPackages() { + output, err := exec.Command("dpkg", "--audit").Output() + if err != nil || !dpkgAuditReportsBroken(string(output)) { + return + } + + verbose.Debug("dpkg reports broken packages, running dpkg --configure -a") + if err := u.RunSudo("dpkg", "--configure", "-a"); err != nil { + verbose.Debug("dpkg --configure -a did not succeed: %v", err) + } +} diff --git a/internal/bootstrap/installer/dpkgrepair_test.go b/internal/bootstrap/installer/dpkgrepair_test.go new file mode 100644 index 0000000..ed69b25 --- /dev/null +++ b/internal/bootstrap/installer/dpkgrepair_test.go @@ -0,0 +1,33 @@ +package installer + +import "testing" + +// A package left half-configured makes every later apt install exit 100, no +// matter what it is asked to install. Bootstrap then gave up before repairing +// the repository, so nothing could be installed at all. +func TestDpkgAuditReportsBroken(t *testing.T) { + tests := []struct { + name string + output string + want bool + }{ + {name: "nothing broken", output: "", want: false}, + {name: "only whitespace", output: "\n \n", want: false}, + { + name: "half-configured package", + output: `The following packages are only half configured, probably due to problems +configuring them the first time. The configuration should be retried using +dpkg --configure or the configure menu option in dselect: + php8.5-fpm server-side, HTML-embedded scripting language (FPM-CGI binary)`, + want: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := dpkgAuditReportsBroken(tt.output); got != tt.want { + t.Errorf("dpkgAuditReportsBroken() = %v, want %v", got, tt.want) + } + }) + } +} diff --git a/internal/bootstrap/installer/fedora.go b/internal/bootstrap/installer/fedora.go index eb59229..305253e 100644 --- a/internal/bootstrap/installer/fedora.go +++ b/internal/bootstrap/installer/fedora.go @@ -204,14 +204,16 @@ func (f *FedoraInstaller) ConfigurePHPFPM(versions []string) error { } } - // Add MageBox pools include to php-fpm.conf if not already present + // Keep exactly one MageBox include. Appending on every run left + // duplicates, which define the same pool twice, and a run under a + // different user added an include matching nothing. Either stops + // PHP-FPM from starting. if f.FileExists(fpmConfPath) { - // Check if include already exists - checkCmd := exec.Command("grep", "-q", mageboxPoolsInclude, fpmConfPath) - if checkCmd.Run() != nil { - // Include not found, add it - if err := f.RunSudo("sh", "-c", fmt.Sprintf("echo '%s' >> %s", mageboxPoolsInclude, fpmConfPath)); err != nil { - return fmt.Errorf("failed to add MageBox pools include to %s: %w", fpmConfPath, err) + if content, readErr := os.ReadFile(fpmConfPath); readErr == nil { + if updated, changed := NormalizeFPMIncludes(string(content), mageboxPoolsInclude); changed { + if err := f.WriteFile(fpmConfPath, updated); err != nil { + return fmt.Errorf("failed to update %s: %w", fpmConfPath, err) + } } } } diff --git a/internal/bootstrap/installer/fpmconf.go b/internal/bootstrap/installer/fpmconf.go new file mode 100644 index 0000000..baf432c --- /dev/null +++ b/internal/bootstrap/installer/fpmconf.go @@ -0,0 +1,47 @@ +package installer + +import ( + "regexp" + "strings" +) + +// mageboxPoolInclude matches any MageBox pools include, whatever home it names. +var mageboxPoolInclude = regexp.MustCompile(`^\s*include\s*=\s*\S*/\.magebox/php/pools/\S*\.conf\s*$`) + +// NormalizeFPMIncludes returns php-fpm.conf content holding exactly one MageBox +// include, the given one, and reports whether anything changed. +// +// Appending on every run left duplicates, which define the same pool twice, and +// a run as root added an include under /root that matches nothing. Either stops +// PHP-FPM from starting, and a PHP-FPM that cannot start breaks every later +// dpkg operation on the package. +func NormalizeFPMIncludes(content, want string) (string, bool) { + lines := strings.Split(content, "\n") + kept := make([]string, 0, len(lines)+1) + seen := false + changed := false + + for _, line := range lines { + if !mageboxPoolInclude.MatchString(line) { + kept = append(kept, line) + continue + } + if strings.TrimSpace(line) == want && !seen { + seen = true + kept = append(kept, line) + continue + } + // A duplicate, or an include naming a different home. + changed = true + } + + result := strings.Join(kept, "\n") + if !seen { + if !strings.HasSuffix(result, "\n") { + result += "\n" + } + result += want + "\n" + changed = true + } + return result, changed +} diff --git a/internal/bootstrap/installer/fpmconf_test.go b/internal/bootstrap/installer/fpmconf_test.go new file mode 100644 index 0000000..3da228f --- /dev/null +++ b/internal/bootstrap/installer/fpmconf_test.go @@ -0,0 +1,61 @@ +package installer + +import "testing" + +// php-fpm.conf accumulated one MageBox include per bootstrap run, including one +// pointing into /root after a run as root. Duplicates define the same pool +// twice and the /root path matches nothing, so PHP-FPM refuses to start, which +// in turn breaks every dpkg operation touching the package. +func TestNormalizeFPMIncludes(t *testing.T) { + const want = "include=/home/jakub/.magebox/php/pools/8.5/*.conf" + + tests := []struct { + name string + content string + wantContent string + wantChanged bool + }{ + { + name: "adds a missing include", + content: "[global]\ninclude=/etc/php/8.5/fpm/pool.d/*.conf\n", + wantContent: "[global]\ninclude=/etc/php/8.5/fpm/pool.d/*.conf\n" + want + "\n", + wantChanged: true, + }, + { + name: "leaves a correct file alone", + content: "[global]\n" + want + "\n", + wantContent: "[global]\n" + want + "\n", + wantChanged: false, + }, + { + name: "collapses duplicates", + content: "[global]\n" + want + "\n" + want + "\n", + wantContent: "[global]\n" + want + "\n", + wantChanged: true, + }, + { + name: "drops an include for another home", + content: "[global]\ninclude=/root/.magebox/php/pools/8.5/*.conf\n" + want + "\n", + wantContent: "[global]\n" + want + "\n", + wantChanged: true, + }, + { + name: "keeps unrelated includes", + content: "[global]\ninclude=/etc/php/8.5/fpm/pool.d/*.conf\ninclude=/root/.magebox/php/pools/8.5/*.conf\n", + wantContent: "[global]\ninclude=/etc/php/8.5/fpm/pool.d/*.conf\n" + want + "\n", + wantChanged: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, changed := NormalizeFPMIncludes(tt.content, want) + if got != tt.wantContent { + t.Errorf("content =\n%q\nwant\n%q", got, tt.wantContent) + } + if changed != tt.wantChanged { + t.Errorf("changed = %v, want %v", changed, tt.wantChanged) + } + }) + } +} diff --git a/internal/bootstrap/installer/phprepo.go b/internal/bootstrap/installer/phprepo.go new file mode 100644 index 0000000..66a7952 --- /dev/null +++ b/internal/bootstrap/installer/phprepo.go @@ -0,0 +1,117 @@ +package installer + +import ( + "fmt" + "net/http" + "os" + "time" +) + +// PHPRepoKind identifies where PHP packages come from. +type PHPRepoKind int + +const ( + // PHPRepoNone means no third-party repository covers this release, so only + // the PHP version shipped by the distribution is available. + PHPRepoNone PHPRepoKind = iota + // PHPRepoPPA is Ondrej Sury's Launchpad PPA. + PHPRepoPPA + // PHPRepoSury is packages.sury.org, which the PPA is being merged into and + // which covers releases the PPA does not. + PHPRepoSury +) + +// PHPRepository is the repository chosen for a release. +type PHPRepository struct { + Kind PHPRepoKind + Suite string +} + +const ( + ondrejPPADistsURL = "https://ppa.launchpadcontent.net/ondrej/php/ubuntu/dists/" + suryDistsURL = "https://packages.sury.org/php/dists/" + // surySourceFile is where MageBox writes the sury apt source. + surySourceFile = "/etc/apt/sources.list.d/magebox-php.list" + // suryKeyring is installed by sury's own keyring package. + suryKeyring = "/usr/share/keyrings/deb.sury.org-php.gpg" +) + +// PickPHPRepository chooses where PHP packages should come from. +// +// The PPA is preferred while it covers the running release. It stops at Ubuntu +// 24.04 and is being folded into packages.sury.org, which does publish for +// newer releases. Pinning the PPA's older suite instead is worse than useless: +// those packages depend on library versions the newer release no longer has, +// so every install fails on unsatisfiable dependencies. +func PickPHPRepository(codename string, ppaPublished, suryPublished func(string) bool) PHPRepository { + if codename == "" { + return PHPRepository{Kind: PHPRepoNone} + } + if ppaPublished(codename) { + return PHPRepository{Kind: PHPRepoPPA, Suite: codename} + } + if suryPublished(codename) { + return PHPRepository{Kind: PHPRepoSury, Suite: codename} + } + return PHPRepository{Kind: PHPRepoNone} +} + +// ShouldRefuseRootBootstrap reports whether bootstrap must stop. +// +// Run as root, bootstrap builds the environment under /root: the config, +// certificates, PHP-FPM pools and an nginx include pointing into a directory +// nginx cannot read, which then stops nginx from starting at all. +func ShouldRefuseRootBootstrap(uid int) bool { + return uid == 0 +} + +// suiteAvailable reports whether a repository publishes a Release file. +func suiteAvailable(base, suite string) bool { + client := &http.Client{Timeout: 10 * time.Second} + resp, err := client.Head(base + suite + "/Release") + if err != nil { + return false + } + defer func() { _ = resp.Body.Close() }() + return resp.StatusCode == http.StatusOK +} + +// PPASuitePublished reports whether Ondrej's PPA covers a release. +func PPASuitePublished(suite string) bool { return suiteAvailable(ondrejPPADistsURL, suite) } + +// SurySuitePublished reports whether packages.sury.org covers a release. +func SurySuitePublished(suite string) bool { return suiteAvailable(suryDistsURL, suite) } + +// configureSuryRepository installs sury's keyring and apt source, and removes +// the PPA source so apt cannot pull packages built for an older release. +func (u *UbuntuInstaller) configureSuryRepository(codename string) error { + keyringDeb := "/tmp/magebox-sury-keyring.deb" + if err := u.RunCommand(fmt.Sprintf("curl -fsSL -o %s https://packages.sury.org/debsuryorg-archive-keyring.deb", keyringDeb)); err != nil { + return fmt.Errorf("failed to download the sury keyring: %w", err) + } + defer func() { _ = os.Remove(keyringDeb) }() + + if err := u.RunSudo("dpkg", "-i", keyringDeb); err != nil { + return fmt.Errorf("failed to install the sury keyring: %w", err) + } + + source := fmt.Sprintf("deb [signed-by=%s] https://packages.sury.org/php/ %s main\n", suryKeyring, codename) + if err := u.WriteFile(surySourceFile, source); err != nil { + return fmt.Errorf("failed to write %s: %w", surySourceFile, err) + } + + // apt run as the user must be able to read this, or every package in the + // repository looks unavailable. An older MageBox left it root-only. + if err := u.RunSudo("chmod", "0644", surySourceFile); err != nil { + return fmt.Errorf("failed to set permissions on %s: %w", surySourceFile, err) + } + + // A PPA source for a release it does not publish only produces 404s, and a + // PPA pinned to an older suite offers packages that cannot be installed. + for _, file := range ppaSourceFiles(codename) { + if u.FileExists(file) { + _ = u.RunSudo("rm", "-f", file) + } + } + return nil +} diff --git a/internal/bootstrap/installer/phprepo_test.go b/internal/bootstrap/installer/phprepo_test.go new file mode 100644 index 0000000..6421fe4 --- /dev/null +++ b/internal/bootstrap/installer/phprepo_test.go @@ -0,0 +1,84 @@ +package installer + +import "testing" + +func publishing(suites ...string) func(string) bool { + set := make(map[string]bool, len(suites)) + for _, s := range suites { + set[s] = true + } + return func(suite string) bool { return set[suite] } +} + +// Ondrej's PPA stops at 24.04 and is being folded into packages.sury.org, which +// does publish for 26.04. Pinning the PPA's older suite instead produces +// packages whose dependencies (libxml2, libicu74, libzip4t64) cannot be +// satisfied on the newer release, so PHP still fails to install. +func TestPickPHPRepository(t *testing.T) { + tests := []struct { + name string + codename string + ppa func(string) bool + sury func(string) bool + want PHPRepository + }{ + { + name: "PPA covers this release", + codename: "noble", + ppa: publishing("noble"), + sury: publishing("noble"), + want: PHPRepository{Kind: PHPRepoPPA, Suite: "noble"}, + }, + { + name: "only sury covers this release", + codename: "resolute", + ppa: publishing("noble"), + sury: publishing("resolute", "noble"), + want: PHPRepository{Kind: PHPRepoSury, Suite: "resolute"}, + }, + { + name: "neither covers it", + codename: "vivacious", + ppa: publishing("noble"), + sury: publishing("noble"), + want: PHPRepository{Kind: PHPRepoNone}, + }, + { + name: "unknown codename", + codename: "", + ppa: publishing("noble"), + sury: publishing("noble"), + want: PHPRepository{Kind: PHPRepoNone}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := PickPHPRepository(tt.codename, tt.ppa, tt.sury) + if got != tt.want { + t.Errorf("PickPHPRepository(%q) = %+v, want %+v", tt.codename, got, tt.want) + } + }) + } +} + +// Running bootstrap as root writes the whole environment into /root: config, +// certificates, nginx includes and pool directories nginx cannot even read. +func TestShouldRefuseRootBootstrap(t *testing.T) { + tests := []struct { + name string + uid int + want bool + }{ + {name: "regular user", uid: 1000, want: false}, + {name: "root", uid: 0, want: true}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := ShouldRefuseRootBootstrap(tt.uid); got != tt.want { + t.Errorf("ShouldRefuseRootBootstrap(%d) = %v, want %v", tt.uid, got, tt.want) + } + }) + } +} diff --git a/internal/bootstrap/installer/ppa.go b/internal/bootstrap/installer/ppa.go index 5cdd358..6e80012 100644 --- a/internal/bootstrap/installer/ppa.go +++ b/internal/bootstrap/installer/ppa.go @@ -2,72 +2,12 @@ package installer import ( "fmt" - "net/http" "os" "strings" - "time" -) - -// ubuntuCodenames lists Ubuntu releases, newest first. It is used to find the -// newest suite a PPA publishes when it has nothing for the running release. -var ubuntuCodenames = []string{ - "resolute", // 26.04 LTS - "questing", // 25.10 - "plucky", // 25.04 - "oracular", // 24.10 - "noble", // 24.04 LTS - "mantic", // 23.10 - "lunar", // 23.04 - "kinetic", // 22.10 - "jammy", // 22.04 LTS - "focal", // 20.04 LTS -} - -// ondrejPPADists is where the PHP PPA publishes its suites. -const ondrejPPADists = "https://ppa.launchpadcontent.net/ondrej/php/ubuntu/dists/" - -// pickPPASuite returns the suite to configure and whether it is a fallback. -// -// Ondrej's PHP PPA lags new Ubuntu releases by months, and on a release it does -// not cover there is no php8.1 … php8.4 at all. Pinning the newest published -// suite keeps those versions installable; packages are built against an older -// but compatible Ubuntu. -func pickPPASuite(current string, published func(string) bool) (suite string, fallback bool) { - if published(current) { - return current, false - } - - start := 0 - for i, codename := range ubuntuCodenames { - if codename == current { - start = i + 1 - break - } - } - - for _, codename := range ubuntuCodenames[start:] { - if codename == current { - continue - } - if published(codename) { - return codename, true - } - } - - // Nothing reachable (offline, or the PPA moved): leave the release as is. - return current, false -} -// ppaSuitePublished reports whether the PHP PPA has a Release file for a suite. -func ppaSuitePublished(suite string) bool { - client := &http.Client{Timeout: 10 * time.Second} - resp, err := client.Head(ondrejPPADists + suite + "/Release") - if err != nil { - return false - } - defer func() { _ = resp.Body.Close() }() - return resp.StatusCode == http.StatusOK -} + "qoliber/magebox/internal/cli" + "qoliber/magebox/internal/verbose" +) // currentUbuntuCodename reads the running release's codename, "" if unknown. func currentUbuntuCodename() string { @@ -92,41 +32,24 @@ func ppaSourceFiles(codename string) []string { } } -// pinPPASuite rewrites the suite in the PPA source file, leaving the signing -// key and every other line untouched. -func (u *UbuntuInstaller) pinPPASuite(from, to string) error { - for _, file := range ppaSourceFiles(from) { - if !u.FileExists(file) { - continue - } - // deb822 keeps the suite on its own "Suites:" line; the older one-line - // format has it between the URI and the component. - expression := fmt.Sprintf("/^Suites:/s/%s/%s/; /^deb /s/ %s / %s /", from, to, from, to) - if err := u.RunSudo("sed", "-i", "-e", expression, file); err != nil { - return fmt.Errorf("failed to pin the PHP PPA to %s in %s: %w", to, file, err) - } - return nil - } - return fmt.Errorf("could not find the PHP PPA source file for %s", from) -} - -// configurePHPRepository adds Ondrej's PHP PPA, falling back to the newest -// suite it publishes when the running release is not covered yet. +// configurePHPRepository points apt at a PHP repository that covers this +// release, so php8.1 … php8.4 can actually be installed. func (u *UbuntuInstaller) configurePHPRepository() error { - if err := u.RunCommand("sudo add-apt-repository -y ppa:ondrej/php"); err != nil { - return fmt.Errorf("failed to add Ondrej PPA: %w", err) - } - codename := currentUbuntuCodename() - if codename == "" { - return nil - } + repo := PickPHPRepository(codename, PPASuitePublished, SurySuitePublished) - suite, fallback := pickPPASuite(codename, ppaSuitePublished) - if !fallback { - return nil + switch repo.Kind { + case PHPRepoPPA: + if err := u.RunCommand("sudo add-apt-repository -y ppa:ondrej/php"); err != nil { + verbose.Debug("add-apt-repository reported an error: %v", err) + } + case PHPRepoSury: + fmt.Printf(" The PHP PPA does not cover %s; using packages.sury.org instead.\n", codename) + if err := u.configureSuryRepository(repo.Suite); err != nil { + return err + } + default: + cli.PrintWarning("No PHP repository covers %s yet; only the PHP version shipped by Ubuntu can be installed.", codename) } - - fmt.Printf(" The PHP PPA does not publish packages for %s yet; using its %s packages instead.\n", codename, suite) - return u.pinPPASuite(codename, suite) + return nil } diff --git a/internal/bootstrap/installer/ppa_test.go b/internal/bootstrap/installer/ppa_test.go deleted file mode 100644 index a0cab7a..0000000 --- a/internal/bootstrap/installer/ppa_test.go +++ /dev/null @@ -1,83 +0,0 @@ -package installer - -import "testing" - -// Ondrej's PHP PPA publishes nothing for Ubuntu 26.04, so bootstrap must pin -// the newest suite it does publish. Without this, PHP 8.1 through 8.4 simply do -// not exist in apt and only Ubuntu's own PHP can be installed. -func TestPickPPASuite(t *testing.T) { - published := func(suites ...string) func(string) bool { - set := make(map[string]bool, len(suites)) - for _, s := range suites { - set[s] = true - } - return func(suite string) bool { return set[suite] } - } - - tests := []struct { - name string - current string - published func(string) bool - wantSuite string - wantFallback bool - }{ - { - name: "current release is published", - current: "noble", - published: published("noble", "jammy"), - wantSuite: "noble", - }, - { - name: "falls back to the newest published release", - current: "resolute", - published: published("noble", "jammy"), - wantSuite: "noble", - wantFallback: true, - }, - { - name: "skips unpublished releases in between", - current: "questing", - published: published("jammy"), - wantSuite: "jammy", - wantFallback: true, - }, - { - name: "unknown newer codename starts at the top of the list", - current: "vivacious", - published: published("noble"), - wantSuite: "noble", - wantFallback: true, - }, - { - name: "nothing published keeps the current release", - current: "resolute", - published: published(), - wantSuite: "resolute", - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - suite, fallback := pickPPASuite(tt.current, tt.published) - if suite != tt.wantSuite || fallback != tt.wantFallback { - t.Errorf("pickPPASuite(%q) = (%q, %v), want (%q, %v)", tt.current, suite, fallback, tt.wantSuite, tt.wantFallback) - } - }) - } -} - -func TestUbuntuCodenamesAreOrderedNewestFirst(t *testing.T) { - if len(ubuntuCodenames) < 5 { - t.Fatalf("expected a meaningful list of codenames, got %d", len(ubuntuCodenames)) - } - if ubuntuCodenames[len(ubuntuCodenames)-1] != "focal" { - t.Errorf("oldest entry = %q, want focal (20.04, the oldest supported release)", ubuntuCodenames[len(ubuntuCodenames)-1]) - } - seen := make(map[string]bool) - for _, c := range ubuntuCodenames { - if seen[c] { - t.Errorf("duplicate codename %q", c) - } - seen[c] = true - } -} diff --git a/internal/bootstrap/installer/ubuntu.go b/internal/bootstrap/installer/ubuntu.go index a162bef..dc54caf 100644 --- a/internal/bootstrap/installer/ubuntu.go +++ b/internal/bootstrap/installer/ubuntu.go @@ -4,14 +4,17 @@ package installer import ( + "bytes" "fmt" "os" "os/exec" "path/filepath" "strings" + "qoliber/magebox/internal/cli" "qoliber/magebox/internal/config" "qoliber/magebox/internal/platform" + "qoliber/magebox/internal/verbose" ) // UbuntuInstaller handles installation on Ubuntu/Debian @@ -24,13 +27,31 @@ type UbuntuInstaller struct { // (e.g. php8.5-opcache is built into php8.5-cli) or otherwise aren't yet // published for a given PHP version. func isAptPackageAvailable(pkg string) bool { - out, err := exec.Command("apt-cache", "show", pkg).Output() + cmd := exec.Command("apt-cache", "show", pkg) + var stderr bytes.Buffer + cmd.Stderr = &stderr + out, err := cmd.Output() + + // apt-cache cannot tell a genuinely missing package from one it could not + // look up, so a source file it may not read produces the same empty answer. + // Filtering packages out on that basis made bootstrap skip every PHP + // package and report success. Let apt decide instead: a real install either + // works or fails with a message worth reading. + if !aptCacheAnswerReliable(stderr.String()) { + return true + } if err != nil { return false } return len(strings.TrimSpace(string(out))) > 0 } +// aptCacheAnswerReliable reports whether apt-cache could see everything it +// needed, judged by what it warned about. +func aptCacheAnswerReliable(stderr string) bool { + return !strings.Contains(stderr, "Permission denied") +} + // NewUbuntuInstaller creates a new Ubuntu/Debian installer func NewUbuntuInstaller(p *platform.Platform) *UbuntuInstaller { return &UbuntuInstaller{ @@ -98,14 +119,22 @@ func (u *UbuntuInstaller) ValidateOSVersion() (OSVersionInfo, error) { // InstallPrerequisites installs system prerequisites func (u *UbuntuInstaller) InstallPrerequisites() error { - // Update package lists + // A repository with nothing for this release makes apt update exit + // non-zero. That must not end bootstrap, because the repository it would + // abort over is usually the one the next step repairs. if err := u.RunSudo("apt", "update"); err != nil { - return fmt.Errorf("failed to update apt: %w", err) + verbose.Debug("apt update reported errors before the PHP repository was configured: %v", err) } - // Install basic tools + // A package left half-configured by an earlier failure makes every apt + // install exit 100, whatever it is asked for. + u.repairBrokenPackages() + + // Install basic tools. A failure here must not end bootstrap: it is usually + // a broken package elsewhere on the system, and stopping would skip the + // repository configuration below, which is what makes PHP installable. if err := u.RunSudo("apt", "install", "-y", "curl", "git", "unzip", "software-properties-common"); err != nil { - return err + cli.PrintWarning("Could not install base tools: %v", err) } // Add Ondrej PPA for PHP, pinned to a suite it actually publishes @@ -113,8 +142,12 @@ func (u *UbuntuInstaller) InstallPrerequisites() error { return err } - // Update after adding PPA - return u.RunSudo("apt", "update") + // Update again so the repaired repository is read. Other broken sources on + // the machine are the user's to fix; they must not fail bootstrap. + if err := u.RunSudo("apt", "update"); err != nil { + cli.PrintWarning("apt update reported errors; packages from a failing repository may be unavailable") + } + return nil } // InstallPHP installs a specific PHP version via Ondrej PPA @@ -310,14 +343,16 @@ func (u *UbuntuInstaller) ConfigurePHPFPM(versions []string) error { } } - // Add MageBox pools include to php-fpm.conf if not already present + // Keep exactly one MageBox include. Appending on every run left + // duplicates, which define the same pool twice, and a run under a + // different user added an include matching nothing. Either stops + // PHP-FPM from starting. if u.FileExists(fpmConfPath) { - // Check if include already exists - checkCmd := exec.Command("grep", "-q", mageboxPoolsInclude, fpmConfPath) - if checkCmd.Run() != nil { - // Include not found, add it - if err := u.RunSudo("sh", "-c", fmt.Sprintf("echo '%s' >> %s", mageboxPoolsInclude, fpmConfPath)); err != nil { - return fmt.Errorf("failed to add MageBox pools include to %s: %w", fpmConfPath, err) + if content, readErr := os.ReadFile(fpmConfPath); readErr == nil { + if updated, changed := NormalizeFPMIncludes(string(content), mageboxPoolsInclude); changed { + if err := u.WriteFile(fpmConfPath, updated); err != nil { + return fmt.Errorf("failed to update %s: %w", fpmConfPath, err) + } } } } diff --git a/internal/dns/dnsmasq.go b/internal/dns/dnsmasq.go index 77199ae..0f4b2bb 100644 --- a/internal/dns/dnsmasq.go +++ b/internal/dns/dnsmasq.go @@ -4,6 +4,7 @@ import ( "bytes" _ "embed" "fmt" + "net" "os" "os/exec" "path/filepath" @@ -76,9 +77,35 @@ func (m *DnsmasqManager) getTLD() string { return globalCfg.GetTLD() } -// IsInstalled checks if dnsmasq is installed +// IsInstalled checks if dnsmasq is installed and usable by MageBox. +// +// On Linux the binary alone is not enough: Ubuntu's dnsmasq-base package ships +// it without a service unit, and MageBox starts dnsmasq through systemd. +// Treating that as installed makes bootstrap skip the install and then fail to +// start the service. func (m *DnsmasqManager) IsInstalled() bool { - return platform.CommandExists("dnsmasq") + if !platform.CommandExists("dnsmasq") { + return false + } + if m.platform.Type != platform.Linux || !platform.CommandExists("systemctl") { + return true + } + + output, err := exec.Command("systemctl", "list-unit-files", "dnsmasq.service").Output() + if err != nil { + return false + } + return systemdUnitListed(string(output), "dnsmasq.service") +} + +// systemdUnitListed reports whether systemctl listed the unit. +func systemdUnitListed(output, unit string) bool { + for _, line := range strings.Split(output, "\n") { + if strings.HasPrefix(strings.TrimSpace(line), unit) { + return true + } + } + return false } // IsConfigured checks if dnsmasq is configured for MageBox @@ -468,7 +495,65 @@ func (m *DnsmasqManager) setupSystemdResolved() error { return fmt.Errorf("failed to write resolved config: %w", err) } + // systemd-resolved reads its configuration after dropping privileges, so a + // root-only file (the mode a temp file is copied with) is refused with + // "Permission denied" and silently ignored. + if err := exec.Command("sudo", "chmod", "0644", confPath).Run(); err != nil { + return fmt.Errorf("failed to set permissions on resolved config: %w", err) + } + // Restart systemd-resolved cmd = exec.Command("sudo", "systemctl", "restart", "systemd-resolved") return cmd.Run() } + +// ResolvedDropInPath is the systemd-resolved drop-in MageBox installs to send +// .test lookups to its dnsmasq. +const ResolvedDropInPath = "/etc/systemd/resolved.conf.d/magebox.conf" + +// NeedsResolvedCleanup reports whether the systemd-resolved drop-in has to go. +// +// The drop-in is written before dnsmasq is known to work. If dnsmasq then +// cannot be started, leaving it behind points every .test lookup at a resolver +// that is not there, so names fail outright instead of falling through to +// /etc/hosts. +func NeedsResolvedCleanup(dnsmasqWorking, dropInPresent bool) bool { + return !dnsmasqWorking && dropInPresent +} + +// ResolvedDropInPresent reports whether the drop-in is installed. +func ResolvedDropInPresent() bool { + _, err := os.Stat(ResolvedDropInPath) + return err == nil +} + +// RemoveSystemdResolvedConfig deletes the drop-in and restarts +// systemd-resolved, handing .test lookups back to the system defaults. +func (m *DnsmasqManager) RemoveSystemdResolvedConfig() error { + if !ResolvedDropInPresent() { + return nil + } + if err := exec.Command("sudo", "rm", "-f", ResolvedDropInPath).Run(); err != nil { + return fmt.Errorf("failed to remove %s: %w", ResolvedDropInPath, err) + } + if err := exec.Command("sudo", "systemctl", "restart", "systemd-resolved").Run(); err != nil { + return fmt.Errorf("failed to restart systemd-resolved: %w", err) + } + return nil +} + +// SystemResolves reports whether the machine's own resolver answers for a +// domain. +// +// Querying dnsmasq directly proves only that dnsmasq works. Everything else on +// the machine goes through the system resolver, so bootstrap reported DNS as +// working while browsers and curl still failed. +func SystemResolves(domain string) bool { + if platform.CommandExists("getent") { + // getent goes through NSS, the same path ordinary programs take. + output, err := exec.Command("getent", "hosts", domain).Output() + return err == nil && len(strings.TrimSpace(string(output))) > 0 + } + addrs, err := net.LookupHost(domain) + return err == nil && len(addrs) > 0 +} diff --git a/internal/dns/dnsmasq_test.go b/internal/dns/dnsmasq_test.go index 81d3477..972abff 100644 --- a/internal/dns/dnsmasq_test.go +++ b/internal/dns/dnsmasq_test.go @@ -152,3 +152,75 @@ func TestDnsmasqStatus(t *testing.T) { t.Errorf("TestDomain = %v, want test.test", status.TestDomain) } } + +// Ubuntu's dnsmasq-base package ships the binary without a service unit. +// MageBox starts dnsmasq through systemd, so the binary alone is not enough: +// without this check bootstrap reports dnsmasq as installed and then fails to +// start it with a bare exit code. +func TestSystemdUnitListed(t *testing.T) { + tests := []struct { + name string + output string + unit string + want bool + }{ + { + name: "unit present", + output: "UNIT FILE STATE PRESET\ndnsmasq.service enabled enabled\n\n1 unit files listed.", + unit: "dnsmasq.service", + want: true, + }, + { + name: "no unit files", + output: "UNIT FILE STATE PRESET\n\n0 unit files listed.", + unit: "dnsmasq.service", + want: false, + }, + { + name: "different unit listed", + output: "UNIT FILE STATE PRESET\nnginx.service enabled enabled\n\n1 unit files listed.", + unit: "dnsmasq.service", + want: false, + }, + { + name: "empty output", + output: "", + unit: "dnsmasq.service", + want: false, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := systemdUnitListed(tt.output, tt.unit); got != tt.want { + t.Errorf("systemdUnitListed(%q) = %v, want %v", tt.unit, got, tt.want) + } + }) + } +} + +// When dnsmasq cannot be started, MageBox falls back to /etc/hosts. Leaving the +// systemd-resolved drop-in in place then points every .test lookup at a +// resolver that does not exist, so names fail outright instead of falling +// through to the hosts file. +func TestNeedsResolvedCleanup(t *testing.T) { + tests := []struct { + name string + dnsmasqOK bool + dropInFound bool + want bool + }{ + {name: "dnsmasq works, drop-in belongs there", dnsmasqOK: true, dropInFound: true, want: false}, + {name: "fell back with a stale drop-in", dnsmasqOK: false, dropInFound: true, want: true}, + {name: "fell back with nothing to clean", dnsmasqOK: false, dropInFound: false, want: false}, + {name: "dnsmasq works, no drop-in yet", dnsmasqOK: true, dropInFound: false, want: false}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := NeedsResolvedCleanup(tt.dnsmasqOK, tt.dropInFound); got != tt.want { + t.Errorf("NeedsResolvedCleanup(%v, %v) = %v, want %v", tt.dnsmasqOK, tt.dropInFound, got, tt.want) + } + }) + } +} diff --git a/internal/nginx/certs.go b/internal/nginx/certs.go new file mode 100644 index 0000000..015d893 --- /dev/null +++ b/internal/nginx/certs.go @@ -0,0 +1,76 @@ +package nginx + +import ( + "os" + "path/filepath" + "strings" +) + +// SSLCertificatePaths returns the certificate and key files a vhost references. +func SSLCertificatePaths(content string) []string { + var paths []string + for _, line := range strings.Split(content, "\n") { + trimmed := strings.TrimSpace(line) + if strings.HasPrefix(trimmed, "#") { + continue + } + fields := strings.Fields(trimmed) + if len(fields) < 2 { + continue + } + if fields[0] != "ssl_certificate" && fields[0] != "ssl_certificate_key" { + continue + } + paths = append(paths, strings.TrimSuffix(fields[1], ";")) + } + return paths +} + +// DomainFromCertPath recovers the domain from a certificate path, which +// MageBox stores as //cert.pem. +func DomainFromCertPath(path string) string { + if path == "" { + return "" + } + switch filepath.Base(path) { + case "cert.pem", "key.pem": + default: + // Managed by something other than MageBox: not ours to regenerate. + return "" + } + dir := filepath.Base(filepath.Dir(path)) + if dir == "." || dir == string(filepath.Separator) { + return "" + } + return dir +} + +// MissingCertificates reports the certificate files referenced by the vhosts in +// dir that do not exist, keyed by the vhost that references them. +// +// nginx refuses to start when a single certificate is missing, so one stale +// vhost takes every project on the machine offline. +func MissingCertificates(vhostsDir string) (map[string][]string, error) { + entries, err := os.ReadDir(vhostsDir) + if err != nil { + return nil, err + } + + missing := make(map[string][]string) + for _, entry := range entries { + if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".conf") { + continue + } + path := filepath.Join(vhostsDir, entry.Name()) + content, err := os.ReadFile(path) + if err != nil { + continue + } + for _, cert := range SSLCertificatePaths(string(content)) { + if _, err := os.Stat(cert); err != nil { + missing[path] = append(missing[path], cert) + } + } + } + return missing, nil +} diff --git a/internal/nginx/certs_test.go b/internal/nginx/certs_test.go new file mode 100644 index 0000000..0dd0480 --- /dev/null +++ b/internal/nginx/certs_test.go @@ -0,0 +1,121 @@ +package nginx + +import ( + "os" + "path/filepath" + "testing" +) + +func TestSSLCertificatePaths(t *testing.T) { + tests := []struct { + name string + content string + want []string + }{ + { + name: "certificate and key", + content: `server { + listen 443 ssl; + ssl_certificate /home/jakub/.magebox/certs/shop.test/cert.pem; + ssl_certificate_key /home/jakub/.magebox/certs/shop.test/key.pem; +}`, + want: []string{"/home/jakub/.magebox/certs/shop.test/cert.pem", "/home/jakub/.magebox/certs/shop.test/key.pem"}, + }, + { + name: "commented lines are ignored", + content: " # ssl_certificate /old/cert.pem;\n ssl_certificate /new/cert.pem;", + want: []string{"/new/cert.pem"}, + }, + { + name: "directives that merely start the same are ignored", + content: " ssl_certificate_by_lua_file /etc/nginx/hook.lua;", + want: nil, + }, + { + name: "no ssl at all", + content: "server {\n listen 80;\n}", + want: nil, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := SSLCertificatePaths(tt.content) + if len(got) != len(tt.want) { + t.Fatalf("got %d paths %v, want %d %v", len(got), got, len(tt.want), tt.want) + } + for i := range tt.want { + if got[i] != tt.want[i] { + t.Errorf("path %d = %q, want %q", i, got[i], tt.want[i]) + } + } + }) + } +} + +// Only MageBox's own layout can be regenerated; a certificate managed +// elsewhere must be left alone and reported instead. +func TestDomainFromCertPath(t *testing.T) { + tests := []struct { + path string + want string + }{ + {path: "/home/jakub/.magebox/certs/shop.test/cert.pem", want: "shop.test"}, + {path: "/home/jakub/.magebox/certs/shop.test/key.pem", want: "shop.test"}, + {path: "/etc/ssl/other.pem", want: ""}, + {path: "/home/jakub/.magebox/certs/shop.test/fullchain.pem", want: ""}, + {path: "", want: ""}, + } + + for _, tt := range tests { + t.Run(tt.path, func(t *testing.T) { + if got := DomainFromCertPath(tt.path); got != tt.want { + t.Errorf("DomainFromCertPath(%q) = %q, want %q", tt.path, got, tt.want) + } + }) + } +} + +// One vhost pointing at a certificate that is not there stops nginx from +// starting at all, which takes every other project down with it. +func TestMissingCertificates(t *testing.T) { + dir := t.TempDir() + certDir := filepath.Join(dir, "certs", "present.test") + if err := os.MkdirAll(certDir, 0755); err != nil { + t.Fatal(err) + } + existing := filepath.Join(certDir, "cert.pem") + if err := os.WriteFile(existing, []byte("x"), 0644); err != nil { + t.Fatal(err) + } + + vhosts := filepath.Join(dir, "vhosts") + if err := os.MkdirAll(vhosts, 0755); err != nil { + t.Fatal(err) + } + gone := filepath.Join(dir, "certs", "gone.test", "cert.pem") + write := func(name, content string) { + if err := os.WriteFile(filepath.Join(vhosts, name), []byte(content), 0644); err != nil { + t.Fatal(err) + } + } + write("ok.conf", "ssl_certificate "+existing+";") + write("broken.conf", "ssl_certificate "+gone+";") + write("plain.conf", "listen 80;") + write("notes.txt", "ssl_certificate /ignored/cert.pem;") + + missing, err := MissingCertificates(vhosts) + if err != nil { + t.Fatalf("MissingCertificates failed: %v", err) + } + if len(missing) != 1 { + t.Fatalf("expected one broken vhost, got %d: %v", len(missing), missing) + } + paths, ok := missing[filepath.Join(vhosts, "broken.conf")] + if !ok { + t.Fatalf("broken.conf not reported: %v", missing) + } + if len(paths) != 1 || paths[0] != gone { + t.Errorf("reported %v, want [%s]", paths, gone) + } +} diff --git a/internal/ssl/catrust.go b/internal/ssl/catrust.go new file mode 100644 index 0000000..8555056 --- /dev/null +++ b/internal/ssl/catrust.go @@ -0,0 +1,88 @@ +package ssl + +import ( + "crypto/x509" + "encoding/pem" + "os" + "os/exec" + "path/filepath" + "strings" + + "qoliber/magebox/internal/platform" +) + +// nssListsCASerial reports whether a certutil listing trusts the CA with this +// serial. +// +// mkcert names its authority "mkcert development CA ", so +// the nickname identifies exactly which CA a browser trusts. A machine restored +// from another install often trusts a stale one, which no file check catches. +func nssListsCASerial(certutilOutput, serialDecimal string) bool { + if serialDecimal == "" { + return false + } + for _, line := range strings.Split(certutilOutput, "\n") { + if strings.Contains(line, serialDecimal) { + return true + } + } + return false +} + +// caSerialDecimal returns the serial of the local CA, as mkcert writes it into +// the trust store nickname. +func (m *Manager) caSerialDecimal() string { + caRoot, err := m.getCARoot() + if err != nil { + return "" + } + content, err := os.ReadFile(filepath.Join(caRoot, "rootCA.pem")) + if err != nil { + return "" + } + block, _ := pem.Decode(content) + if block == nil { + return "" + } + cert, err := x509.ParseCertificate(block.Bytes) + if err != nil { + return "" + } + return cert.SerialNumber.String() +} + +// nssDatabases are the browser trust stores mkcert writes to. +func nssDatabases() []string { + home, err := os.UserHomeDir() + if err != nil { + return nil + } + return []string{filepath.Join(home, ".pki", "nssdb")} +} + +// IsCATrustedByBrowsers reports whether the browser trust store holds the +// current CA. It answers true when it cannot tell, so a missing certutil or an +// absent store never triggers a pointless reinstall. +func (m *Manager) IsCATrustedByBrowsers() bool { + if !platform.CommandExists("certutil") { + return true + } + serial := m.caSerialDecimal() + if serial == "" { + return true + } + + for _, db := range nssDatabases() { + if _, err := os.Stat(db); err != nil { + continue + } + output, err := exec.Command("certutil", "-L", "-d", "sql:"+db).Output() + if err != nil { + continue + } + if !nssListsCASerial(string(output), serial) { + return false + } + } + return true +} diff --git a/internal/ssl/catrust_test.go b/internal/ssl/catrust_test.go new file mode 100644 index 0000000..58e87fc --- /dev/null +++ b/internal/ssl/catrust_test.go @@ -0,0 +1,34 @@ +package ssl + +import "testing" + +// mkcert names its authority "mkcert development CA ", so a +// listing from the browser trust store shows which authority is trusted. A +// machine carried over from another install trusts a stale one, and MageBox +// reported the CA as installed because the file existed on disk, while every +// HTTPS site showed a warning. +func TestNSSListsCASerial(t *testing.T) { + const listing = `mkcert development CA 244346547749281684915500604760826213949 C,, +Some Other CA ,, +` + + tests := []struct { + name string + output string + serial string + want bool + }{ + {name: "serial is trusted", output: listing, serial: "244346547749281684915500604760826213949", want: true}, + {name: "a different mkcert CA is trusted", output: listing, serial: "271385820751692510796476016112517704270", want: false}, + {name: "empty store", output: "", serial: "244346547749281684915500604760826213949", want: false}, + {name: "no serial known", output: listing, serial: "", want: false}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := nssListsCASerial(tt.output, tt.serial); got != tt.want { + t.Errorf("nssListsCASerial(serial %q) = %v, want %v", tt.serial, got, tt.want) + } + }) + } +} diff --git a/internal/ssl/mkcert.go b/internal/ssl/mkcert.go index a6662ba..81073fc 100644 --- a/internal/ssl/mkcert.go +++ b/internal/ssl/mkcert.go @@ -66,9 +66,13 @@ func (m *Manager) EnsureCAInstalled() error { return fmt.Errorf("failed to get CA root: %w", err) } - // Check if CA files exist - if _, err := os.Stat(filepath.Join(caRoot, "rootCA.pem")); os.IsNotExist(err) { - // Install the CA + // A CA file on disk proves nothing about trust. On a machine restored from + // another install the browser often trusts a different, stale authority, so + // every HTTPS site warns while MageBox reports the CA as installed. + _, statErr := os.Stat(filepath.Join(caRoot, "rootCA.pem")) + if os.IsNotExist(statErr) || !m.IsCATrustedByBrowsers() { + // mkcert -install is idempotent and adds the CA to the system and + // browser trust stores. cmd := exec.Command("mkcert", "-install") cmd.Stdout = os.Stdout cmd.Stderr = os.Stderr diff --git a/vitepress/guide/bootstrap.md b/vitepress/guide/bootstrap.md index c0e158a..bdd59fc 100644 --- a/vitepress/guide/bootstrap.md +++ b/vitepress/guide/bootstrap.md @@ -217,17 +217,32 @@ curl -I https://mystore.test On Linux, MageBox uses a different approach than macOS for privileged ports and service management. +### Repairs on Every Run + +`magebox bootstrap` is safe to re-run and repairs state it finds broken: + +- **Sudoers rules** are regenerated whenever the installed file is missing, rejected by the local sudo, or out of date. An existing file is never taken as proof of a working one. +- **Vhost certificates** are checked against disk. nginx refuses to start when a single certificate is missing, which takes every project offline, so bootstrap regenerates the ones MageBox manages and names the file to remove for any it does not. +- **The PHP repository** is pointed at a source that covers this release, as described below, and its apt source file is made readable so packages do not silently look unavailable. +- **Half-configured packages** are finished with `dpkg --configure -a`. One of them makes every `apt install` fail, whatever it is asked for. + +Errors from `apt update`, and failures installing the base tools, are reported as warnings rather than ending the run, because the thing that failed is usually what the next step repairs. + ### PHP Packages on Ubuntu and Debian -PHP comes from Ondrej Sury's PPA, which lags new Ubuntu releases by months. On a release it does not cover yet, such as 26.04, that PPA holds no packages at all, so PHP 8.1 through 8.4 cannot be installed and only the PHP version shipped by Ubuntu itself is available. +PHP comes from a third-party repository, because Ubuntu ships a single version. Bootstrap picks the one that covers the running release: -Bootstrap checks which suites the PPA publishes and pins the newest one it has, reporting what it did: +- **Ondrej Sury's PPA**, while it publishes for that release. +- **packages.sury.org**, for releases the PPA has not caught up with. The PPA is being merged into it, and it is the canonical source for Ubuntu 26.04. +- **Neither**, with a warning that only the PHP version shipped by Ubuntu can be installed. ``` -The PHP PPA does not publish packages for resolute yet; using its noble packages instead. +The PHP PPA does not cover resolute; using packages.sury.org instead. ``` -Those packages are built for the previous release. They normally install and run fine, but if a dependency has moved on, bootstrap reports the failure for that PHP version and continues with the rest. +::: warning Do not pin an older suite +Pointing the PPA at the newest suite it does publish looks like a fix and is not: those packages depend on library versions (`libxml2`, `libicu74`, `libzip4t64`) a newer Ubuntu no longer ships, so every install fails on unsatisfiable dependencies. Bootstrap therefore switches repository rather than suite. +::: ### Nginx User Configuration