Vacuum Stability of a Complex Singlet
There are many ways to skin a cat, and many ways to derive constraints for tree-level vacuum stability of a scalar potential. It is sufficient to consider only the quartic part of the potential, since mass and cubic terms are small in the limit of large field values. The simplest non-trivial quartic potential that can serve as an example is that of a complex scalar singlet $S$ with no explicit CP violation – all the couplings are real: \begin{equation} V = \lambda_{S} |S|^{4} + \frac{\lambda'_{S}}{2} (S^{4} + S^{\dagger 4}) + \frac{\lambda''_{S}}{2} |S|^{2} (S^{2} + S^{\dagger 2}). \label{V:complex} \end{equation} We can alternatively write $S$ in terms of its Cartesian or polar components, \begin{equation} S = \frac{S_{R} + i S_{I}}{\sqrt{2}} = s e^{i \phi_{S}}, \end{equation} in which the potential takes the form \begin{equation} V = \frac{1}{4} \left[ (\lambda_{S} + \lambda'_{S} + \lambda''_{S}) S_{R}^{4} + 2 (\lambda_{S} - 3 \lambda'_{S}) S_{R}^{2} S_{I}^{2} + (\lambda_{S} + \lambda'_{S} - \lambda''_{S}) S_{I}^{4} \right] \label{V:Cartesian} \end{equation} or \begin{equation} V = \left( \lambda_{S} + \lambda'_{S} \cos 4 \phi_{S} + \lambda''_{S} \cos 2 \phi_{S} \right) s^{4}. \label{V:polar} \end{equation}
From \eqref{V:Cartesian}, we can write out the matrix of quartic couplings in the $(S_{R}^{2}, S_{I}^{2})$ basis: \begin{equation} \Lambda = \frac{1}{4} \begin{pmatrix} \lambda_{S} + \lambda'_{S} + \lambda''_{S} & \lambda_{S} - 3 \lambda'_{S} \\ \lambda_{S} - 3 \lambda'_{S} & \lambda_{S} + \lambda'_{S} - \lambda''_{S} \end{pmatrix}. \label{eq:Lambda:Cartesian} \end{equation} Because $S_{R}^{2}$ and $S_{I}^{2}$ are non-negative, the matrix $\Lambda$ has to be co-positive for the potential to be bounded below. Obviously, the diagonal terms have to be positive because we can set separately either $S_{R}^{2}$ or $S_{I}^{2}$ to zero. In addition, the non-diagonal term cannot be too negative. Altogether, the vacuum stability conditions are \begin{align} \lambda_{S} + \lambda'_{S} + \lambda''_{S} & \geqslant 0, \label{eq:vac:stab:SR} \\ \lambda_{S} + \lambda'_{S} - \lambda''_{S} & \geqslant 0, \label{eq:vac:stab:SI} \\ \lambda_{S} - 3 \lambda'_{S} + \sqrt{ (\lambda_{S} + \lambda'_{S})^{2} - \lambda_{S}^{\prime\prime 2} } &\geqslant 0. \label{eq:vac:stab:Cartesian:last} \end{align}
Equivalently, the coefficient of $s^{4}$ in \eqref{V:polar} must to be positive. Because $\phi_{S}$ is a free parameter, we have to minimise with respect to it. The extremum condition is \begin{equation} 2 \lambda'_S \sin 4 \phi_S + \lambda''_S \sin 2 \phi_S = (\lambda'_S + 4 \lambda''_S \cos 2 \phi_S) \sin 2 \phi_S = 0, \end{equation} yielding $\phi_S = \pm n \frac{\pi}{2}$ and $\phi_S = \frac{1}{2} \left[ \pm \arccos \left(-\frac{\lambda''_S}{4 \lambda'_S} \right) + 2 n \pi \right]$. The former solution reproduces \begin{equation} \lambda_S + \lambda'_S \pm \lambda''_S \geqslant 0, \label{eq:S:self-couplings:copos:min:always} \end{equation} while the latter solution gives \begin{equation} \lambda_{S} - \lambda'_{S} - \frac{\lambda_{S}^{\prime\prime 2}}{8 \lambda'_{S}} \geqslant 0. \label{eq:S:self-couplings:copos:min} \end{equation} Note that this condition only has to hold if the argument of the arccosine is within its domain \begin{equation} -1 \leqslant -\frac{\lambda''_S}{4 \lambda'_S} \leqslant 1. \label{eq:arccos:cond} \end{equation}
By the way, Sylvester's criterion yields a similar condition for the usual positivity of singlet self-couplings: \begin{equation} 8 (\lambda_{S} - \lambda'_{S}) \lambda'_{S} - \lambda_{S}^{\prime\prime 2} \geqslant 0. \label{eq:S:self-couplings:pos} \end{equation}
The allowed region is shown in
for $\lambda_{S} = 1/2$ (dark grey) and $\lambda_{S} = 1$ (light grey). In the light red region, \eqref{eq:arccos:cond} is not true and only \eqref{eq:S:self-couplings:copos:min:always} holds. If we neglected that and imposed \eqref{eq:S:self-couplings:copos:min}, we would exclude part of this region in error.
Another way to test copositivity is Kaplan's test: a symmetric matrix $A$ is copositive if and only if every principal submatrix $B$ of $A$ has no eigenvector $v > 0$ with associated eigenvalue $\lambda \leqslant 0$. Its advantage is that it generalises easily to copositive tensors. For matrices, the tensor eigenvalue equation \begin{equation} \Lambda v^{m-1} = \lambda v^{[m-1]}, \end{equation} where $m$ is the order of the tensor $\Lambda$, coincides with the characteristic equation (the vector $v^{[n]}$ has each component of $v$ taken to the power of $n$). Just like for matrices, the diagonal entries of tensors have to be non-negative, giving \eqref{eq:vac:stab:SR} and \eqref{eq:vac:stab:SI}. Solving the characteristic equation, we find for the components of the two eigenvectors and eigenvalues \begin{equation} (S_{I}^{2})_{\pm} = \frac{-\lambda''_{S} \pm \sqrt{(\lambda_{S} - 3 \lambda'_{S})^{2} + \lambda_{S}^{\prime\prime 2}}}% {\lambda_{S} - 3 \lambda'_{S}} S_{R}^{2}, \quad \lambda_{\pm} = \frac{1}{4} \left( \lambda_{S} + \lambda'_{S} \mp \sqrt{(\lambda_{S} - 3 \lambda'_{S})^{2} + \lambda_{S}^{\prime\prime 2}} \right), \end{equation} where we can take $S_{R}^{2} > 0$. To satisfy Kaplan's test, we must have \begin{equation} S_{I}^{2} > 0 \implies \lambda > 0.\end{equation}, which, together with the conditions \eqref{eq:vac:stab:SR} and \eqref{eq:vac:stab:SI}, reproduces the dark and light grey regions (the condition with $S_{I}^{2})_{-}$ appears to be stronger).
There is another, less general way to derive the conditions. One can go from the $S_{R,I}^{2}$ basis to one of \begin{equation} s_0 = |S|^{2} \quad \text{and} \quad s_1 = \frac{S^{2} + S^{\dagger 2}}{2}. \end{equation} Indeed, \begin{equation} s_0 = \frac{S_{R}^{2} + S_{I}^{2}}{2} \quad \text{and} \quad s_1 = \frac{S_{R}^{2} - S_{I}^{2}}{2}, \end{equation} and we can express \begin{equation} \frac{1}{2} (S^4 + S^{\dagger 4}) = 2 \left( \frac{S^{2} + S^{\dagger 2}}{2} \right)^{2} - |S|^{2} = 2 s_1^{2} - s_0^{2}. \end{equation}
The scalar potential becomes \begin{equation} \begin{split} V &= \lambda_{S} s_0^{2} + \lambda'_{S} (2 s_1^{2} - s_0^{2}) + \lambda''_{S} s_0 s_1 \\ &\equiv \Lambda_{00} s_0^{2} + 2 \Lambda_{01} s_0 s_1 + \Lambda_{11} s_1^{2}. %, % \\ % &= s_0^{2} (\Lambda_{00} + 2 \Lambda_{01} s + \Lambda_{11} s^{2}), \end{split} \label{eq:V:singlet:invs} \end{equation}
The matrix of couplings in the $s_0, s_1$ basis is \begin{equation} \Lambda = \begin{pmatrix} \lambda_S - \lambda'_S && \frac{\lambda''_S}{2} \\ \frac{\lambda''_S}{2} && 2 \lambda'_S \end{pmatrix}. \end{equation}
By definition $s_0 \geqslant 0$; also $s_0^{2} - s_1^{2} = S_{R}^{2} S_{I}^{2} \geqslant 0$ that defines the forward light cone in the $SO(1,1)$ field space, in analogy with the $SO(1,3)$ of the two Higgs doublet model.
This even works with explicit CP violation: we can use a phase shift of $S$ to fix the phases $\phi_{\lambda''_{S}} = \phi_{\lambda'_{S}}/2$.
We can diagonalise the tensor $\Lambda$ with a ’Lorentz transformation’ \begin{equation} \begin{pmatrix} \phantom{-}\cosh \phi && -\sinh \phi \\ -\sinh \phi && \phantom{-}\cosh \phi \end{pmatrix} \begin{pmatrix} \Lambda_{00} && \Lambda_{01} \\ \Lambda_{01} && \Lambda_{11} \end{pmatrix} \begin{pmatrix} \phantom{-}\cosh \phi && -\sinh \phi \\ -\sinh \phi && \phantom{-}\cosh \phi \end{pmatrix} = \begin{pmatrix} \Lambda_{0} && 0 \\ 0 && -\Lambda_{1} \end{pmatrix}. \end{equation} $\Lambda$ has a time-like eigenvalue $\Lambda_0$ and a space-like eigenvalue $-\Lambda_1$. The eigenvalues are \begin{align} \Lambda_{0} &= \frac{1}{2} \left[ \Lambda_{00} - \Lambda_{11} +\sqrt{ (\Lambda_{00} + \Lambda_{11})^{2} - 4 \Lambda_{01}^{2} } \right], \\ \Lambda_{1} &= \frac{1}{2} \left[ \Lambda_{00} - \Lambda_{11} - \sqrt{ (\Lambda_{00} + \Lambda_{11})^{2} - 4 \Lambda_{01}^{2} } \right] \end{align} or \begin{align} \Lambda_{0} &= \frac{1}{2} \left[ \lambda_{S} - 3 \lambda'_{S} +\sqrt{ (\lambda_{S} + \lambda'_{S})^{2} - \lambda_{S}^{\prime \prime 2} } \right], \\ \Lambda_{1} &= \frac{1}{2} \left[ \lambda_{S} - 3 \lambda'_{S} - \sqrt{ (\lambda_{S} + \lambda'_{S})^{2} - \lambda_{S}^{\prime \prime 2} } \right]. \end{align}
For the vacuum to be bounded below, one needs \begin{equation} \Lambda_0 \geqslant 0 \quad \text{and} \quad \Lambda_0 \geqslant \Lambda_1. \end{equation}
We see that $\Lambda_0 \geqslant 0$ directly corresponds to \eqref{eq:vac:stab:Cartesian:last}. From $\Lambda_{0} \geqslant \Lambda_{1}$ one gets \begin{equation} (\lambda_S + \lambda'_S + \lambda''_S) (\lambda_S + \lambda'_S - \lambda''_S) > 0. \end{equation} Both factors could be either positive or negative. But when $\lambda''_{S} = 0$, then $\Lambda_{0} > \Lambda_{1}$ gives $\lambda_{S} + \lambda'_{S} > 0$. Therefore they have to be both positive and we have recovered the full set of conditions \eqref{eq:vac:stab:SR}, \eqref{eq:vac:stab:SI} and \eqref{eq:vac:stab:Cartesian:last}.