Excited State-Specific CASSCF Theory for the Torsion of Ethylene

State-specific complete active space self-consistent field (SS-CASSCF) theory has emerged as a promising route to accurately predict electronically excited energy surfaces away from molecular equilibria. However, its accuracy and practicality for chemical systems of photochemical interest have yet to be fully determined. We investigate the performance of the SS-CASSCF theory for the low-lying ground and excited states in the double bond rotation of ethylene. We show that state-specific approximations with a minimal (2e,2o) active space provide comparable accuracy to state-averaged calculations with much larger active spaces, while optimizing the orbitals for each excited state significantly improves the spatial diffusivity of the wave function. However, the incorrect ordering of state-specific solutions causes excited state solutions to coalesce and disappear, creating unphysical discontinuities in the potential energy surface. Our findings highlight the theoretical challenges that must be overcome to realize practical applications of state-specific electronic structure theory for computational photochemistry.


I. INTRODUCTION
Simulations of dynamic photochemical processes rely on faithful descriptions of ground-and excited-state energy surfaces away from molecular equilibria, but obtaining accurate and efficient predictions of electronic excitations remains a major challenge. 1The prevalence of open-shell ground and excited states in photochemistry means that single-reference methods, such as equation-ofmotion coupled cluster (EOM-CC) 2 and time-dependent density functional theory (TD-DFT), 3 are generally restricted to molecular structures around the equilibrium geometry.5][6] However, state-averaging can give discontinuous energy surfaces due to "root-flipping" when electronic states cross, 7 large active spaces are required to capture all the relevant states, and using a common set of orbitals does not account for bespoke orbital relaxation in charge transfer and Rydberg excitations.
9][30][31][32] Compared to state-averaging, these approaches provide bespoke orbitals for each excitation, meaning that smaller active spaces can be used. 32Using a minimal multi-configurational expansion to capture the key open-shell configurations is expected to alleviate the issues of disappearing SCF solutions.Previous work has shown that unphysical solutions can still arise if the wrong active space is chosen, and solutions can undergo symmetry breaking or disappear as the molecular structure changes. 32However, the prevalence and significance of these irregularities for excited energy surfaces in larger molecules and basis sets of photochemical interest remain to be determined, preventing a firm evaluation of the long-term viability of SS-CASSCF theory.
In this contribution, we assess the performance of SS-CASSCF theory for the excited states in the double bond torsion of ethylene, which have been the subject of numerous theoretical and experimental studies over the past 50 years (see Ref. 33 for an excellent overview).The low-lying states of interest include the singlet and triplet π → 3s and π → 3p Rydberg excitations, the π → π * single excitation (V), and the (π) 2 → (π * ) 2 double excitation (Z).0][41] Furthermore, Angeli has highlighted the importance of dynamic σ-polarisation and subsequent orbital contraction in the V state. 42t the planar D 2h structure, the bonding π and antibonding π * orbitals transform as b 3u and b 2g , respectively, where the C -C bond coincides with the z-axis and the molecule lies in the yz-plane.The ground state and π → π * open-shell singlet excitation correspond to the 1 1 A g and 1 1 B 1u states.1][52] Accurate excited-state energies along this torsional mode are therefore essential, but SA-CASSCF is susceptible to root-flipping. 51Since each state is dominated by at most two determinants, we expect a state-specific (2e, 2o) active space to give a qualitatively correct description.
In this work, we investigate the applicability of the SS-CASSCF (2,2) approach for the ground and excited states in the torsion of ethylene.We show that multiple ground state solutions can occur, and we identify suitable stationary points for the low-lying Rydberg excitations, and the V and Z excited states.We find that SS-CASSCF (2,2) can provide similar accuracy to SA-CASSCF calculations with much larger active spaces and can recover the correct diffusivity of the valence V state.On the other hand, we show that the incorrect ordering of excitations due to missing dynamic correlation or non-diffuse basis functions can cause solutions to disappear, giving unphysical energy surfaces.Our findings highlight the promises and pitfalls of SS-CASSCF for practical excited-state applications.

II. THEORY A. State-specific CASSCF theory
Electronic states with unpaired electrons are inherently multi-configurational and must be modelled as a superposition of multiple Slater determinants using configuration interaction (CI).The complete active space (CAS) approach is the most common way to choose the subset of dominant configurations required to capture this "static" electron correlation.In CASCI, a subset of relevant active orbitals are chosen and a CI expansion is built using every possible way of arranging the active electrons in these partially occupied orbitals.The remaining inactive and virtual orbitals are fully occupied, and empty, respectively, in each configuration. 4,53As a truncated CI expansion, the CASCI wave function depends strongly on the choice of orbitals in the inactive, active, and virtual spaces.Therefore, the optimal wave function is usually identified by optimising the orbital and CI coefficients self-consistently with the CASSCF approach. 4n each optimisation step, the CASCI wave function is defined as where C IJ are the CI expansion coefficients for state J in terms of the active Slater determinants |Φ I ⟩.Variations in the CI and orbital coefficients can be represented using an exponential parametrisation as The anti-Hermitian operator R performs orbital rotations and is expressed in terms of the current orbitals [54][55][56] where Ê− pq = σϵ↑,↓ â † qσ âpσ −â † pσ âqσ is the anti-Hermitian singlet excitation operator. 57Similarly, the Ŝ operator transforms the CI expansion by considering the transfer operators between the target state |Ψ J ⟩ and the orthogonal states |Ψ K ⟩ in the CASCI space 56 , The energy E J (R, S) = ⟨Ψ J | e − Ŝ e − R Ĥe Re Ŝ |Ψ J ⟩ is then a function of the variables S K and R pq and the optimal CASSCF solutions are stationary points on the corresponding electronic energy landscape.

B. Computational details
Since exact excited states are higher-index saddle points of the electronic energy landscape, 8 we expect SS-CASSCF excited states to also be saddle points of the energy.9][60] We employ the eigenvector-following technique 61 to target stationary points with a particular Hessian index, as described in Ref. 32.For open-shell single excitations, an initial guess can be prepared by first optimising the orbitals for a suitable configuration state function (CSF) following the framework outlined in Ref. 57.Once an optimal SS-CASSCF solution has been found, it can be used as an initial guess for the next molecular geometry, allowing it to be tracked across the full potential energy surface.Since the Hessian index may change along a binding curve, the mode-controlled Newton-Raphson optimiser described in Ref. 32 is used to reconverge solutions at each geometry without prior knowledge of the Hessian index.
All calculations are performed using an in-house computational package developed in our group, which forms an extension to PySCF. 62We consider the aug-cc-pVDZ basis set, 63,64 which includes support for diffuse Rydberg states, and the smaller 6-31G basis set. 65The convergence threshold is set to a root-mean-square gradient value of 10 −7 E h .Figures are plotted using Mathematica 12.0 66 and orbitals are visualised using the VMD software. 67

A. Summary of SS-CASSCF (2,2) solutions
Using the aug-cc-pVDZ basis set, we first characterised the SS-CASSCF (2,2) solution space by starting from random MO and CI coefficients.We considered the planar D 2h geometry used in Ref. 68, which is provided in the Supporting Information.Low-energy solutions were targetted by searching for stationary points with Hessian indices between 0-10 using eigenvector-following.Up to 1000 random starting points were tested for each Hessian index.An extremely large number of low-energy solutions were identified, as illustrated in Fig. 1, making a complete characterisation of the solution space impossible.Instead, we focussed our attention on the solutions corresponding to local minima, the low-energy singlet and triplet single excitations, and the Z double excitation.Starting from a pre-optimised open-shell CSF allowed suitable stationary points to be found for the (π → 3s), (π → 3p), (π → π * ) excitations, among others.The (π) 2 → (π * ) 2 double excitation was identified by starting at the corresponding non-aufbau Slater determinant.Tracing the relevant solutions across the double bond rotation resulted in the ground-and excited-state energy surfaces shown in Fig. 2. Some solutions disappear along the torsional rotation.This disappearance can only occur if two stationary points coalesce on the CASSCF energy landscape at a pair annihilation point, 23,32 which mathematically corresponds to a fold catastrophe. 69This coalescence is associated with the onset of a zero eigenvalue in the Hessian matrix of second derivatives with respect to the wave function parameters, and similar phenomena occur for multiple Hartree-Fock solutions. 18,23,32,70The other solution involved in the pair annihilation can be identified using a line search in the direction of the eigenvector corresponding to the zero Hessian eigenvalue, as detailed in Appendix A.
In the following Sections, we characterise the local minima (Section III B) and the valence and Rydberg excitations (Section III C).Finally, we highlight how the SS-CASSCF solutions change if we use a smaller basis set that cannot describe Rydberg states (Section III D).

B. Multiple local minima
Although there is only one minimum on the exact energy landscape, 8 the SS-CASSCF (2,2) approximation yields five minima at the planar structure, corresponding to a unique global minimum and a four-fold degenerate set of local minima.The partially occupied natural orbitals for these solutions reveal that the global minimum corresponds to the expected {π, π * } active orbitals with occupations of 1.9150 and 0.0850, respectively (Fig. 3A).In contrast, the active orbitals for the local minima break the spatial symmetry and correspond to the quasi-localised C -H σ and σ * orbitals, with the four-fold degeneracy arising from the four C -H bonds (Fig. 3B).Since the true ground state is dominated by one closed-shell configuration, both active spaces include one orbital that is almost doubly occupied and one that is almost unoccupied.The active orbital with n occ ≈ 2 can be swapped for a doubly occupied inactive orbital without significantly changing the energy, leading to multiple representations of the ground state, as described in Ref. 32.Therefore, in the absence of strong static correlation at the D 2h geometry, the different minima attempt to capture dynamic correlation in either the C -H σ or C -C π bonds.
Although both sets of minima provide a reasonable approximation to the planar geometry, choosing the right active orbitals is essential for computing physically meaningful energy surfaces. 32The global minimum can be followed across the full torsion to give a smooth rotational barrier (Fig. 3C) because the {π, π * } active orbitals can correctly break the C -C π bond.In contrast, the energy of the C -H {σ, σ * } local minimum does not reach a maximum at 90 deg, and the solution eventually coalesces

C. Valence and Rydberg excitations
The low-lying singly excited states in ethylene correspond to excitations from the π orbital to a 3s or 3p Rydberg orbital, and the valence π → π * excitation.A SS-CASSCF (2,2) solution for each of the corresponding singlet and triplet excitations can be identified at the planar geometry.The orbital assignment and excitation energies are tabulated in Table I, alongside literature benchmark values.Remarkably, the SS-CASSCF (2,2) excitation energies correspond closely to SA-CASSCF results computed with a much larger (2,11) active space. 38his result highlights that the state-specific approach can provide accurate energies with significantly smaller active spaces by considering only the active orbitals that are directly involved in the excitation.Compared to the theoretical best estimates (TBE) from Ref. 33, the SS-CASSCF (2,2) Rydberg excitation energies are consistently underestimated by around 0.6 eV, as shown by the mean unsigned error (MUE) in Table I.Since the SS-CASSCF approximation predominantly captures static electron correlation, this consistent shift suggests that there is an imbalance between the dynamic correlation in the ground and Rydberg states, supporting the findings of Ref. 38.In particular, the spatially compact nature of the ground state leads to regions of higher electron density and thus greater dynamic correlation than the more diffuse Rydberg states.Therefore, SS-CASSCF (2,2) the ground-state energy and, by extension, the Rydberg excitation energies.The second-order moment ⟨x 2 ⟩ for the Rydberg states have an error around 2.5 a 2 0 , as shown in Table II, indicating that the state-specific wave functions are at least qualitatively accurate.
Whether the 1 1 B 1u π → π * state has predominant valence or Rydberg character has long been disputed due to the challenge of reproducing the experimental band absorption maximum at 7.6 eV.Recent studies have confirmed that nonadiabatic effects 36,47,48 shift this experimental value from the vertical excitation energy that is closer to 8.0 eV. 33,49,68The degree of Rydberg character can be measured through the ⟨x 2 ⟩ value, which can vary significantly for a small change in energy, 41 while dynamic correlation and σ-polarisation are expected to cause the excited state π * orbital to contract. 42This spatial contraction is not seen in state-averaged CASSCF, [42][43][44] as shown by the large x 2 value for SA-CASSCF (2,11) 38 in Table II.In contrast, the SS-CASSCF (2,2) excited state clearly yields a more contracted π * orbital (Fig. 4) than the ground state solution, giving a x 2 value that closely matches the TBE.
The improvement in the spatial diffusivity of the SS-CASSCF wave function is also reflected in the oscillator strength for the π → π * excitation.Since the statespecific ground and excited states are represented with different sets of orbitals, we use the extended nonorthogonal Wick's theorem 71,72 implemented in the LibGNME software package 73 to evaluate the transition dipole moment.Compared to the 0.17 a.u.error for the SA-CASSCF (2,11) result from Ref. 38, the SS-CASSCF (2,2) approach predicts the oscillator strength with a deviation of 0.035 a.u.from the TBE (Table II).This improvement suggests that the state-specific approach reduces the contamination from nearby Rydberg states, which have a weaker oscillator strength than the valence excitation.Compared to the −0.6 eV underestimate for the Rydberg excitation energies, the SS-CASSCF (2,2) approximation overestimates the V excitation energy by 0.36 eV.This overestimate can be understood because the π → π * is dominated by zwitterionic resonance structures with a larger dynamic correlation energy than the ground state, which is not captured by the CASSCF approximation. 39,40,42Therefore, the SS-CASSCF (2,2) approximation erroneously predicts that the valence 1 1 B 1u state is higher in energy than the Rydberg 1 1 B 1g and 1 1 B 2g states at the planar geometry.This incorrect ordering has profound consequences on the corresponding excited-state energy surfaces along the torsional rotation.As the molecule twists away from the planar geometry, the spatial point group changes from D 2h to D 2d .Under this descent in symmetry, the planar 1 1 B 1u and both transform as the same 1 B 1 irreducible representation, meaning that they can now couple through the Hamiltonian.The π → π * and π → 3p y excited states become lower and higher in energy, respectively, eventually leading to an unphysical avoided crossing (cyan in Fig. 5).
State-specific approximations are known to have un- physical solutions or coalescence points in the vicinity of avoided crossings. 17,23,32Here, we see that the higher energy solution, corresponding to the planar π → π * state, continuously transforms into the π → 3p y state at the avoided crossing, as shown in Fig. 5.In contrast, the lower energy solution continues to increase in energy until it eventually disappears in a pairwise coalescence point at 42 deg.The other solution involved in the coalescence can be followed back to 37 deg, where it coalesces with a third solution that corresponds to the π → π * state after the avoided crossing.Therefore, the lower π → 3p y solution is the only physically meaningful state that cannot be followed across the full torsional rotation, creating potential issues for the use of SS-CASSCF theory in ab initio excited-state molecular dynamics.We attempted to avoid this issue using a (2e, 3o) active space that contained both the π * and 3p y with no success.On the other hand, the state-specific philosophy successfully avoids the more widespread discontinuities that occur in state-averaged calculations, as seen in Fig. 6 of Ref. 51.
Finally, we consider the double excitation (π) 2 → (π * ) 2 , which cannot be captured by linear response formalisms such as TD-DFT.Starting from the non-aufbau Slater determinant at the planar geometry, the corresponding SS-CASSCF (2,2) solution can be identified with an excitation energy of ∆E = 14.46 eV and provides a continuous energy surface across the full torsional rotation (Fig. 2).This state is less well covered in the literature, but benchmark values from the QUEST dataset 74,75 and Ref. 52 predict an excitation energy closer to 13-13.6 eV.Therefore, the SS-CASSCF (2,2) overestimates the double excitation energy, which we believe is a result of the unbalanced dynamic correlation between the ground and excited states, as already seen for the π → π * excitation.

D. Consequences of a non-diffuse basis set
The presence of low-energy Rydberg states means that diffuse basis functions are considered to be essential for accurately predicting the excited states in ethylene. 33,35,37e also performed SS-CASSCF (2,2) calculations using the 6-31G basis set, highlighting how the lack of diffuse basis functions can fundamentally change the pattern of state-specific solutions in ethylene.While the ground state exhibited a global minimum and four-fold degenerate local minima that are directly analogous to the aug-cc-pVDZ basis, we were unable to find any physically meaningful approximations to the singly excited π → π * or the doubly excited (π) 2 → (π * ) 2 energy surfaces.
To target the π → π * excited state, we started the SS-CASSCF (2,2) optimisation from the output of a stateaveraged CASSCF (2,2) calculation at the planar geometry.The planar molecular structure was identified through a geometry optimisation using the B3LYP functional and is provided in the Supporting Information.Starting from the state-averaged π → π * initial guess gave a stationary point with symmetry-pure orbitals, with the natural orbitals corresponding to the localised zwitterionic configurations (Fig. 6A).However, this solution only exists up to a torsional angle of 0.02 deg, where it disappears in a pair annihilation point (Fig. 6: Inset 2).A complex pattern of coalescing solutions can be found that ultimately connects the π → π * solution to another solution that emerges at 1.5 deg, which increases in energy for higher torsional angles (cyan in Fig. 6A).
Alternatively, searching for the π → π * state at 90 deg yields a solution that exists all the way to 0 deg (black in Fig. 6).However, the corresponding natural orbitals at the planar geometry represent the 1 B 1g σ → π * excitation, which is known to be higher in energy than the π → π * state. 33Ultimately, the smaller 6-31G basis set results in the incorrect ordering of the 1 B 1g and 1 B 1u excited states because it cannot describe the diffuse character of the π → π * state, as indicated by the small x 2 value of 12.24 a 2 0 .Like the interaction between the π → π * excitation and the Rydberg states using the aug-cc-pVDZ basis, this ordering problem creates an unphysical avoided crossing that causes SS-CASSCF (2,2) solutions to coalesce and disappear as the double bond rotates, leading to catastrophic potential energy surfaces.
Similarly, starting from the state-averaged states allows a symmetry-pure SS-CASSCF (2,2) solution to be identified for the (π) 2 → (π * ) 2 double excitation (purple in Fig. 6B).However, this solution also disappears as the molecule is twisted and cannot be traced beyond 3.8 deg, where it coalesces with another solution (green in Fig. 6B).This second state can be traced back to the planar geometry, where it forms a pair of degenerate solutions with natural orbitals that break the spatial symmetry (the degeneracy is lifted for non-zero torsional angles).The other degenerate solution can be followed across the full torsional mode for angles between 0 and 180 deg (grey in Fig. 6B).However, as these degenerate solutions break the spatial symmetry and cross in energy at 0 deg, neither predicts a stationary point in the excited energy surface at the planar geometry.Consequently, the SS-CASSCF (2,2) approximation is not able to provide any meaningful potential energy surface for the (π) 2 → (π * ) 2 Z state of ethylene using the 6-31G basis, and it is vital that the basis set is sufficient for the excited states of interest.

IV. CONCLUDING REMARKS
Excited state-specific approximations promise to overcome the challenges of state-averaged CASSCF theory for predicting excited energy surfaces by facilitating calculations with smaller active spaces and avoiding root-flipping discontinuities.In this work, we assessed the performance of the SS-CASSCF (2,2) approach for the valence and Rydberg excitations in the torsion of ethylene, using the aug-cc-pVDZ and 6-31G basis sets.While a large number of SS-CASSCF (2,2) solutions exist, we were able to target physically meaningful stationary points for the lowlying excited states at the planar D 2h structure using the aug-cc-pVDZ basis set.These solutions provided excitation energies and properties that are comparable to much larger state-averaged approximations, highlighting that SS-CASSCF can be applied with only the active orbitals that are involved in each excitation.Furthermore, most of the SS-CASSCF (2,2) solutions using aug-cc-pVDZ can be continuously followed across the torsional rotation, avoiding the root-flipping problems in SA-CASSCF and the limitations of single-reference linear-response methods.
The imbalance between the missing dynamic correlation in Rydberg and valence excited states means that SS-CASSCF (2,2) theory fails to provide the correct state ordering in planar ethylene.This incorrect ordering of the π → 3p y and π → π * states using the aug-cc-pVDZ basis set creates an artificial avoided crossing away from the planar geometry that manifests as a pair annihilation point, where one of the states coalesces with another unphysical solution and disappears.Since the reference SS-CASSCF (2,2) solution mathematically disappears, these irregularities cannot be remedied by post-CASSCF correlation methods such as CASPT2, [76][77][78] multireference CI, 79 or even multi-state CASPT2. 80Instead, a state-specific wave function approximation that is optimised in the presence of dynamic correlation will be required to stop states from disappearing.Therefore, there is a trade-off between coalescing solutions and root-flipping discontinuities in state-specific and state-averaged CASSCF, respectively Furthermore, SS-CASSCF(2,2) calculations with the 6-31G basis set cannot capture the diffuse character of the π → π * state at all, which is predicted to be too high in energy.This error causes an artificial avoided crossing with the σ → π * excitation, and we were unable to find any meaningful energy surfaces for the π → π * or (π) 2 → (π * ) 2 states.These observations emphasise the importance of using sufficient basis sets for the excited states of interest, and also highlight the danger of assessing state-specific approximations using inadequate basis sets.
Ultimately, the coalescence and disappearance of solutions remains the primary obstacle to practical excited state-specific calculations.These coalescence points are mainly due to the unbalanced description of valence and Rydberg excitations.While this imbalance may be at-tributed to the lack of dynamic correlation, an alternative perspective is that the SS-CASSCF approximation simply is not the right reference for molecular excited states.Since the ethylene single excitations correspond to openshell singlets, further restricting the wave function to a single CSF would not change our results.Instead, we believe that new wave function approximations, which explicitly include the effects of dynamic σ-polarisation and orbital contraction in excited states, may provide more accurate and efficient energy surfaces for photochemistry, and we intend to pursue this direction in future work.

FIG. 1 :
FIG. 1: Number of SS-CASSCF (2,2) solutions identified at the D 2h geometry (aug-cc-pVDZ) using random starting guesses.Inset: The number of stationary points associated with the closed-shell ground state.

78 FIG. 5 :
FIG. 5: SS-CASSCF (2,2) predicts the wrong ordering for the π → π * and π → 3p y states at the planar geometry, leading to an avoided crossing along the torsional rotation.The lower energy solution disappears at a pair annihilation point (42 deg) and a new discontinuous SS-CASSCF (2,2) solution emerges (37 deg), which represents the π → π * state at larger torsional angles.Rydberg states with different symmetries are unaffected.

nn
FIG. 6: SS-CASSCF (2,2) with the 6-31G basis set does not provide physically meaningful energy surfaces for the singly excited π → π * state or the doubly excited (π) 2 → (π * ) 2 state.(A) This approximation predicts the wrong ordering of the π → π * and σ → π * states at the planar geometry, leading to a series of symmetry-broken (SB) solutions and an unphysical avoided crossing.(B) The symmetry-pure solution (purple) corresponding to the (π) 2 → (π * ) 2 excitation disappears at a torsional angle of 3.8 deg, giving an unphysical potential energy surface.

TABLE I :
Vertical excitation energies (eV) computed with SS-and SA-CASSCF are compared against theoretical best estimates (TBE).