| The coupling between cavity and atomic systems,as one of the important platforms for quantum simulation,has been extensively studied in both experimental and theoretical aspects over the past few decades.This field is known as cavity quantum electrodynamics(cavity-QED)physics ranging from few-body problems such as JaynesCummings model to many-body physics such as the polariton condensation and the Dicke superradiance.On the application side,such light-atom hybrid systems play important roles in quantum information processing.In the field of fundamental research,such a system provides an ideal platform for quantum simulation,which can simulate models in condensed matter physics,and explore novel quantum states and their nonequilibrium dynamics as well as other phenomena.Recently,interesting dynamical instabilities in the superradiant self-organization have attracted much attention,where nonsteady behavior such as the limit cycle may emerge without an explicit time-dependent external driving,which shares strong similarities with time crystals.To obtain stable limit cycles in the BEC-cavity system,prior studies have employed the blue-detuned driving,or utilized the spinor BECs with competing density and spin couplings with cavity mode.In contrast,for the experimentally more accessible single-component BEC-cavity system with a red-detuned pump,most previous studies have focused on the stable steady-state superradiance,the instability properties are not well explored.In this paper,we investigate dynamical instabilities of a BEC inside a high-finesse optical cavity,with a transverse driving field red-detuned from the relevant atomic resonance.We map out the dynamical phase diagrams and investigate the effects of atomic interaction and noise on the dynamics.The main research results of this paper:1.We revealed new limit cycles arising from the instability due to the interplay between cavity field dissipation and atomic resonance frequency shift.By numerical simulations and analysis,we generated the dynamical phase diagrams of the system.As the pump rate increases,the system first undergoes a transition from normal phase to superradiant phase.Further increasing the pump rate,we observed an excessive negative feedback effect on the atomic density modulation through shifting the phase of cavity mode,leading to the emergence of instability behavior,including the superradiant limit cycles and chaos.2.We found that the two superradiant limit cycles exhibit an intriguing phenomenon of merge.The oscillations and recurrences can also be seen in the condensate wave function for the superradiant limit cycles.The two limit cycles may first merge together as the pump rate increases,leading to a single limit cycle and restoring the Z2 symmetry.At the merging point,the system can switch between the two orbits and the dynamics losses the periodicity3.We predicted a new limit cycle phenomenon with purely atomic excitation.Through studying the dynamical evolution of the system,we found a new limit cycle behavior that is distinct from the superradiant limit cycle,namely,the limit cycle phenomenon with only atomic excitation.The cavity field is suppressed to zero by the interference between scatterings from different momentum states.Moreover,using the truncated Wigner approximation method,we found that the limit cycles with purely atomic excitation are not affected by Langevin noise and quantum fluctuations. |