| Ultracold molecules have very low temperature and abundant rovibrational levels, allows ones to study intermolecular interaction and dynamics process of molecular reaction in quantum mechanism. So far, ultracold molecules have many appilications in many research fields, including precision measurement, many-body physics, quantum simulation, quantum compution, quantum information engineering and many other frontier physical topics, which gives rise to a wide interest of scientists. At present, the main ways for formation of ultracold molecules are photoassociation of ultracold atoms, Feshbach resonance and stimulated Raman adiabatic transition obtained by an effective combination of two mentioned methods. Because of the short lifetime of Feshbach molecular state used as the initial state of the stimulated Raman adiabatic transition, the scattering atomic state with a long lifetime has been proposed as the initial state of the stimulated Raman adiabatic transition near a Feshbach resonance, which is theoretically based on the enhanced photoassociation near Feshbach resonance. In this thesis, our destination is the research on the controlled photoassociation of ultracold Cs molecules with a Feshbach resonance and we have realized the enhanced photoassociation using the Feshbach resonance. The cold Cs atomic sample, prepared by the degenerated three-demensional Raman sideband cooling, is loaded into the magnetically levitated crossed optical dipole trap. The photoassociation spectra and a d-wave Feshbach resonance have been measured for the optically trapped Cs atoms. The photoassociation is investigated near the d-wave Feshbach resonance of ultracold Cs atoms and the enhanced photoassociation near Feshbach resonance is observed. The Feshbach resonance induces Fano effect in the photoassociation, and we propose the Fano resonance theory that resultes from magneto-optical quantum interference in atom-molecular system. For the laser intensity induced shift of the resonant frequency on the transition between scattering atomic and excited molecular states in the photoassociation, we study on the influence of a wide external magnetic field and Feshbach resonance on the slope of frequency shift. Additionally, Fano resonance theory has been employed to explain the phenomenon of controlled laser intensity induced frequency shift near Feshbach resonance and we have obtained a good agreement between theory and experiment.The main works are summarized as follows:1. The enhanced degenerated three dimensional Raman sideband cooling has been used to prepare ultracold Cs atoms polarized in the F=3, mF=3 state with the temperature of~1.7μK, which suppresses the inelastic endothermic collisions of Cs atoms in F= 3, mF= 3 state and improves the loading efficiency of the following optical dipole trap.2. The cold Cs atoms prepared by the enhanced degenerated three demensional Raman sideband cooling are loaded efficiently into a large-volume crossed optical dipole trap with the help of magnetic levitation technique. We have measured the loading rates of optical dipole trap with different magnetic field gradients and bias fields. The theoretical analysis gives a good agreement with the experimental results.3. The photoassociation and Feshbach resonance are performed for the ultracold Cs atoms trapped in optical dipole trap. The rovibrational spectra of ultracold excited Cs molecules in the long-range 0g- state and a d-wave Feshbach resonance near 48G of Cs atoms in the F=3, mF=3 state have been detected.4. The photoassociation of ultracold Cs atoms near the d-wave Feshbach resonance is investigated, we have obtained the enhanced photoassociation. The enhanced photoassociation near Feshbach resonance can be explained by considering the influence of Feshbach resonance on the Franck-Condon factor in photoassociation. The Fano resonance model induced by the magneto-optical quantum interference in atom-molecular system has been established to explain the Fano effect in the photoassociation near the Feshbach resonance. At the same time, we have obtained the controlled photoassociation with a wide external magnetic field and a qualitative theoretical explaination is given.5. The dependence of slope of laser intensity induced frequency shift in photoassociation of ultracold Cs atoms on a wide bias field is measured and we find that the external magnetic field can be used to control the slope of frequency shift. In addition, the influence of Feshbach resonance on the slope of frequency shift is also investigated where the Fano resonance theory in atom-molecular system is used to fit the variation of slope of frequency shift as magnetic field and gives a good agreement. |