| In recent years, a new type of material, topological insulators, has attracted huge attention in condensed matter physics. Topological insulators are different from ordinary insulators in the sense that the bulk is insulating but the electronic states on the surfaces are the higher dimensional quantum spin Hall phase. An important prediction of surface states is their insensitivity to any scattering and disorder that does not break the time-reversal symmetry. The surface electron therefore is expected to have a long mean free path. Shortly after the theoretical prediction of three dimensional (3D) topological insulators in real materials, the topological phases were experimentally observed by photoemission measurements in bismuth based compounds, such as Bi 1-xSbx, Bi 2Te3, and Bi2Se3. In particular, the surface states in Bi2Te3 and Bi2Se3 feature a single Dirac cone on each surface. Electrons behave as massless (Dirac) relativistic particles with a net spin. Unlike conventional Dirac fermions in graphene, the helical Dirac fermions in topological insulators are expected to show unusual transport properties, including an anomalous half-integer quantization of the Hall conductance, a realization of Majorana fermions, and the striking topological magnetoelectric effect.;In this thesis, we describe the transport study of topological insulators Bi1-xSbx and Bi2Te3. We first performed the size-dependent measurements of Bi1-xSb x and find that the carrier density significantly depends on the geometric size of the sample. This implies a two-dimensional conduction band coexisting with bulk conduction. By doing Hall measurements in high magnetic fields, we characterized the basic transport properties of surface states from the Shubnikov-de Haas oscillations. Moreover, we uncovered a Hall anomaly in the weak field limit, which enables the surface current to be seen directly. Finally, we observed fractional-filling states in the Landau levels from the thermopower measurement of topological insulator Bi2Te3. |
