Confinement Properties of the 3D Topological Insulators Bi₂Se₃, Bi₂Te₃, and Sb₂Te₃

Recent discoveries have spurred the theoretical prediction and experimental realization of novel materials that have topological properties arising from band inversion. Such topological insulators have conductive surface or edge states but are insulating in the bulk. How the signatures of topological behavior evolve when the system size is reduced is noteworthy from both a fundamental and an application-oriented point of view, as such understanding may form the basis for tailoring systems to be in specific topological phases. This thesis investigates the softly confined topological insulator family of Bi₂Se₃ and its properties when subjected to an in-plane magnetic field. The model system provides a useful platform for systematic study of the transition between the normal and the topological phases, including the development of band inversion and the formation of massless-Dirac-fermion surface states. The effects of bare size quantization, two-dimensional-subband mixing, and electron-hole asymmetry are disentangled and their corresponding physical consequences elucidated.  When a magnetic field is present, it is found that the Dirac cone which is formed in surface states, splits into two cones separated in momentum space and that these cones exhibit properties of Weyl fermions. The effective Zeeman splitting is much larger for the surface states than for the bulk states. Furthermore, the g-factor of the surface states depends on the size of the material. The mathematical model presented here may be realizable experimentally in the frame of optical lattices in ultra cold atom gases.