This short course will introduce the framework of distribution-free statistical inference, and will provide an in-depth overview of both theoretical foundations and practical methodologies in this field. We will cover methods including holdout set methods, conformal prediction, cross-validation based methods, calibration procedures, and more, with emphasis on how these methods can be adapted to a range of settings to achieve robust uncertainty quantification without compromising on accuracy. The course will also introduce the theoretical results behind these methods, including the role of exchangeability (and its variants) in establishing the distribution-free validity of these methods, as well as more classical theoretical results establishing how these distribution-free methods relate to the answers we would obtain via parametric models or other classical assumption-based techniques. Our theoretical overview will also cover hardness results that carve out the space of inference questions that are possible or impossible to answer within the distribution-free framework.