Optimization is often the computational bottleneck in disciplines such as statistics, biology, physics, finance or economics. Many optimization problems can be directly cast in the well- studied convex optimization framework. For non-convex problems, it is often possible to derive convex or spectral relaxations, i.e., derive approximations schemes using spectral or convex optimization tools. Convex and spectral relaxations usually provide guarantees on the quality of the retrieved solutions, which often transcribes in better performance and robustness in practical applications, compared to naive greedy schemes. In this thesis, we focus on the problems of phase retrieval, seriation and ranking from pairwise comparisons. For each of these combinatorial problems we formulate convex and spectral relaxations that are robust, flexible and scalable.