This paper sheds light on minimizing the maximum tardiness with processing and setup times under both learning effect and deterioration. In this paper, all the jobs have processing and setup times under effects of learning and deterioration. By the effects of learning and deterioration, we mean that the processing time of a job is defined by an increasing function of its execution start time and position in the sequence. We provide a branch and bound algorithm to minimize the maximum tardiness under effects of learning and deterioration with setup times. Computational experiments show that the proposed algorithm can solve instances up to 800 jobs in reasonable time.