I was able to remove two questions from a course created using the red X. When I go to Test Options and my students still see 12 questions instead of 10 even though the extra 2 questions are blank screens. Snippet below shows that the 12 is greyed out and I cannot change it to 10.
