Also make sure your brake shoes are adjusted "in". Sometimes the drum develops a lip due to wear from the brake shoes and prevents the drum from coming off.
Remove the rubber plug on the lower part of the backing plate to gain access to the adjusting star. You will want to rotate the star using a downward motion with a flat screw driver or brake adjusting tool to go against the normal action of the adjusting lever.
