Lower gear is higher number?

If you're talking speed yes, if you're talking torque, no. If you're talking ratio, no.

The number is the ratio of the input to the output. High gear in your transmission will be a ratio of 1 to 1. Every time your engine makes one revolution the output shaft will turn one time. If you drive at 3,000 rpm your driveshaft will turn at 3,000 rpm in high gear.
If you put it in first gear with a ratio of say 2.56 to one, the engine will turn 2.56 times for every one revolution of the output shaft. At 3,000 rpm the driveshaft will only be turning 1,171 rpm. 2.56 is a higher number than 1 but the speed is lower. Torque out is torque in times gear ratio. My engine makes 525 pounds feet of torque at 3,000 rpm. Ignoring friction, clutch slippage and other losses we have 525 X 2.56=1,344. In high gear we have 525 in and 525 out.

Same thing with the differential, the ratio is the comparison of the number of teeth in the ring gear to the number of teeth in the pinion. In a 4.11 rear the ring gear has 37 teeth and the pinion has 9 37/9=4.1111 thus every time the driveshaft turns one revolution the axle turns 0.243 times or for every one turn of the axle the driveshaft will turn 4.11 times. At 3,000 rpm engine speed in high gear the axles are only turning 729.9 rpm. With a 3.55 rear the axles turn at 845 rpm. 4.11 is larger than 3.55 but it has less speed at the same rpm

Higher number less speed but higher number higher torque. You move slower in first gear than you do in high gear and slower with a 411 than with a 355, but the torque is much higher in first than high gear and much higher with a 411 than a 355. Engines with higher low-rpm torque can perform better with a lower gear ratio than engines with less low-rpm torque. Our Pontiac engines traditionally made pretty good torque at low rpm and could tolerate a lower gear ratio than some other engines. Chev 327s made good horsepower at high rpm but less low-rpm torque.

The confusion comes from mistaking the speed ratio with the gear ratio. A 4.11:1 rear gear has a higher ratio than a 3.55:1 rear gear, higher number higher ratio.

Differential people refer to higher ratio, higher number gears as low gears. They provide more torque and higher rpm for a given road speed. They call lower ratio, lower numbered gears as high gears. They provide less torque and lower rpm for the same road speed. Don't ask me why, maybe thinking as with transmission first gear is lower than third but first has a higher ratio than third. I always refer to the ratio can't go wrong that way.

RPM formula:
rpm=mphXtransmission ratioXaxle ratioX336/tire diameter

lower gear-redlight gear
higher gear-highway gear

Muddle things up even more?