The resistor in series with the LED is usually called a "current-limiting resistor". In an LED, the current through that LED is roughly an exponential function of the voltage across the LED (once you pass the threshold voltage that turns on the LED.) So you might have a red LED with a threshold voltage of roughly 1.8V. Once you exceed 1.8V, the current will increase fairly quickly. Of course, this means that the power going through the LED will increase by quite a bit as well, and if too much power goes through the LED it will overheat and burn out. Usually LEDs have a max current rating; (10-20mA are pretty common for the cheap common LEDs.) You want to prevent the current from exceeding that, so you put a resistor in series with it. (The Arduino UNO runs off 5V; if you stick 5V across the LED only, 5V is MUCH bigger than the 1.8V threshold of the LED, and it'll draw a lot of current and burn out.)
https://www.sparkfun.com/tutorials/219
For the button sensor, I'm assuming that you mean that there is switch connecting the digital pin to 5V, and a resistor between the digital pin and GND. That resistor would be called a "pull-down resistor." There's the flipped version which is a "pull-up resistor":
https://learn.sparkfun.com/tutorials/pull-up-resistors. The concept is fairly the same.
The Arduino can only measure voltages, not current, so you need a circuit that can switch between 5V (HIGH) and GND (LOW). If you don't have the resistor there, then when the switch is not pressed, then your digital pin is "floating", or isn't measuring any voltage. Once you press the switch though, it is connected to 5V, which should read you a "HIGH". The floating voltage is not what you want; it is somewhat random and can give you both HIGH and LOW. Thus, you need the digital pin to be connected to GND when the switch is NOT pressed.
If you put a wire between the digital pin and GND, that might work, but what happens when you press the switch? Now you basically have a wire straight from 5V to GND, which is a short circuit. So instead, you use a resistor instead, as that limits the current when the switch is pressed, but still allows you to see GND when the switch is not pressed.