Heat a large pot of salted water over high heat. While waiting for water to boil, peel potatoes and cut into 1-2 inch chunks.
When water boils, add potatoes and cook for 15-20 minutes until very tender. Drain and return to pot.
While potatoes are boiling, melt butter in a medium saucepan over medium-low heat. Add smashed garlic and cook, stirring occasionally, until garlic is golden-brown, about 3-5 minutes. Remove from heat.
Mash potatoes to desired consistency with a potato masher or ricer.
Garlic can be removed from the butter with a slotted spoon for smoother mashed potatoes, or left in if you want toasty garlic chunks. Add to potatoes along with 1/4 cup of milk, ½ tsp salt, and ¼ tsp pepper.
Mix until smooth. Add more milk as needed to obtain desired consistency, and adjust seasoning with salt and pepper to taste.
Serve immediately or keep warm over low heat, stirring occasionally, until ready to serve.