In a classroom, limiting government debt in relation to GDP can be defended. The idea is to reassure investors (a.k.a. "financial markets") that the debt burden isn't becoming heavier so they will continue lending at low interest rates. But in real life, the logic doesn't work. Governments inevitably face deep recessions, wars or other emergencies that require heavy borrowing. To stabilize debt to GDP, you have to aim much lower than the target in good times, meaning that you should balance the budget (or run modest surpluses) after the economy has recovered from recessions.
The problem of what is a sustainable sovereign debt load is complex. There are (at least) two states of the world. In one state, investors have confidence in the country's economy and politics, and the country can sustain just about any debt load. In another state, investors have lost confidence, and the country has to really tighten its fiscal policy--and quite often the country cannot do so, which is why investors lost confidence in the first place.
Samuelson argues for balancing the budget in non-recessionary times. There is no purely economic theory that requires this. However, from a political economy perspective, some focal point is needed in order to discipline budgets. I think returning to a full-employment surplus of zero might be a good focal point.
Note: when you encounter some economic jargon, you can often find it in the Concise Encyclopedia of Economics, available in the search box at the top of Econlog. By specifying "encyclopedia" and searching for "full-employment surplus," you get these results.