why would one consider hedging/derivative strategies, e.g., longs and shorts, collars and arbitrage? Stated differently (I think), a long term buy and hold of this assumed portfolio would seem to maximize its return without these hedging strategies. Yet the sense I get from the portfolio literature is that even for a modest investment return objective (whatever that may be), one can increase this modest return without increasing risk by replacing a portion of the assumed portfolio with some hedging type assets. What's the risk theory here?
In the capital asset pricing model, you optimize risk-return by holding a diversified portfolio. Derivatives are redundant assets. I think that is the path that Fischer Black took to deriving the option pricing formula. I think he started out wondering why anyone would use derivatives, but he ended up in a different place (a cynic could say that the different place was Wall Street, which profits from derivatives).
One could argue that in the real world there individuals face institutional constraints on portfolio diversification (for example, your wealth is closely tied to a single firm). In addition, firms face bankruptcy costs, so that they do not follow Modigliani-Miller. Given those constraints, there is a role for derivatives in mitigating the risks that imperfectly diversified agents would otherwise have to bear.
However, it is my view that if you look closely enough, you will see derivatives used primarily either to reduce transaction costs (as in stock index futures, which allow you to trade a basket of stocks more cheaply) or for regulatory arbitrage. One of the big mistakes that regulators made during the mortgage securities boom was to view derivatives as creating a new world of risk management. The novelty was in evading capital requirements, not in making banks fundamentally safer.
P.S. I can tell from the early comments that if you don't know the capital asset pricing model you won't understand this post. CAPM is one of the most difficult concepts to explain I have ever encountered. Past attempts here and here.
I strongly recommend going through the effort to try to understand CAPM. I think it is very insightful.