I get the concern, and share it, but I still think you gotta take it. It's an NFL head coaching gig. And if you have talent, you find your level. If he has HC talent, even if the Browns job is a bust, he'll get another chance (see Belichick, Carroll). If it turns out his peak is elite OC, he can fail at multiple HC gigs and still end up being a top OC in the league, like Josh McDaniels who has been fired as a HC twice, and is the OC for a team that just punched their ticket to the Super Bowl.
I mean, even the guys who have fallen on their faces spectacularly with the Browns all found work after. Stefanski immediately found another NFL head coaching gig. Kitchens is still getting opportunities (and getting fired from them). Even Hue Jackson managed to find work after going an astonishing 3-36 in his time as the Browns HC.