He means that it's not a game because a game is interactive, and you don't actually
do anything in Dear Esther besides holding the "move forward" key for the whole duration of the game. I'm inclined to agree with him on that, though I think he didn't quite get to the main issue here.
I don't think that the problem with it is that it doesn't have any failure state - there are plenty of visual novels which I would still consider games. Unlike Beyond, in those types of games, the choices are often all equivalently "valuable" from the point of view of the game, so none of them can really be considered success/failure states; they're just different branches of the story, only one of which you get to see per playthrough, none of which are considered "the good version/success" or "the bad version/failure". The fact that you still have choice there is, to me, what makes them still be games. The problem with Dear Esther, I think, is that it doesn't have any meaningful choice. You can't make any decisions which alter your experience of the game, making it no different than a book or a movie in content. Interactivity (which really boils down to player choice), to me, is the essence of what a game is; if that is absent, then what remains can't be considered a game. On this issue, I completely agree with TB's stance.
On the other hand, I disagree with TB on the point that all games
should be aiming for a balance, or an equal marriage between narrative and mechanics. I think there's a spectrum between those two, and a game can fall anywhere on that spectrum and still be "ideal". Sure, there's plenty of new, interesting ideas that haven't been done at all yet which would, in fact, make great games, if they explored and took advantage of being right in the middle of that spectrum. I don't doubt that at all, the example he mentioned with Brothers is a testament to that.
But I don't think taking any particular existing game and moving it from wherever it is on this spectrum to any other place, be it the middle or anywhere else, will inherently make the game any better.
Take The Stanley Parable, for example. It has essentially no gameplay mechanics. Other than very rare, occasional instances of you being able to click on things, there is no interactivity in the game besides being able to move. The difference between that and Dear Esther though, is there's a huge variety of choices you can make just by moving, reacting to the environment and reacting to what the narrator is saying, and the consequences of those choices make a very big difference in how you experience the game. I would definitely consider it a game. Would the game have been better served by having more mechanics, though? Would it really have been better if you had more buttons to press or menus to navigate to do the same thing? I don't think it would. The whole simplicity of it is part of what makes the game's message so strong in the first place. For that particular game, having very little interaction is exactly what works out to be the most effective at achieving the experience the designer was trying to convey in the first place.
That's a game that goes pretty much as far as possible in the direction of pure narrative while still maintaining enough interactivity and choice to be a game.
On the other hand, you have fighting games. That's the exact opposite side of the spectrum: there is no narrative. When it does exist, it's of no consequence to the contents of the actual game and is completely optional. The actual meat of the game is completely devoid of story. It literally is nothing but a collection of mechanics. And yet, as plenty of people can attest, they're also still incredibly deep, engaging and valuable experiences, with no narrative (or no narrative of any consequence) at all. Would they be any better, fulfilling or engaging if they had better narratives? I don't really think so. That's not the point of those games, and it's not why people enjoy them.