Compared to the joyous bliss of freedom and released awareness, the Buddha saw the plight of beings in bondage and confusion as an unremitting condition of suffering. Hence in the diagnosis that he submitted to the world, the first noble truth (or "holy fact", as I prefer) was that all misknowledge-driven living is constant misery. The Dhammacakka-pavattana Sutta records him as saying simply, "All this is misery", the commentarial tradition informing us that the "this" refers to the "contaminated aggregates" (sasravaskandha) of the misknowing body-mind-complex.
So it must have easily seemed to people who felt happy now and then that the Buddha was profoundly pessimistic. They tended to fasten on the first Holy Fact of suffering as proof of a grim outlook indeed. It was not clear to them that he called this the first "holy fact" because he was aware that it was a fact only for a holy person - ordinary egocentric individuals consider this an error, not a fact or a truth. Ordinary egocentric life alternates between pain and pleasure, suffering and happiness. Buddha understood that alteration, and called ordinary happiness "suffering of change", because of its instability. Well then who's right and who wrong? Are people right or Buddha? Is it merely a matter of opinion? It would be a matter of opinion if the perceptions of the two types of beings, egocentric beings and Buddhas, were equal. But a Buddha is supposed to have a superior perception, to see through illusion and discern reality. So an egocentric person who would consider a Buddha to be another type of egocentric person would certainly argue it was his word against Buddha's. But the Buddha argued that it is fact and not opinion. It is a fact that all this is suffering for an unenlightened person, and that this fact is only known by an enlightened person. The unenlightened are like Plato's cave-dwellers.
Only the one who has escaped and seen the sun knows that they are trapped in the shadows. To think this through for ourselves, we have to understand how the Buddha analyzed the ordinary person's misperception, to see if the analysis makes sense.