Carrying a large screen around has many drawbacks:
Weight/bulk – larger phones are heavier and don’t fit neatly into pockets.
Battery usage – driving a larger screen uses battery unnecessarily in scenarios that don’t require such a large display.
Cost – Larger screens are more expensive to produce, and to replace when they break.
Ease of use – Our thumbs simply don’t stretch that far!
Conversely, smaller screens also have their issues:
Limited space for information/graphical content.
Multi-tasking is effectively impossible.
Entertainment – smaller screens are very poor for watching movies or other content.