While minor in the scheme of things, that scene was by no means pointless.
The information from the old man provided the perfect cover story for those 3, originating from a village in which almost everyone was killed, located at a place not marked on maps.
And with the old man dead, no one remained to refute their cover story, should anyone ever question them.
Which makes many wonder: was the old man killed by those 3, so there'll be no witnesses left?
(In the aftermath of Annie's capture, Hanji did indeed go back to investigate her "identity", which traced back to Reiner and Bertholdt, on the basis of coming from the same reported village)
Furthermore, go back to Chapter 16. When Armin asked Reiner and Bertholdt about their backgrounds, Bertholdt's dialogue that follows was practically a recital of the old man's account. Reiner was even sweating at the way Bertholdt used that story.
(I'm amused that even back then, Armin was unwittingly the bane of the trio's existence)
Just read the chapter on mangastream and I made that same leap in logic since I don't get why he'd kill himself after "apparently" finding out that he isn't the last survivor of his village. It's certainly open ended enough that it can be interpreted either way.