According to Plutarch, this conversation is taken from a lost dialogue ascribed to Aristotle, entitled, On the Soul. This passage also shows up in Nietzsche’s Birth of Tragedy chapter 3.

Plutarch Consolation to Apollonius [Moralia, 115a-c]

“There is also the saying you know has been passed around the mouth of many humans over the years.” “what is that?” he asked. The other one, interrupted, “that it is best of all not to exist and then second it is better to die than to live. This has been demonstrated by many examples from the divine.

For certainly they say this concerning Midas after the hunt when he caught Silenus and was asking him and finding out from him what is best for mortals and what should be most preferred. But Silenus was willing to say nothing, but remained stubbornly silent.

After he tried nearly every kind of approach, he persuaded him to provide some answer—so compelled, he said, “brief-lived offspring of a laboring god and harsh fate, why do you force me to tell you what it is better not to know? A life lived in ignorance of your most intimate griefs is the least painful.

But for humans it is not at all possible to have the best thing of all or to have any share of the best nature—since the best thing for all men and women is not to be born. But the second best thing after this and the first available to mortals, is to die as soon as possible after being born.” It is clear that he said this because the way that exists in death is better than the one in life.”

τὸ διὰ στόματος ὂν τοῖς ἀνθρώποις ὁρᾷς ὡς ἐκ πολλῶν ἐτῶν περιφέρεται θρυλούμενον.” “τί τοῦτ᾿;” ἔφη. κἀκεῖνος ὑπολαβών “ὡς ἄρα μὴ γενέσθαι μέν,” ἔφη, “ἄριστον πάντων, τὸ δὲ τεθνάναι τοῦ ζῆν ἐστι κρεῖττον. καὶ πολλοῖς οὕτω παρὰ τοῦ δαιμονίου μεμαρτύρηται. τοῦτο μὲν ἐκείνῳ τῷ Μίδᾳ λέγουσι δήπου μετὰ τὴν θήραν ὡς ἔλαβε τὸν Σειληνὸν διερωτῶντι καὶ πυνθανομένῳ τί ποτ᾿ ἐστὶ τὸ βέλτιστοντοῖς ἀνθρώποις καὶ τί τὸ πάντων αἱρετώτατον, τὸ μὲν πρῶτον οὐδὲν ἐθέλειν εἰπεῖν ἀλλὰ σιωπᾶν ἀρρήτως· ἐπειδὴ δέ ποτε μόγις πᾶσαν μηχανὴν μηχανώμενος προσηγάγετο φθέγξασθαί τι πρὸς αὐτόν, οὕτως ἀναγκαζόμενον εἰπεῖν, ‘δαίμονος ἐπιπόνου καὶ τύχης χαλεπῆς ἐφήμερον σπέρμα, τί με βιάζεσθε λέγειν ἃ ὑμῖν ἄρειον μὴ γνῶναι; μετ᾿ ἀγνοίας γὰρ τῶν οἰκείων κακῶν ἀλυπότατος ὁ βίος. ἀνθρώποις δὲ πάμπαν οὐκ ἔστι γενέσθαι τὸ πάντων ἄριστον οὐδὲ μετασχεῖν τῆς τοῦ βελτίστου φύσεως (ἄριστον γὰρ πᾶσι καὶ πάσαις τὸ μὴ γενέσθαι)· τὸ μέντοι μετὰ τοῦτο καὶ πρῶτον τῶν ἀνθρώπῳ ἀνυστῶν, δεύτερον δέ, τὸ γενομένους ἀποθανεῖν ὡς τάχιστα.’ δῆλον οὖν ὡς οὔσης κρείττονος τῆς ἐν τῷ τεθνάναι διαγωγῆς ἢ τῆς ἐν τῷ ζῆν, οὕτως ἀπεφήνατο.”

This comment seems proverbial–split in similar attributions in hexameter and elegiac poetry.

In the Contest of Homer and Hesiod

“Son of Meles, Homer who knows the mysteries of the gods,
Tell me foremost what is best for mortals?”
Homer answered:

“First, it is best for mortals to not be born.
If born, to pass through Hades’ gates as soon as possible.”

ἀρ υἱὲ Μέλητος ῞Ομηρε θεῶν ἄπο μήδεα εἰδὼς
εἴπ’ ἄγε μοι πάμπρωτα τί φέρτατόν ἐστι βροτοῖσιν;
῞Ομηρος·
ἀρχὴν μὲν μὴ φῦναι ἐπιχθονίοισιν ἄριστον,
φύντα δ’ ὅμως ὤκιστα πύλας ᾿Αίδαο περῆσαι.

The passage floats around some. Stobaeus (4.52.22) attributes it to Alcidamas’ Mousaion but the most widely cited source is Theognis. It is listed without attribution by the paroemiographer Michael Apostolos, with the explanation that this is a proverb “[attributed] to people living in misfortune”  (ἐπὶ τῶν δυστυχῶς βιούντων, 3.85.3)

Theognis, 425-428

“First, it is best for mortals to not be born.
Not to see the rays of the piercing sun
If born, to pass through Hades’ gates as soon as possible.
And to lie with a great pile of earth heaped above you.”

πάντων μὲν μὴ φῦναι ἐπιχθονίοισιν ἄριστον,
μηδ᾿ ἐσιδεῖν αὐγὰς ὀξέος ἠελίου,
φύντα δ᾿ ὅπως ὤκιστα πύλας Ἀΐδαο περῆσαι
καὶ κεῖσθαι πολλὴν γῆν ἐπαμησάμενον.

The Loeb note to this passage suggest that Theognis is merely adding to the hexameter lines, since the pentameter lines add nothing. But I think this is problematic. Consider the similar doublet to the first 2 lines above in Bacchylides.

Bacchylides 5.159–161

“And answering him, he said:
“It is best for mortals not to be born
Nor to see the sun.”

καί νιν ἀμειβόμενος
τᾶδ᾿ ἔφα· ‘θνατοῖσι μὴ φῦναι φέριστον
μηδ᾿ ἀελίου προσιδεῖν

Note how Bacchylides acknowledges the proverbial–or at least ‘other’–status of these lines by putting it into the mouths of one of his characters. Notice the stability of the infinitive construction μὴ φῦναι with the mobility of the dative θνατοῖσι and the lexical variations of θνατοῖσι instead of ἐπιχθονίοισιν and φέριστον instead of ἄριστον.

Sophocles, Oedipus Colonos 1225–1227

“Not being born conquers
every argument. But, then, if someone does emerge,
to return where you came from as fast as possible
is second best by far.”

Μὴ φῦναι τὸν ἅπαντα νι-
κᾷ λόγον· τὸ δ’, ἐπεὶ φανῇ,
βῆναι κεῖθεν ὅθεν περ ἥ-
κει, πολὺ δεύτερον, ὡς τάχιστα.

Sextus Empiricus (Outlines of Pyrrhonism 3.230–231) compares the Theognis passage to this fragment from Euripides (fr. 449)

“We should have a gathering to mourn
Someone when they are born, when they come to so many evils
And when someone has died and found a break from evils,
We should be happy and bless them as we carry them from their homes.”

ἐχρῆν γὰρ ἡμᾶς σύλλογον ποιουμένους
τὸν φύντα θρηνεῖν, εἰς ὅσ᾿ ἔρχεται κακά,
τὸν δ᾿ αὖ θανόντα καὶ κακῶν πεπαυμένον
χαίροντας εὐφημοῦντας ἐκπέμπειν δόμων.

Valerius Maximus claimed that Thracians actually did mourn births and celebrate funerals.

A clearer reflection on the proverb is Euripides fr. 235 (from Bellerophon):

“I agree with the thing reported everywhere,
That it is best for a mortal not to be born.”

ἐγὼ τὸ μὲν δὴ πανταχοῦ θρυλούμενον
κράτιστον εἶναι φημὶ μὴ φῦναι βροτῷ·

Note the different superlative at the beginning of the phrase and the singular βροτῷ. Based on the flexibility of the expression and the riffing on it, I would suggest that this is a broadly dispersed cultural idea that has proverbial status at a very early period. Note how Euripides, in another fragment, toys with the more broadly used phrase:

Euripides, fr. 908

“Not existing is better for mortals than being born.”

Τὸ μὴ γενέσθαι κρεῖσσον ἢ φῦναι βροτοῖς.

Epicurus (Diogenes Laertius, 10.127) thinks that anyone who believes this and says it is a fool since “if he says it because he believes it, how is it he does not just stop living? For this is ready for him to do, if it is completely believed by him.” (εἰ μὲν γὰρ πεποιθὼς τοῦτό φησι, πῶς οὐκ ἀπέρχεται τοῦ ζῆν; ἐν ἑτοίμῳ γὰρ αὐτῷ τοῦτ᾿ ἔστιν, εἴπερ ἦν βεβουλευμένον αὐτῷ βεβαίως).

And there is, of course, the Ancient Near Eastern context to consider!

"happier than either are those who have not yet come into being and have never witnessed the miseries that go on under the sun."– Ecc 4:3

וְטוֹב֙ מִשְּׁנֵיהֶ֔ם אֵ֥ת אֲשֶׁר־עֲדֶ֖ן לֹ֣א הָיָ֑ה אֲשֶׁ֤ר לֹֽא־רָאָה֙ אֶת־הַמַּעֲשֶׂ֣ה הָרָ֔ע אֲשֶׁ֥ר נַעֲשָׂ֖ה תַּ֥חַת הַשָּֽׁמֶשׁ https://t.co/OCn7BpMk1O

— Emily (Rabbi to be, recovering academic) (@youdontgettorun) May 22, 2019