Private Languages

Maybe this is the Private Language Argument:

Suppose I told you 'I have a private language, which I can use when I am thinking things to myself'.  You might ask 'What do you say to yourself, in your private language?'

I could answer in two ways:

(1) I could give you  a translation, in our shared public language, of what I say to myself.
(2) I could say that my private language was not translatable into a public language.

I could not, of course, simply give you a 'raw' private utterance, with or without a translation.  This would be to render the private language public.

(1) Is what we presently have.  If I tell you what I am thinking, I am doing as much as this - but this does not demonstrate that I have a private language:  It only demonstrates that I can use our public one.

(If we could conclude from our use of a public language that we 'must' have some private 'language of thought', then what would we have to conclude from our use of this private language?  That we must have an even more private language of proto-thought?  What kind of explanation could this possibly provide?)

If I want to demonstrate the possibility of a private language by an argument produced in our public language, I would need to have a way of pointing at the private language in a way which showed (a) that it was a language and (b) that it was not a public language.  But I can't point to it if it's private, and I can only show that it's a language if it can be spoken.

With respect to (2), the idea of a language which is in principle untranslatable is  (obviously) incoherent for Davidsonian reasons.  How could we know it was a language?