Rules and Contradictions

If we are following a (complex) rule which leads us to a contradiction after some series of exercises in which it seems to work, then there must be another rule which avoids the contradiction but works for the successful cases. We might also revise our expectations about the successful cases, of course, but assuming that we can't do this for all successful cases, the conclusion still stands.

Does an argument like this make sense? Yes, but only so long as we hold on to some successful cases. We can't (intelligibly) deny them all (since a denial would depend on some successful rule following).

Does this mean that there are some reliable rules 'in the world' - even if we haven't discovered them yet?

Perhaps, in the sense that this may be a methodologically harmless way of talking - but no, if it is construed as an argument for a metaphysical position.  There are two reasons for this:

(1) For Kripkean/Goodmanesque reasons we know that there will be an indefinite number of potential rules which we might 'discover' here.  A rule which avoids present anomalies and preserves predictive power may produce future anomalies.  Even one which doesn't may not be a unique alternative.  It seems absurd to say that some indefinite set of potential rules is 'really there' in the world.

(2) We are arguing from the a priori (because undeniable) possibility of talk to the necessity of undiscovered rules.  If our talk breaks down, then statements like 'there are undiscovered rules' will also evaporate.