I’m not part of Companies House, just a forum user, but I believe you would be better trying to do this another way.
First - I believe there is a limit to the number of results you can get from advanced search and that is 10000 which hasn’t changed in some time and probably won’t. Example - see this thread (there are others I think - you can search this forum):
Second - Companies House has always stated the purpose of the Public Data API is not to “download all their data” but for more limited or focussed searches. If you do want large chunks of data there are a couple of approaches:
a) For basic Company Information you can use the Companies Bulk Data Product. This is only updated every month I believe (more information on this available from eBrian’s unofficial information site here).
b) If you are interested in changed / new data, it sounds like the best way would be to use the Streaming API service. This will alert you whenever there’s a change, you can choose specific data sets to monitor etc.
c) If you do want “all the data” then I believe the process is normally to download the current month’s bulk data, then keep updating that from the Streaming API until the new month’s data arrives, at which point you’re up to date (and you can just keep up to date using the Streaming API).
d) There are a couple of 3rd parties offering up-to-date data very conveniently as a service (possibly with additional search / filtering possibilities). You should be able to find them via searching this forum. Those services may be chargeable of course.
Good luck.