Recently i keep getting symbols stuck in post-processing in javascript and script error 0 popup. Since i 'm download many symbols I get hundreds or thousands of these popups.
It seems retry after waiting a few minutes the download is successful. AFter lcicking OK in the popup:
This error is caused by users changing the settings, trying to download TOO FAST and exceeding limits of Yahoo.
It never happens in normal situation, only when users go way overboard, go the SETTINGS and change them to absurd values and try to use 10 simultaneous downloads. Don't do that.
You must NOT push Yahoo too much.
Instead, keep the DEFAULT SETTINGS and default are like this:
You guys need to realize that Yahoo Finance is a public web page that is used by many people and bombarding it with thousands of requests per minute is considered abuse (denial of service attack) and they will BAN YOU if you abuse their service. Then you will cry that "AmiQuote stopped working". But YOU DID it. Don't change default settings. Defaults just work. Learn to use the service reasonably.
Thanks using defaults worked. It just takes many hours to update 10k quotes every day. I do analysis after market close and I download the data through yahoo. I'm currently trying 5 simultaneous downloads and it seems to work ok for now. Future workaround may be that I buy an account at polygon.io. Thanks for your suggestion!
No it definitely doesn't take "many hours" to update 10k symbols. It doesn't take even a single hour.
You can now use AmiQuote 4.17 as it has better way to prevent "too many requests" error from Yahoo.
According to my checks 10K symbols can easily be downloaded in JUST 16 MINUTES using AmiQuote 4.17
It does take hours for the default recommend setting because the default setting has 1000ms delay set between requests single threaded. Assuming each request is instant and is only slowed down by the preset delay between requests that equates to 2.78hrs for 10k symbols to complete.
When I use multi threading it becomes much faster with 5 simultaneous downloads at a time (1/5th the time). When i used 10 at a time it ran into over use issues. This is for 4.16 I haven't tried 10 at a time with 4.17 yet.
AmiQuote 4.16 is NOT SUPPORTED anymore.
The fail safe default is the slowest setting. It is still faster than human would ever be able download manually. It is there because people tend to use absurd values, so baseline must be set low. If somebody is reasonable he/she can change the settings to some less conservative. Yahoo service is free to put/change their limits at any time. That is why settings are accessible at all. I could just REMOVE all settings and FORBID users from changing anything, just to make sure that no absurd values are entered so I don't hear any complaints that suddenly things don't work because of too aggressive settings. But no... I make the software flexible and it is my mistake because I would sleep much better if software did not allow any flexibility.
Even with just one thread it can download 230 symbols per minute (with initial delay set to 50ms). So even with just one thread, which is conservative, the download time for 10K should be something like 44 minutes.
Sorry, but no matter how hard you would try you won't be able to download and import 230 symbols per minute manually. Not even 1/10 of that. So I guess you should appreciate that AmiQuote is working fast even with ONE thread.
Interestingly you did not mention the settings at all in your original post nor the fact that you have changed from 1 simultaneous download to 10. And you knew very well that you changed the settings and after that you ran into trouble but you did not say a word. Hiding information makes answering much harder if not impossible. In your best interest all information should be provided. So in the future please always provide all information especially about all settings that you changed yourself.
